You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Esteve Fernandez (JIRA)" <ji...@apache.org> on 2009/02/04 09:25:59 UTC
[jira] Created: (THRIFT-311) ASIO client & server
ASIO client & server
--------------------
Key: THRIFT-311
URL: https://issues.apache.org/jira/browse/THRIFT-311
Project: Thrift
Issue Type: New Feature
Components: Library (C++)
Reporter: Esteve Fernandez
Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Erik Frey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671685#action_12671685 ]
Erik Frey commented on THRIFT-311:
----------------------------------
Very promising timings!
Regarding coroutines, you could model a TSocket class so that it behaves synchronously with coroutines:
http://www.crystalclearsoftware.com/soc/coroutine/coroutine/asio.html
This would require no changes to the compiler. If I can wrangle up some free time, I'll post an example.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Comment: was deleted
(was: Updated patches for Thrift MSVC Port
27 August 2009
There's nothing like taking your code out into the real world to find out
what you screwed up.
These patches (for 2 different SVN revisions) do the following:
* Fixes a bug in all flavors of TAsioServerSocket that made the interrupt()
method do nothing. The result was that calling stop() on a TServer whose
underlying socket used the TAsio* implementation did nothing and the server
thread would not be stopped. On Windows, the named pipe connect call was made
to be interruptible.
* In order to test that you really can stop the servers, I changed stress-test
so that it stops the server it has running when the test has completed
successfully, and waits for the thread to join. This means that we know
the server object's address and remember it, and that we specifically
make the server thread joinable when it is created.
* On Windows, added the capability to select static runtime linking for the
libraries and executables. To use this configuration option, your Boost
and Zlib libraries must be linked the same way. The default linkage is still
to the DLL runtime libraries. Run "cscript configure.js --help" for
usage information. If you select this option, the default Boost library name
paths are adjusted automatically. Since the zlib project does not support
this type of linkage out of the box, the same default patterns will be used
for either type of runtime linking. See the next item if you need to handle
modified file names for zlib libraries.
* On Windows, changed the configure script so that it accepts configuration-
specific library name search patterns for both Boost and Zlib. This lets you
specify things like:
--with-zlib-lib-name-pattern[win32][static][debug]=zlib-Win32-sd\.lib
and that pattern will only be used to search for the zlib library for
the configuration tuple (win32, static, debug). Of course, if you are
doing something nonstandard (as we are where I work), then you probably
need to specify the pattern for every tuple of interest. But at least
you can.
* On Windows, I put dummy versions of the property sheet files the configure
script generates into the configGenerated directory. This lets you open the
Visual Studio projects even if you haven't run the configure script. The
main reason is that you might need to upgrade all of the VS projects to
be compatible with a newer version of Visual Studio, and it's easier if
you can just open them without needing to configure first.
If anyone is playing with the patch, I apologize for the server bug. I had
halfway implemented it, but then never tested it. And if anyone wanted to
play on Windows but was thwarted by static runtime linking or nonstandard
library names, I apologize for the delay in supporting them.
I have a sneaking suspicion that there might be a need for some way
to customize the VS project settings via user-defined property sheets. (Well,
there might be this requirement if anyone but me ever uses this stuff.)
I didn't need them, so didn't put in the work required to add the capability.
It would require a configure command line parameter something like
--prop-sheet=myPropSheet.vsprops
and the configure script would write a property sheet that inherits from
that one, and all of the Visual Studio projects would inherit the property
sheet that we write. If there are no --prop-sheet arguments, then we would
write a property sheet file that doesn't do anything. Not particularly hard,
just tedious and time consuming to test.
There are 2 patch files:
* The first is a new patch that is to be applied to Subversion rev 803313.
It is called:
ThriftMsvcPatchForSvnRev803313_Rev1.txt.zip
(yes, I remembered to compress it this time) and it supersedes the previous
patch file called:
ThriftMsvcPatchForSvnRev803313.txt
(i.e. just apply this patch directly to a checkout of 803313, NOT to
a checkout that you already patched.)
* The second is made against a newer HEAD rev of the repository. It is
made to be applied to a checkout of Subversion rev 808166, and is called:
ThriftMsvcPatchForSvnRev808166.txt.zip
Apply the patches with -p8, as before.)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Nitay Joffe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726199#action_12726199 ]
Nitay Joffe commented on THRIFT-311:
------------------------------------
It's been a while... what's the status of this and the related porting to Windows work?
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12670429#action_12670429 ]
Esteve Fernandez commented on THRIFT-311:
-----------------------------------------
I used 0.3.8, because that's the version that Ubuntu Hardy shipped, but I tested it with 1.3.1 as well.
I didn't use newer fancy stuff like async_read_until, match conditions, etc., though. But I don't think we need that, and it would make it incompatible with older versions of ASIO (such as 0.3.8).
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726500#action_12726500 ]
Rush Manbert commented on THRIFT-311:
-------------------------------------
I've got what I believe is a complete port to Asio and Boost::Thread running on my Mac and PC. I developed the code by making standalone projects for the two stress tests and the concurrency test that just included all the library source along with the test code. That made it easy to work on and debug, but meant that I would need to bring it all back into the distribution tree eventually. I am working on that now for the *nix distro. I think I've got autoconf figured out, but the new config parameters allow for 8 build variations (excluding debug/release), so it's a lot to test.
Once that is done, I need to check out the head rev from the repository and merge my changes into it on my Mac, then move the merged tree to my Windows machine, create the library and test Visual Studio projects (Again, on the Windows machine I have a single VS solution that builds the two stress tests and the concurrency test, using sources that I would occasionally bring over from the Mac.), and write the configuration script for Windows. Then test it. Sigh. But there are only 2 variants on Windows, so there's not so much to test.
When all of that's finished and tested I'm going to talk to David and Mark about how to 1) Make it available for others to test, and 2) Get it accepted back into the Thrift distribution.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: thrift_main.cpp
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Attachment: (was: ThriftMsvcPatchForSvnRev808166.txt.zip)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Comment: was deleted
(was: I realized that I had forgotten to delete aclocal/pkg.m4 before I generated the patch file. So if you apply the patch, a pkg.m4 file will be added to your aclocal subdirectory. If you're on a Mac, you can probably just leave it there. If you're on a different platform, you probably want to delete it.
Sorry about that.)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Norman Casagrande (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12670302#action_12670302 ]
Norman Casagrande commented on THRIFT-311:
------------------------------------------
That's a nice start!
You seem to use the independent version of asio. Which version is it? Why not using the one included in boost since 1.35?
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Issue Type: Sub-task (was: New Feature)
Parent: THRIFT-1
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Norman Casagrande (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12670554#action_12670554 ]
Norman Casagrande commented on THRIFT-311:
------------------------------------------
I would not use 0.38. It is a beta and the API has changed quite a bit since version 1.0 which was added in boost 1.35.
If you want to play with the newer boost, there are the debian packages for 1.35 available for ubuntu (that's what I am using right now).
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: thrift_server.hpp
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (THRIFT-311) ASIO client & server
Posted by Rush Manbert <ru...@manbert.com>.
On Sep 18, 2009, at 11:09 AM, Esteve Fernandez wrote:
> Hi
>
> On Friday 18 September 2009 19:57:41 Rush Manbert wrote:
>> Yes, I will. Sorry, I didn't mean to hijack the JIRA issue with my
>> stuff.
>
> No worries, it's just that your code seems to be aimed at having a
> working
> Windows port (which is great, because we don't support that platform
> yet),
> not specifically at making an ASIO-enabled version, although it
> coincidentally does :-)
Yes, that's all I was going for.
>
> Creating a new ticket with a more specific title will help other
> users find
> your code and contribute to it.
I agree. It'll take me a couple days to get things together. I
actually brought the library into our Windows development environment
and got bitten on the butt by the stdint header. I chose one
implementation, but someone else here had selected a different one for
our Windows libraries to use, and they don't mix. (And, of course,
Boost brings in its own definitions within the boost namespace. Had to
work around that already.) I need to make that configurable for the
Windows build. Sigh... More permutations to test.
>
> In any case, sorry if I sounded rude, I didn't mean to offend you.
Not at all. No offense taken. My apology to you was sincere. I hadn't
realized that the intent of 311 was to have a fully async
implementation.
Best regards,
Rush
Re: [jira] Commented: (THRIFT-311) ASIO client & server
Posted by Esteve Fernandez <es...@sindominio.net>.
Hi
On Friday 18 September 2009 19:57:41 Rush Manbert wrote:
> Yes, I will. Sorry, I didn't mean to hijack the JIRA issue with my
> stuff.
No worries, it's just that your code seems to be aimed at having a working
Windows port (which is great, because we don't support that platform yet),
not specifically at making an ASIO-enabled version, although it
coincidentally does :-)
Creating a new ticket with a more specific title will help other users find
your code and contribute to it.
In any case, sorry if I sounded rude, I didn't mean to offend you.
Cheers.
Re: [jira] Commented: (THRIFT-311) ASIO client & server
Posted by Rush Manbert <ru...@manbert.com>.
On Sep 18, 2009, at 10:44 AM, Esteve Fernandez (JIRA) wrote:
<snip>
>
>
>> It looks like for my application, I may need to take a similar
>> approach to Erik Bernhardsson's.
>> He posted some patches to the Thrift lists, however, they haven't
>> shown up on JIRA.
>>
>> What I may do next is: extract that, play with it on my desktop a
>> bit, and open a separate
>> JIRA sub-task for THRIFT-1 in parallel with this one.
>
> As I said in THRIFT-579, I'm not sure about the legal implications
> of posting someone else's code.
>
> In any case, the original intent for this ticket was already to
> build an asynchronous version of the C++ generator. Rush, could you
> file a new ticket (MSVC port, e.g.) and attach your code?
Yes, I will. Sorry, I didn't mean to hijack the JIRA issue with my
stuff.
- Rush
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12757286#action_12757286 ]
Esteve Fernandez commented on THRIFT-311:
-----------------------------------------
> It appears as of this writing that Boost.Coroutine is still sitting in the Boostpro vault; it is not
> yet integrated with mainline Boost.
>
> There has been no major list traffic about it since April, and it seems to have some
> performance and scalability issues on Windows.
> The original author mentions in the documentation that there are issues with GCC's
> implementation of thread-local storage as well.
Coroutines are just a convenience for using futures without having to write tons of callback functions (see Twisted's inlineCallbacks).
> Boost.Future is not integrated yet either. Futures look syntactically nice, but tricky to work
> into an explicitly coroutined I/O model without additional support from something like CoCo or
> Boost.Coroutine; kind of rules them out for shipping product.
Sorry, but I fail to see how you would implement a fully asynchronous version of Thrift, without futures or threads. I don't understand why you're dropping futures, the Twisted generator uses Deferreds (similar to futures) but doesn't use coroutines, and it's fully asynchronous.
> It looks like for my application, I may need to take a similar approach to Erik Bernhardsson's.
> He posted some patches to the Thrift lists, however, they haven't shown up on JIRA.
>
> What I may do next is: extract that, play with it on my desktop a bit, and open a separate
> JIRA sub-task for THRIFT-1 in parallel with this one.
As I said in THRIFT-579, I'm not sure about the legal implications of posting someone else's code.
In any case, the original intent for this ticket was already to build an asynchronous version of the C++ generator. Rush, could you file a new ticket (MSVC port, e.g.) and attach your code?
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: MsvcPatchSupportScripts.zip, thrift-808166-Patched.zip, thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp, ThriftMsvcPatchForSvnRev803313.txt, ThriftMsvcPatchForSvnRev803313_Rev1.txt.zip, ThriftMsvcPatchForSvnRev808166.txt.zip
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671319#action_12671319 ]
Esteve Fernandez commented on THRIFT-311:
-----------------------------------------
Norman: you're right that 0.3.8 is probably too old. But I was able to compile the Thrift ASIO server against ASIO 1.3.1 without changes. Anyway, I didn't use any newer features, but I wouldn't argue against using more a more recent version, if it brings a backwards-incompatible feature that we find useful.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: thrift_connection.cpp
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671683#action_12671683 ]
Esteve Fernandez commented on THRIFT-311:
-----------------------------------------
It's just the product of a Sunday morning :-) so I haven't done any serious benchmarking and I'm sure there's still room for improvement.
Using the same client script and the same number of worker threads (10, although it doesn't really matter, as I only used one connection), this is what I got after 555000 calls (using the same connection):
TNonblockingServer ~5080 calls/sec
ThriftASIOThreadPoolServer ~5250 calls/sec
I still have to benchmark how many concurrent connections/sec I can achieve. Anyway, I don't have much experience with TNonblockingServer and used it "as is", so I'd happy to know if there's anything I could do to tune it :-)
BTW, I'd like to see a coroutine Thrift server too :-) But it would require changes to the Thrift compiler, just like THRIFT-148
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Bruce Simpson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754652#action_12754652 ]
Bruce Simpson commented on THRIFT-311:
--------------------------------------
I like the intent behind this change, but I'm concerned that it may be a lot for the Thrift guys to digest at once.
I'm running 'conservative', because it seems that moving to ASIO for the project I'm working on may be too much to take on just right now.
To constrain the scope of the changes, it might be an idea to limit the changes to *only* the TAsioNonblockingServer, and any changes needed inside the concurrency/ part of the library. ASIO need not imply dependency on Boost.Thread.
I suspect that for what Rush is trying to do for his client, Boost.Thread is needed. Building one's own rwlocks or condvars on Win32 is generally not advisable if you like quick satisfaction...
Having said all that, this work is an excellent starting point.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: MsvcPatchSupportScripts.zip, thrift-808166-Patched.zip, thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp, ThriftMsvcPatchForSvnRev803313.txt, ThriftMsvcPatchForSvnRev803313_Rev1.txt.zip, ThriftMsvcPatchForSvnRev808166.txt.zip
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Attachment: (was: ThriftMsvcPatchForSvnRev803313.txt)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (THRIFT-311) ASIO client & server
Posted by "Erik Frey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671645#action_12671645 ]
erikfrey edited comment on THRIFT-311 at 2/8/09 9:01 AM:
----------------------------------------------------------
I think a thread pool that does i/o and runs the processor (your current design) is the best/most asio-y way to go. If you make a separate thread pool to process requests, you're just shifting from a queue that backs up on i/o to a queue that backs up on tasks.
I would love to see a coroutine thrift server, if just to see the raw throughput it could pump out :) There's a boost vault project called boost.coroutine ( http://www.crystalclearsoftware.com/soc/coroutine/index.html ) that plays pretty nicely with asio. If used properly, you could probably write a coroutine socket class that behaved like a blocking socket, but used coroutine under the hood to yield for any i/o.
My main concern with coroutines is how nicely they would play with concurrency in the user's processor code. Usual mutex/thread type stuff would not work!
Have you tried benchmarking this asio server against TNonBlockingServer? I'm curious how they compare, particularly on multi-core.
was (Author: erikfrey):
I think a thread pool that does i/o and runs the processor (your current design) is the best/most asio-y way to go. If you make a separate thread pool to process requests, you're just shifting from a queue that backs up on i/o to a queue that backs up on tasks.
I would love to see a coroutine thrift server, if just to see the raw throughput it could pump out :) There's a boost vault project called boost.coroutine ( http://www.crystalclearsoftware.com/soc/coroutine/index.html ) that plays pretty nicely with asio. If used properly, you could probably write a coroutine socket class that behaved like a blocking socket, but used coroutine under the hood to yield for any i/o.
My main concern with coroutines is how nicely they would play with concurrency in the user's processor code. Usual mutex/thread type stuff would not work!
Have you tried benchmarking against TNonBlockingServer? I'm curious how they compare, particularly on multi-core.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: thrift_handler.hpp
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: ThriftCalculatorASIOServer.cpp
Simple ASIO TCP server that implements the Calculator service.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: thrift_handler.cpp
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760323#action_12760323 ]
Rush Manbert edited comment on THRIFT-311 at 9/28/09 11:17 AM:
---------------------------------------------------------------
Last comment on this issue, I promise!
I have deleted most of my comments and all files related to my Windows port of the C++ library. Please see JIRA-591 if you;'re interested.
was (Author: rush):
Last comment on this issue, I promise!
I have deleted all of my comments and files related to my Windows port of the C++ library. Please see JIRA-591 if you;'re interested.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744094#action_12744094 ]
Rush Manbert commented on THRIFT-311:
-------------------------------------
I realized that I had forgotten to delete aclocal/pkg.m4 before I generated the patch file. So if you apply the patch, a pkg.m4 file will be added to your aclocal subdirectory. If you're on a Mac, you can probably just leave it there. If you're on a different platform, you probably want to delete it.
Sorry about that.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: MsvcPatchSupportScripts.zip, thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp, ThriftMsvcPatchForSvnRev803313.txt
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Comment: was deleted
(was: I decided that I should also post a zip of the thrift distribution with my patch applied. That way, Windows people who don't have access to a machine to apply the patch can try it out. The file is:
thrift-808166-Patched.zip
Windows users should be able to download, unzip, configure, and build.)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733671#action_12733671 ]
Rush Manbert commented on THRIFT-311:
-------------------------------------
Thought I'd give an update on my Asio/Visual Studio work.
I gave a patch to Bruce Simpson, because he was interested in the Asio version for use in XORP. He has tested on Gentoo Linux and FreeBSD and provided invaluable feedback.
I took that code version back to Windows and have created and tested all of the Visual Studio projects to build the libraries, test programs, and tutorial. Sadly, they rely on generated code that I copied out of the Mac tree after building there, but someone will get the Visual Studio-built or Java version of the compiler running on Windows one of these days. (But it won't be me ;-)
I still need to write the configure script for Windows, and put together some sort of install mechanism for the libraries. I have a couple of people interested in testing this code, so I'll enlist them at that time. My eventual plan is to provide a single patch and try to lobby for a committer to apply it and make a branch from the result that people could check out, test, and fix as necessary. The problem I see is that my one patch will address all the Jira issues I have opened, plus provide other fixes for which there are no Jira issues. But it's not separable that way, so I don't know what else to do.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: thrift_connection_v2.cpp
Fixes double free errors, I forgot to remove calls to delete. Now that it uses shared_from_this, it's no longer needed.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: (was: ThriftCalculatorASIOServer.cpp)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726550#action_12726550 ]
Rush Manbert commented on THRIFT-311:
-------------------------------------
Sorry, what I should have said in the first line of that last comment was:
I've got what I believe is a complete port OF THE C++ RUNTIME LIBRARY to Asio and Boost::Thread running on my Mac and PC.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Comment: was deleted
(was: The patch file and the support scripts archive.)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Comment: was deleted
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733675#action_12733675 ]
Esteve Fernandez commented on THRIFT-311:
-----------------------------------------
It sounds great, Rush. Could you upload your patch to JIRA or create a repository somewhere? (Launchpad, Bitbucket, Github, etc.) It's far easier to track these changes and you'd probably receive a lot of feedback, instead of having to send your patch privately to different recipients. Also, we wouldn't duplicate efforts: Erik and Matthias' patch, your patch, the files in this ticket, etc.
BTW, I have two questions. Does it follow Thrift's interface or ASIO's? Is it asynchronous (as in Twisted) or uses ASIO synchronously?
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Bruce Simpson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754654#action_12754654 ]
Bruce Simpson commented on THRIFT-311:
--------------------------------------
It appears as of this writing that Boost.Coroutine is still sitting in the Boostpro vault; it is not yet integrated with mainline Boost.
There has been no major list traffic about it since April, and it seems to have some performance and scalability issues on Windows.
The original author mentions in the documentation that there are issues with GCC's implementation of thread-local storage as well.
Boost.Future is not integrated yet either. Futures look syntactically nice, but tricky to work into an explicitly coroutined I/O model without additional support from something like CoCo or Boost.Coroutine; kind of rules them out for shipping product.
It looks like for my application, I may need to take a similar approach to Erik Bernhardsson's. He posted some patches to the Thrift lists, however, they haven't shown up on JIRA.
What I may do next is: extract that, play with it on my desktop a bit, and open a separate JIRA sub-task for THRIFT-1 in parallel with this one.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: MsvcPatchSupportScripts.zip, thrift-808166-Patched.zip, thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp, ThriftMsvcPatchForSvnRev803313.txt, ThriftMsvcPatchForSvnRev803313_Rev1.txt.zip, ThriftMsvcPatchForSvnRev808166.txt.zip
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Attachment: thrift-808166-Patched.zip
I decided that I should also post a zip of the thrift distribution with my patch applied. That way, Windows people who don't have access to a machine to apply the patch can try it out. The file is:
thrift-808166-Patched.zip
Windows users should be able to download, unzip, configure, and build.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: MsvcPatchSupportScripts.zip, thrift-808166-Patched.zip, thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp, ThriftMsvcPatchForSvnRev803313.txt, ThriftMsvcPatchForSvnRev803313_Rev1.txt.zip, ThriftMsvcPatchForSvnRev808166.txt.zip
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671583#action_12671583 ]
Esteve Fernandez commented on THRIFT-311:
-----------------------------------------
Here's an updated version of a Thrift+ASIO server. It uses a single thread for accepting connections and dispatches them to a configurable thread pool. You'd need to make your handlers thread-safe, though.
I have some ideas for making servers truly asynchronous, we could use io_service.post(...) for dispatching processor calls to a thread pool or add support for coroutines in the Thrift compiler. The former is really easy to accomplish, but might put too much work on the thread pool and hinder servers scalability. Whereas the latter requires modifying the Thrift compiler, but would probably make servers have better performance (this is the approach used in THRIFT-148)
BTW, I've tested it in Linux, but not in Windows. However I added the specific bits to make it compile in Windows (copied from the ASIO examples), so it would be great if someone could test it :-)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Erik Frey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671645#action_12671645 ]
Erik Frey commented on THRIFT-311:
----------------------------------
I think a thread pool that does i/o and runs the processor (your current design) is the best/most asio-y way to go. If you make a separate thread pool to process requests, you're just shifting from a queue that backs up on i/o to a queue that backs up on tasks.
I would love to see a coroutine thrift server, if just to see the raw throughput it could pump out :) There's a boost vault project called <a href="http://www.crystalclearsoftware.com/soc/coroutine/index.html">boost.coroutine</a> that plays pretty nicely with asio. If used properly, you could probably write a coroutine socket class that behaved like a blocking socket, but used coroutine under the hood to yield for any i/o.
My main concern with coroutines is how nicely they would play with concurrency in the user's processor code. Usual mutex/thread type stuff would not work!
Have you tried benchmarking against TNonBlockingServer? I'm curious how they compare, particularly on multi-core.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: thrift_connection.hpp
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Comment: was deleted
(was: It has been a long time coming, but I believe that the MSVC port is finally ready to post and let other people test.
The patch file was generated against the SVN repository revision 803313. It patches the entire distribution tree, but only touches things that are C++ related.
The patch addresses the following unresolved Jira issues:
Thrift-456 (https://issues.apache.org/jira/browse/THRIFT-456)
Thrift-457 (https://issues.apache.org/jira/browse/THRIFT-457)
Thrift-487 (https://issues.apache.org/jira/browse/THRIFT-487)
Thrift-488 (https://issues.apache.org/jira/browse/THRIFT-488)
It also makes changes that are not issues in Jira. For instance:
1. Autoconf support has been extended to all of the C++ tests, as well as the tutorial.
2. The stress-test and stress-test-nb have been extensively reworked to make them more versatile and reliable. I relied on these a lot during my development to test the new socket and server code.
3. The concurrency test has been extensively reworked to make it be reliable and more versatile. I work on a 8 core machine and the test as distributed just doesn't work reliably in that environment. I used this test to verify all of my Boost threading changes.
4. All of the sources have been touched to add Windows-specific support, but I suppose that could be seen as part of this Jira issue. :-)
Disclaimers:
-----------
I had never done anything with autoconf before this. What I have done may not be correct. If changes are required, please let me know.
The configure script displays information at the end that is related to the new configuration stuff. It was useful for me while developing and I thought it might be a good idea to leave it in for a while.
I'm really not a Windows guy, I just do what needs doing. If any of you are Windows experts, and you have suggestions or corrections, please let me know.
See the "Known Issues" section in the msvc/README file.
Licensing
Obviously, I checked the box that says I grant the license to Apache for this code. There is one file, though, that already had a license header within it. That is the file lib/cpp/src/VisualStudioStdint.h, and it contains this notice:
// ISO C9x compliant stdint.h for Miscrosoft Visual Studio
// Based on ISO/IEC 9899:TC2 Committee draft (May 6, 2005) WG14/N1124
//
// Copyright (c) 2006 Alexander Chemeris
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
//
// 3. The name of the author may be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
// WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
// EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
// OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
// WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
// OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
// ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
///////////////////////////////////////////////////////////////////////////////
I did not add an Apache license header to this file. I do not know what should be done. It can obviously be used, but there must be some requirement for dual licensing.
To apply the patch, do the following:
------------------------------------
1. Checkout from SVN:
svn co --revision 803313 http://svn.apache.org/repos/asf/incubator/thrift/trunk thrift
Note the --revision argument. The patch might work against other revisions, or it might not.
2. Make a copy of your distribution directory, because patch will patch the directory it works on in place.
3. Get the patch file from Jira by following this link:
https://issues.apache.org/jira/browse/THRIFT-311
Use the links there to download the patch file.
4. cd to the renamed distribution directory
5. Apply the patch by execution:
patch -p 8 <patchFile
where patchFile is the path to your downloaded patch file.
After applying the patch, the top level README file will have information about the new "msvc" subdirectory. The new file CPP_LIB_CFG has information about the configurations that are possible on *nix systems. Configurations can keep things the way they are currently, or can add-to and/or replace existing functionality with new functionality provided by Asio/Boost. (I had to do it this way so that I could use the existing test code with my new classes.)
The "msvc" subdirectory has its own README file that contains information particular to the msvc port. There are other msvc subdirectories spread through the tree. They follow a pattern, and they are all documented in the new README.
Tested Platforms:
----------------
Mac OS X 10.5.7 (me)
Gentoo Linux (Bruce Simpson)
FreeBSD (Bruce Simpson)
Windows XP (me)
Tested Boost and Asio versions:
------------------------------
Standalone Asio 1.2.0 and 1.4.1
On the Mac: Boost 1.36.0, 1.37.0, 1.38.0, and 1.39.0
On Windows XP: Boost 1.36.0
Along with the patch, I will attach a zip file called "MsvcPatchSupportScripts.zip" that contains a number of scripts that are useful for testing all of the variations (there are 10) on a *nix system. These scripts assume a very specific directory structure. It is documented in only one of them. Sorry. If you run "./buildAll.sh --help" at the command line, it will show you what is expected. It also documents two environment variables you can use with the scripts. They are called THRIFT_DEV_ROOT and THRIFT_DISTRO_DIR. Note that the scripts are designed to be run from the directory ABOVE your patched distribution directory.
The various scripts are:
buildAll.sh - Builds all the possible configuration variants by copying the distribution directory, then configuring and building. If you are on a Mac, be sure to copy your pkg.m4 file into the aclocal subdirectory of the distribution tree before you run this script. The build results are written into files called Log_* in the same directory as the script.
buildTutorialAll.sh - After buildAll.sh has run, this script iterates over all of the directories that were created and verifies that the C++ tutorial can be built both using the auto-generated Makefile and the Tutorial.mk makefile. It cleans up after itself.
clean.sh - Finds and removes all of the subdirectory trees and log files that are written by buildAll.sh.
concurrencyTest.sh - After buildAll.sh has run, this script iterates over all the directories that were created, runs the concurrency test in each of them and verifies that the test runs successfully.
cpCfgAndBuild.sh - This script knows how to configure and build Thrift in a single directory, but all it cares about is the C++ stuff. It also runs "make check". This script is used by the buildAll.sh script. This script currently defines the macro THRIFT_ENABLE_CONFIG_WARNINGS via CPPFLAGS. That generates warning messages at library compile time that tell you about the configuration. Remove the macro definition from the script if you don't want the warnings. (Just search for the macro name in the script file. What to do is pretty obvious once you find it.)
doTest.sh - This script runs the standard "make check" test suite on a single directory. It is used by the testAll.sh script.
findThriftDistro.sh - This script attempts to locate the Thrift distribution directory. It looks at the directory passed as the script argument if there is one, or it examines the environment variables THRIFT_DEV_ROOT and THRIFT_DISTRO_DIR if they are set. This script is used by many of the others, and you shouldn't need to mess with it.
testAll.sh - After buildAll.sh has run, this script iterates over all of the directories that were created and runs the standard test suite in each of them.
I hope others find this to be useful.)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733707#action_12733707 ]
Rush Manbert commented on THRIFT-311:
-------------------------------------
I'll see what I can do about uploading a patch. I need to bring the Windows code back and merge to the Mac in order to generate the patch, and I've got a lot of meetings to attend during the next few days. The patch would have all the code changes and the VS projects implemented, but not the configure script. But you guys are all smart and should be able to deal with configuring the Windows stuff by hand. It just requires editing a couple of property sheet files to change paths.
As far as the interface, it exactly follows the Thrift interface. I have made workalike parallel Asio-based classes (TAsioSocket, etc.) that act just like the Thrift classes. For threading, I added Boost.thread as a configure option on *nix. The only newness is that I added local sockets (windows named pipes or Unix domain sockets) to the Asio-based classes. That's because the company I work for needs them. So all the classes I have added are synchronous except for TAsioNonblockingServer and TAsioLocalNonblockingServer.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671914#action_12671914 ]
Esteve Fernandez commented on THRIFT-311:
-----------------------------------------
It looks pretty nice, but I think the compiler should be modified in order to support asynchronous processor calls, which should improve performance a bit. THRIFT-148 generates process methods that return Deferreds (futures) which let's you write handler methods that play nicely with the reactor.
BTW, using a simple asynchronous Python Twisted client, I get ~6500 calls/sec against the ASIO server, which seems pretty fast to me :-)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742928#action_12742928 ]
Rush Manbert commented on THRIFT-311:
-------------------------------------
It has been a long time coming, but I believe that the MSVC port is finally ready to post and let other people test.
The patch file was generated against the SVN repository revision 803313. It patches the entire distribution tree, but only touches things that are C++ related.
The patch addresses the following unresolved Jira issues:
Thrift-456 (https://issues.apache.org/jira/browse/THRIFT-456)
Thrift-457 (https://issues.apache.org/jira/browse/THRIFT-457)
Thrift-487 (https://issues.apache.org/jira/browse/THRIFT-487)
Thrift-488 (https://issues.apache.org/jira/browse/THRIFT-488)
It also makes changes that are not issues in Jira. For instance:
1. Autoconf support has been extended to all of the C++ tests, as well as the tutorial.
2. The stress-test and stress-test-nb have been extensively reworked to make them more versatile and reliable. I relied on these a lot during my development to test the new socket and server code.
3. The concurrency test has been extensively reworked to make it be reliable and more versatile. I work on a 8 core machine and the test as distributed just doesn't work reliably in that environment. I used this test to verify all of my Boost threading changes.
4. All of the sources have been touched to add Windows-specific support, but I suppose that could be seen as part of this Jira issue. :-)
Disclaimers:
-----------
I had never done anything with autoconf before this. What I have done may not be correct. If changes are required, please let me know.
The configure script displays information at the end that is related to the new configuration stuff. It was useful for me while developing and I thought it might be a good idea to leave it in for a while.
I'm really not a Windows guy, I just do what needs doing. If any of you are Windows experts, and you have suggestions or corrections, please let me know.
See the "Known Issues" section in the msvc/README file.
Licensing
Obviously, I checked the box that says I grant the license to Apache for this code. There is one file, though, that already had a license header within it. That is the file lib/cpp/src/VisualStudioStdint.h, and it contains this notice:
// ISO C9x compliant stdint.h for Miscrosoft Visual Studio
// Based on ISO/IEC 9899:TC2 Committee draft (May 6, 2005) WG14/N1124
//
// Copyright (c) 2006 Alexander Chemeris
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
//
// 1. Redistributions of source code must retain the above copyright notice,
// this list of conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
//
// 3. The name of the author may be used to endorse or promote products
// derived from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
// WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
// EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
// OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
// WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
// OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
// ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
//
///////////////////////////////////////////////////////////////////////////////
I did not add an Apache license header to this file. I do not know what should be done. It can obviously be used, but there must be some requirement for dual licensing.
To apply the patch, do the following:
------------------------------------
1. Checkout from SVN:
svn co --revision 803313 http://svn.apache.org/repos/asf/incubator/thrift/trunk thrift
Note the --revision argument. The patch might work against other revisions, or it might not.
2. Make a copy of your distribution directory, because patch will patch the directory it works on in place.
3. Get the patch file from Jira by following this link:
https://issues.apache.org/jira/browse/THRIFT-311
Use the links there to download the patch file.
4. cd to the renamed distribution directory
5. Apply the patch by execution:
patch -p 8 <patchFile
where patchFile is the path to your downloaded patch file.
After applying the patch, the top level README file will have information about the new "msvc" subdirectory. The new file CPP_LIB_CFG has information about the configurations that are possible on *nix systems. Configurations can keep things the way they are currently, or can add-to and/or replace existing functionality with new functionality provided by Asio/Boost. (I had to do it this way so that I could use the existing test code with my new classes.)
The "msvc" subdirectory has its own README file that contains information particular to the msvc port. There are other msvc subdirectories spread through the tree. They follow a pattern, and they are all documented in the new README.
Tested Platforms:
----------------
Mac OS X 10.5.7 (me)
Gentoo Linux (Bruce Simpson)
FreeBSD (Bruce Simpson)
Windows XP (me)
Tested Boost and Asio versions:
------------------------------
Standalone Asio 1.2.0 and 1.4.1
On the Mac: Boost 1.36.0, 1.37.0, 1.38.0, and 1.39.0
On Windows XP: Boost 1.36.0
Along with the patch, I will attach a zip file called "MsvcPatchSupportScripts.zip" that contains a number of scripts that are useful for testing all of the variations (there are 10) on a *nix system. These scripts assume a very specific directory structure. It is documented in only one of them. Sorry. If you run "./buildAll.sh --help" at the command line, it will show you what is expected. It also documents two environment variables you can use with the scripts. They are called THRIFT_DEV_ROOT and THRIFT_DISTRO_DIR. Note that the scripts are designed to be run from the directory ABOVE your patched distribution directory.
The various scripts are:
buildAll.sh - Builds all the possible configuration variants by copying the distribution directory, then configuring and building. If you are on a Mac, be sure to copy your pkg.m4 file into the aclocal subdirectory of the distribution tree before you run this script. The build results are written into files called Log_* in the same directory as the script.
buildTutorialAll.sh - After buildAll.sh has run, this script iterates over all of the directories that were created and verifies that the C++ tutorial can be built both using the auto-generated Makefile and the Tutorial.mk makefile. It cleans up after itself.
clean.sh - Finds and removes all of the subdirectory trees and log files that are written by buildAll.sh.
concurrencyTest.sh - After buildAll.sh has run, this script iterates over all the directories that were created, runs the concurrency test in each of them and verifies that the test runs successfully.
cpCfgAndBuild.sh - This script knows how to configure and build Thrift in a single directory, but all it cares about is the C++ stuff. It also runs "make check". This script is used by the buildAll.sh script. This script currently defines the macro THRIFT_ENABLE_CONFIG_WARNINGS via CPPFLAGS. That generates warning messages at library compile time that tell you about the configuration. Remove the macro definition from the script if you don't want the warnings. (Just search for the macro name in the script file. What to do is pretty obvious once you find it.)
doTest.sh - This script runs the standard "make check" test suite on a single directory. It is used by the testAll.sh script.
findThriftDistro.sh - This script attempts to locate the Thrift distribution directory. It looks at the directory passed as the script argument if there is one, or it examines the environment variables THRIFT_DEV_ROOT and THRIFT_DISTRO_DIR if they are set. This script is used by many of the others, and you shouldn't need to mess with it.
testAll.sh - After buildAll.sh has run, this script iterates over all of the directories that were created and runs the standard test suite in each of them.
I hope others find this to be useful.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: MsvcPatchSupportScripts.zip, thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp, ThriftMsvcPatchForSvnRev803313.txt
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: thrift_server.cpp
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (THRIFT-311) ASIO client & server
Posted by "Erik Frey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671645#action_12671645 ]
erikfrey edited comment on THRIFT-311 at 2/8/09 9:00 AM:
----------------------------------------------------------
I think a thread pool that does i/o and runs the processor (your current design) is the best/most asio-y way to go. If you make a separate thread pool to process requests, you're just shifting from a queue that backs up on i/o to a queue that backs up on tasks.
I would love to see a coroutine thrift server, if just to see the raw throughput it could pump out :) There's a boost vault project called boost.coroutine ( http://www.crystalclearsoftware.com/soc/coroutine/index.html ) that plays pretty nicely with asio. If used properly, you could probably write a coroutine socket class that behaved like a blocking socket, but used coroutine under the hood to yield for any i/o.
My main concern with coroutines is how nicely they would play with concurrency in the user's processor code. Usual mutex/thread type stuff would not work!
Have you tried benchmarking against TNonBlockingServer? I'm curious how they compare, particularly on multi-core.
was (Author: erikfrey):
I think a thread pool that does i/o and runs the processor (your current design) is the best/most asio-y way to go. If you make a separate thread pool to process requests, you're just shifting from a queue that backs up on i/o to a queue that backs up on tasks.
I would love to see a coroutine thrift server, if just to see the raw throughput it could pump out :) There's a boost vault project called <a href="http://www.crystalclearsoftware.com/soc/coroutine/index.html">boost.coroutine</a> that plays pretty nicely with asio. If used properly, you could probably write a coroutine socket class that behaved like a blocking socket, but used coroutine under the hood to yield for any i/o.
My main concern with coroutines is how nicely they would play with concurrency in the user's processor code. Usual mutex/thread type stuff would not work!
Have you tried benchmarking against TNonBlockingServer? I'm curious how they compare, particularly on multi-core.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760323#action_12760323 ]
Rush Manbert commented on THRIFT-311:
-------------------------------------
Last comment on this issue, I promise!
I have deleted all of my comments and files related to my Windows port of the C++ library. Please see JIRA-591 if you;'re interested.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Attachment: (was: ThriftMsvcPatchForSvnRev803313_Rev1.txt.zip)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Attachment: ThriftMsvcPatchForSvnRev808166.txt.zip
ThriftMsvcPatchForSvnRev803313_Rev1.txt.zip
Updated patches for Thrift MSVC Port
27 August 2009
There's nothing like taking your code out into the real world to find out
what you screwed up.
These patches (for 2 different SVN revisions) do the following:
* Fixes a bug in all flavors of TAsioServerSocket that made the interrupt()
method do nothing. The result was that calling stop() on a TServer whose
underlying socket used the TAsio* implementation did nothing and the server
thread would not be stopped. On Windows, the named pipe connect call was made
to be interruptible.
* In order to test that you really can stop the servers, I changed stress-test
so that it stops the server it has running when the test has completed
successfully, and waits for the thread to join. This means that we know
the server object's address and remember it, and that we specifically
make the server thread joinable when it is created.
* On Windows, added the capability to select static runtime linking for the
libraries and executables. To use this configuration option, your Boost
and Zlib libraries must be linked the same way. The default linkage is still
to the DLL runtime libraries. Run "cscript configure.js --help" for
usage information. If you select this option, the default Boost library name
paths are adjusted automatically. Since the zlib project does not support
this type of linkage out of the box, the same default patterns will be used
for either type of runtime linking. See the next item if you need to handle
modified file names for zlib libraries.
* On Windows, changed the configure script so that it accepts configuration-
specific library name search patterns for both Boost and Zlib. This lets you
specify things like:
--with-zlib-lib-name-pattern[win32][static][debug]=zlib-Win32-sd\.lib
and that pattern will only be used to search for the zlib library for
the configuration tuple (win32, static, debug). Of course, if you are
doing something nonstandard (as we are where I work), then you probably
need to specify the pattern for every tuple of interest. But at least
you can.
* On Windows, I put dummy versions of the property sheet files the configure
script generates into the configGenerated directory. This lets you open the
Visual Studio projects even if you haven't run the configure script. The
main reason is that you might need to upgrade all of the VS projects to
be compatible with a newer version of Visual Studio, and it's easier if
you can just open them without needing to configure first.
If anyone is playing with the patch, I apologize for the server bug. I had
halfway implemented it, but then never tested it. And if anyone wanted to
play on Windows but was thwarted by static runtime linking or nonstandard
library names, I apologize for the delay in supporting them.
I have a sneaking suspicion that there might be a need for some way
to customize the VS project settings via user-defined property sheets. (Well,
there might be this requirement if anyone but me ever uses this stuff.)
I didn't need them, so didn't put in the work required to add the capability.
It would require a configure command line parameter something like
--prop-sheet=myPropSheet.vsprops
and the configure script would write a property sheet that inherits from
that one, and all of the Visual Studio projects would inherit the property
sheet that we write. If there are no --prop-sheet arguments, then we would
write a property sheet file that doesn't do anything. Not particularly hard,
just tedious and time consuming to test.
There are 2 patch files:
* The first is a new patch that is to be applied to Subversion rev 803313.
It is called:
ThriftMsvcPatchForSvnRev803313_Rev1.txt.zip
(yes, I remembered to compress it this time) and it supersedes the previous
patch file called:
ThriftMsvcPatchForSvnRev803313.txt
(i.e. just apply this patch directly to a checkout of 803313, NOT to
a checkout that you already patched.)
* The second is made against a newer HEAD rev of the repository. It is
made to be applied to a checkout of Subversion rev 808166, and is called:
ThriftMsvcPatchForSvnRev808166.txt.zip
Apply the patches with -p8, as before.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: MsvcPatchSupportScripts.zip, thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp, ThriftMsvcPatchForSvnRev803313.txt, ThriftMsvcPatchForSvnRev803313_Rev1.txt.zip, ThriftMsvcPatchForSvnRev808166.txt.zip
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Attachment: (was: MsvcPatchSupportScripts.zip)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteve Fernandez updated THRIFT-311:
------------------------------------
Attachment: ThriftCalculatorASIOServer.cpp
Simple ASIO TCP server that implements the Calculator service.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: New Feature
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Attachment: (was: thrift-808166-Patched.zip)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-311) ASIO client & server
Posted by "Rush Manbert (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rush Manbert updated THRIFT-311:
--------------------------------
Attachment: MsvcPatchSupportScripts.zip
ThriftMsvcPatchForSvnRev803313.txt
The patch file and the support scripts archive.
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: MsvcPatchSupportScripts.zip, thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp, ThriftMsvcPatchForSvnRev803313.txt
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-311) ASIO client & server
Posted by "Esteve Fernandez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732673#action_12732673 ]
Esteve Fernandez commented on THRIFT-311:
-----------------------------------------
Nitay, these files are intended only as a prototype, but they are fully functional. In any case, have you compiled them in Windows? I don't have access to a Windows machine, but ASIO should be cross platform, let me know if it doesn't work.
Anyway, Erik Bernhardsson and Mattias de Zalenski recently sent a patch to the thrift-dev mailing list which implements an asynchronous client and server using ASIO for Thrift. If you're interested in a fully asynchronous version of Thrift in C++, you're welcome to join the discussion! :-)
> ASIO client & server
> --------------------
>
> Key: THRIFT-311
> URL: https://issues.apache.org/jira/browse/THRIFT-311
> Project: Thrift
> Issue Type: Sub-task
> Components: Library (C++)
> Reporter: Esteve Fernandez
> Attachments: thrift_connection.cpp, thrift_connection.hpp, thrift_connection_v2.cpp, thrift_handler.cpp, thrift_handler.hpp, thrift_main.cpp, thrift_server.cpp, thrift_server.hpp, ThriftCalculatorASIOServer.cpp
>
>
> Given the recent discussion on a Windows port and moving to ASIO (http://mail-archives.apache.org/mod_mbox/incubator-thrift-dev/200901.mbox/%3C03D5F82D-574D-418B-A409-C521227888A9@manbert.com%3E), I decided to hack a little Thrift asynchronous prototype server using ASIO and here's the result. It implements the Calculator service that can be found in the tutorial and, just like TNonblockingServer, it uses a FramedTransport.
> It's just a quick prototype, but I think it's enough for building a more generic server/protocol. I've only tested it in Linux, but I think there's nothing platform-dependent and can be compiled "as is" in Windows.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.