You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Eric Jensen (JIRA)" <ji...@apache.org> on 2010/09/02 00:22:01 UTC
[jira] Created: (THRIFT-888) async client should also have
nonblocking connect
async client should also have nonblocking connect
-------------------------------------------------
Key: THRIFT-888
URL: https://issues.apache.org/jira/browse/THRIFT-888
Project: Thrift
Issue Type: Improvement
Components: Java - Library
Affects Versions: 0.4, 0.5
Reporter: Eric Jensen
Fix For: 0.5
or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Jensen updated THRIFT-888:
-------------------------------
Attachment: (was: async_connect.diff)
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Jensen updated THRIFT-888:
-------------------------------
Attachment: async_connect.diff
untested, and needs some tests written for it if you're feeling ambitious, but please review. i don't think this has any interaction with the server socket because it doesn't seem to use TNonblockingSocket?
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
> Attachments: async_connect.diff
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Jensen updated THRIFT-888:
-------------------------------
Attachment: (was: async_connect_2.diff)
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Jensen updated THRIFT-888:
-------------------------------
Attachment: (was: async_connect_final.diff)
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury closed THRIFT-888.
--------------------------------
Assignee: Eric Jensen
Resolution: Fixed
I just committed this. Thanks for the patch!
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Assignee: Eric Jensen
> Fix For: 0.5
>
> Attachments: async_connect_final.diff
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907454#action_12907454 ]
Eric Jensen commented on THRIFT-888:
------------------------------------
I fixed the indentation and logging in the attached patch. Ning is going to rewrite the test in a separate patch.
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Assignee: Eric Jensen
> Fix For: 0.5
>
> Attachments: async_connect_final.diff
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Jensen updated THRIFT-888:
-------------------------------
Attachment: async_connect_final.diff
updated to address bryan's comments
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
> Attachments: async_connect_final.diff
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Jensen updated THRIFT-888:
-------------------------------
Attachment: async_connect_2.diff
first draft
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
> Attachments: async_connect.diff, async_connect_2.diff
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Jensen updated THRIFT-888:
-------------------------------
Comment: was deleted
(was: untested, and needs some tests written for it if you're feeling ambitious, but please review. i don't think this has any interaction with the server socket because it doesn't seem to use TNonblockingSocket?)
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906166#action_12906166 ]
Eric Jensen commented on THRIFT-888:
------------------------------------
Final patch attached for integration. Reviewed this heavily with Ning and he approves. The core change is that it provides lazy non-blocking connect behavior from TNonblockinSocket when it interacts with TAsyncClientManager by adding a new state to the method call. Also included:
1. bumped the SELECT_TIME down so we can have finer grained timeouts
2. catch runtime exceptions within the entire manager select thread so that ones generated from a callback don't kill it
3. refactored the async test a bit: Initial attempts at this had transient failures in the JankyRunnable portion of TestTAsyncClientManager where the caller would either never be called back for the connect or we would get a SocketException: Connection reset by peer on finishConnect. When I profiled the test I found that the single TNonblockingServer$SelectThread was busy doing reads and writes all the way up until the timeout. Even though we make these calls in parallel, we dispatch them in serial in the server and that appeared to be the bottleneck, so it was possible that we started a whole bunch of them but didn't finish any of them until roughly the same time. if each took 0.5ms in the worst case though, that could be 500*100*0.5ms = 2.5s. so basically the one second wait was just too agressive for running 500 threads with 100 calls per thread. I increased that and reduced the numThreads to 50 (to fix the connection reset problems that i can't explain) and now no longer see any failures after running the test 100 times on snow leopard or linux.
This test is really dirty, mostly because of copied and pasted code but also because it asserts within callbacks, which throws assertion failures that the selector thread subsequently catches. We should store off the throwables and assert outside of the callbacks. Also it appears that part of why the test is so slow and I had to mess so much with num threads and timeouts is because we do synchronization within the callbacks to wake up the calling thread. This means the selector itself is constantly having to notify a bunch of threads, and I caught that as something taking time in the stack traces (since it blocks all other selector progress). A cleaner, faster test would probably be to have a truly async client as the test driver, enqueing the next iteration into a ConcurrentLinkedQueue on the callback rather than doing something synchronized.
That said, I think it is sufficient for now and definitely covers the new non-blocking connect functionality. I fixed a bunch of stuff about how it logged errors and switched to using latches instead of objects since there was a race condition where you could notify before the caller waits.
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
> Attachments: async_connect_final.diff
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Jensen updated THRIFT-888:
-------------------------------
Attachment: async_connect_final.diff
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
> Attachments: async_connect_final.diff
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Eric Jensen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Jensen updated THRIFT-888:
-------------------------------
Comment: was deleted
(was: first draft)
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-888) async client should also have
nonblocking connect
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907431#action_12907431 ]
Bryan Duxbury commented on THRIFT-888:
--------------------------------------
I think this generally looks pretty good. A few comments:
The indentation in TAsyncClientManager$SelectThread.run is a little funky.
If we're going to put a finally block in TNonblockingServer's run method, we should probably at least log whatever gets thrown - I hate silent catches.
If you have concerns about the code quality in the test, I think you'd better just go ahead fix it. Realistically, if you leave it the way it is, it's not going to get changed by anyone else in the foreseeable future.
> async client should also have nonblocking connect
> -------------------------------------------------
>
> Key: THRIFT-888
> URL: https://issues.apache.org/jira/browse/THRIFT-888
> Project: Thrift
> Issue Type: Improvement
> Components: Java - Library
> Affects Versions: 0.4, 0.5
> Reporter: Eric Jensen
> Fix For: 0.5
>
> Attachments: async_connect_final.diff
>
>
> or its purpose of avoiding thread pooling around blocking I/O is defeated. this was discussed briefly in the comments of https://issues.apache.org/jira/browse/THRIFT-862 and will most likely obviate https://issues.apache.org/jira/browse/THRIFT-843
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.