You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Nikita Vetoshkin (JIRA)" <ji...@apache.org> on 2012/12/03 08:48:02 UTC
[jira] [Created] (ZOOKEEPER-1595) Socket's should be read until
exhausted
Nikita Vetoshkin created ZOOKEEPER-1595:
-------------------------------------------
Summary: Socket's should be read until exhausted
Key: ZOOKEEPER-1595
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1595
Project: ZooKeeper
Issue Type: Improvement
Components: server
Environment: Tested on Linux x64 with Oracle JDK6
Reporter: Nikita Vetoshkin
Priority: Minor
{{doIO}} method in {{NIOServerCnxn}} should read (and write too) until {{read}}/{{write}} returns 0.
It's a common practice when working with non-blocking sockets. When an underlying system call (multiplexer) signals, that socket is readable, one should {{recv(2)}} all data from kernel buffer until {{recv}} fails with {{EAGAIN}} or {{EWOULDBLOCK}}.
Patch does two things (I know it's not a good idea to mix several changes, but I could stand it):
* splits {{doIO}} into {{doRead}} and {{doWrite}}
* wraps reading with {{while (true)}}
It's pretty easy to instrument code with a counter and print how many loops we performed until the socket was not readable again.
I wrote a simple python script (http://pastebin.com/N5ifM330) which creates 6000 nodes with 5k data each, having 20 concurrent create requests in progress through one connnection.
With this script and strace attached to JVM I counted epoll_wait syscalls during the test and I got ~9500 before vs ~8000 after.
Run time measurement is very rough, but it's around ~19 secs. before vs 17.5 after.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1595) Socket's should be read until
exhausted
Posted by "Nikita Vetoshkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508555#comment-13508555 ]
Nikita Vetoshkin commented on ZOOKEEPER-1595:
---------------------------------------------
The same should be applied to write operations, but it looks a bit trickier. BTW, the same issue applies to java client.
> Socket's should be read until exhausted
> ---------------------------------------
>
> Key: ZOOKEEPER-1595
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1595
> Project: ZooKeeper
> Issue Type: Improvement
> Components: server
> Environment: Tested on Linux x64 with Oracle JDK6
> Reporter: Nikita Vetoshkin
> Priority: Minor
> Labels: newbie, performance
> Attachments: read_write_until_exhausted.patch
>
>
> {{doIO}} method in {{NIOServerCnxn}} should read (and write too) until {{read}}/{{write}} returns 0.
> It's a common practice when working with non-blocking sockets. When an underlying system call (multiplexer) signals, that socket is readable, one should {{recv(2)}} all data from kernel buffer until {{recv}} fails with {{EAGAIN}} or {{EWOULDBLOCK}}.
> Patch does two things (I know it's not a good idea to mix several changes, but I could stand it):
> * splits {{doIO}} into {{doRead}} and {{doWrite}}
> * wraps reading with {{while (true)}}
> It's pretty easy to instrument code with a counter and print how many loops we performed until the socket was not readable again.
> I wrote a simple python script (http://pastebin.com/N5ifM330) which creates 6000 nodes with 5k data each, having 20 concurrent create requests in progress through one connnection.
> With this script and strace attached to JVM I counted epoll_wait syscalls during the test and I got ~9500 before vs ~8000 after.
> Run time measurement is very rough, but it's around ~19 secs. before vs 17.5 after.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1595) Socket's should be read until
exhausted
Posted by "Nikita Vetoshkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nikita Vetoshkin updated ZOOKEEPER-1595:
----------------------------------------
Attachment: read_write_until_exhausted.patch
Patch v1 against current trunk
> Socket's should be read until exhausted
> ---------------------------------------
>
> Key: ZOOKEEPER-1595
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1595
> Project: ZooKeeper
> Issue Type: Improvement
> Components: server
> Environment: Tested on Linux x64 with Oracle JDK6
> Reporter: Nikita Vetoshkin
> Priority: Minor
> Labels: newbie, performance
> Attachments: read_write_until_exhausted.patch
>
>
> {{doIO}} method in {{NIOServerCnxn}} should read (and write too) until {{read}}/{{write}} returns 0.
> It's a common practice when working with non-blocking sockets. When an underlying system call (multiplexer) signals, that socket is readable, one should {{recv(2)}} all data from kernel buffer until {{recv}} fails with {{EAGAIN}} or {{EWOULDBLOCK}}.
> Patch does two things (I know it's not a good idea to mix several changes, but I could stand it):
> * splits {{doIO}} into {{doRead}} and {{doWrite}}
> * wraps reading with {{while (true)}}
> It's pretty easy to instrument code with a counter and print how many loops we performed until the socket was not readable again.
> I wrote a simple python script (http://pastebin.com/N5ifM330) which creates 6000 nodes with 5k data each, having 20 concurrent create requests in progress through one connnection.
> With this script and strace attached to JVM I counted epoll_wait syscalls during the test and I got ~9500 before vs ~8000 after.
> Run time measurement is very rough, but it's around ~19 secs. before vs 17.5 after.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1595) Sockets should be read until
exhausted
Posted by "Nikita Vetoshkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nikita Vetoshkin updated ZOOKEEPER-1595:
----------------------------------------
Description:
{{doIO}} method in {{NIOServerCnxn}} should read (and write too) until {{read}}/{{write}} returns 0.
It's a common practice when working with non-blocking sockets. When an underlying system call (multiplexer) signals, that socket is readable, one should {{recv(2)}} all data from kernel buffer until {{recv}} fails with {{EAGAIN}} or {{EWOULDBLOCK}}.
Patch does two things (I know it's not a good idea to mix several changes, but I could stand it):
* splits {{doIO}} into {{doRead}} and {{doWrite}}
* wraps reading with {{while (true)}}
It's pretty easy to instrument the code with a counter and print how many loops we performed until the socket was not readable again.
I wrote a simple python script (http://pastebin.com/N5ifM330) which creates 6000 nodes with 5k data each, having 20 concurrent create requests in progress through one connnection.
With this script and strace attached to JVM I counted epoll_wait syscalls during the test and I got ~9500 before vs ~8000 after.
Run time measurement is very rough, but it's around ~19 secs. before vs 17.5 after.
was:
{{doIO}} method in {{NIOServerCnxn}} should read (and write too) until {{read}}/{{write}} returns 0.
It's a common practice when working with non-blocking sockets. When an underlying system call (multiplexer) signals, that socket is readable, one should {{recv(2)}} all data from kernel buffer until {{recv}} fails with {{EAGAIN}} or {{EWOULDBLOCK}}.
Patch does two things (I know it's not a good idea to mix several changes, but I could stand it):
* splits {{doIO}} into {{doRead}} and {{doWrite}}
* wraps reading with {{while (true)}}
It's pretty easy to instrument code with a counter and print how many loops we performed until the socket was not readable again.
I wrote a simple python script (http://pastebin.com/N5ifM330) which creates 6000 nodes with 5k data each, having 20 concurrent create requests in progress through one connnection.
With this script and strace attached to JVM I counted epoll_wait syscalls during the test and I got ~9500 before vs ~8000 after.
Run time measurement is very rough, but it's around ~19 secs. before vs 17.5 after.
> Sockets should be read until exhausted
> --------------------------------------
>
> Key: ZOOKEEPER-1595
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1595
> Project: ZooKeeper
> Issue Type: Improvement
> Components: server
> Environment: Tested on Linux x64 with Oracle JDK6
> Reporter: Nikita Vetoshkin
> Priority: Minor
> Labels: newbie, performance
> Attachments: read_write_until_exhausted.patch
>
>
> {{doIO}} method in {{NIOServerCnxn}} should read (and write too) until {{read}}/{{write}} returns 0.
> It's a common practice when working with non-blocking sockets. When an underlying system call (multiplexer) signals, that socket is readable, one should {{recv(2)}} all data from kernel buffer until {{recv}} fails with {{EAGAIN}} or {{EWOULDBLOCK}}.
> Patch does two things (I know it's not a good idea to mix several changes, but I could stand it):
> * splits {{doIO}} into {{doRead}} and {{doWrite}}
> * wraps reading with {{while (true)}}
> It's pretty easy to instrument the code with a counter and print how many loops we performed until the socket was not readable again.
> I wrote a simple python script (http://pastebin.com/N5ifM330) which creates 6000 nodes with 5k data each, having 20 concurrent create requests in progress through one connnection.
> With this script and strace attached to JVM I counted epoll_wait syscalls during the test and I got ~9500 before vs ~8000 after.
> Run time measurement is very rough, but it's around ~19 secs. before vs 17.5 after.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1595) Sockets should be read until
exhausted
Posted by "Nikita Vetoshkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nikita Vetoshkin updated ZOOKEEPER-1595:
----------------------------------------
Summary: Sockets should be read until exhausted (was: Socket's should be read until exhausted)
> Sockets should be read until exhausted
> --------------------------------------
>
> Key: ZOOKEEPER-1595
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1595
> Project: ZooKeeper
> Issue Type: Improvement
> Components: server
> Environment: Tested on Linux x64 with Oracle JDK6
> Reporter: Nikita Vetoshkin
> Priority: Minor
> Labels: newbie, performance
> Attachments: read_write_until_exhausted.patch
>
>
> {{doIO}} method in {{NIOServerCnxn}} should read (and write too) until {{read}}/{{write}} returns 0.
> It's a common practice when working with non-blocking sockets. When an underlying system call (multiplexer) signals, that socket is readable, one should {{recv(2)}} all data from kernel buffer until {{recv}} fails with {{EAGAIN}} or {{EWOULDBLOCK}}.
> Patch does two things (I know it's not a good idea to mix several changes, but I could stand it):
> * splits {{doIO}} into {{doRead}} and {{doWrite}}
> * wraps reading with {{while (true)}}
> It's pretty easy to instrument code with a counter and print how many loops we performed until the socket was not readable again.
> I wrote a simple python script (http://pastebin.com/N5ifM330) which creates 6000 nodes with 5k data each, having 20 concurrent create requests in progress through one connnection.
> With this script and strace attached to JVM I counted epoll_wait syscalls during the test and I got ~9500 before vs ~8000 after.
> Run time measurement is very rough, but it's around ~19 secs. before vs 17.5 after.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira