You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Eric Evans (JIRA)" <ji...@apache.org> on 2010/02/19 20:56:27 UTC
[jira] Created: (AVRO-423) HTTPTransceiver does not reuse
connections
HTTPTransceiver does not reuse connections
------------------------------------------
Key: AVRO-423
URL: https://issues.apache.org/jira/browse/AVRO-423
Project: Avro
Issue Type: Bug
Components: python
Affects Versions: 1.3.0
Reporter: Eric Evans
Fix For: 1.3.0
The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838502#action_12838502 ]
Doug Cutting commented on AVRO-423:
-----------------------------------
> demonstrate multiple request/response cycles python-to-python, will that work?
That seems sufficient to me.
The interop tests may make multiple HTTP requests per test for the handshake. So it would also be good if the command line tool used by interop testing reused the connection in this case.
Also, the preferred format for patches is a single patch file.
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Attachments: v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835983#action_12835983 ]
Jeff Hammerbacher commented on AVRO-423:
----------------------------------------
Have you tried this patch out with a Python client and Java server, or any other exchange which results in two RPCs? There are a lot of hacks in the current implementation to get around limitations I discovered during testing. I don't have a unit test for this situation (yet), but if you wanted to put one up at https://issues.apache.org/jira/browse/AVRO-333 and show that this patch passes, I'd accept it.
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Fix For: 1.3.0
>
> Attachments: v1-0001-AVRO-423.-reuse-existing-http-connections.txt, v1-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835993#action_12835993 ]
Doug Cutting commented on AVRO-423:
-----------------------------------
Is this a blocker for 1.3? If not, we should change its "fix version".
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Fix For: 1.3.0
>
> Attachments: v1-0001-AVRO-423.-reuse-existing-http-connections.txt, v1-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Evans updated AVRO-423:
----------------------------
Attachment: v1-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
v1-0001-AVRO-423.-reuse-existing-http-connections.txt
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Fix For: 1.3.0
>
> Attachments: v1-0001-AVRO-423.-reuse-existing-http-connections.txt, v1-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Evans updated AVRO-423:
----------------------------
Attachment: (was: v1-0001-AVRO-423.-reuse-existing-http-connections.txt)
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Attachments: v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836010#action_12836010 ]
Eric Evans commented on AVRO-423:
---------------------------------
This Cassandra functional test makes two requests, (and passes).
http://svn.apache.org/repos/asf/incubator/cassandra/trunk/test/system/test_avro_server.py
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Attachments: v1-0001-AVRO-423.-reuse-existing-http-connections.txt, v1-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Hammerbacher updated AVRO-423:
-----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 925147. Thanks, Eric!
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Improvement
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Assignee: Eric Evans
> Fix For: 1.3.2
>
> Attachments: AVRO-423.patch, v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Hammerbacher updated AVRO-423:
-----------------------------------
Fix Version/s: 1.3.2
Status: Patch Available (was: Open)
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Assignee: Eric Evans
> Fix For: 1.3.2
>
> Attachments: AVRO-423.patch, v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Hammerbacher updated AVRO-423:
-----------------------------------
Fix Version/s: (was: 1.3.0)
Not a blocker for 1.3
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Attachments: v1-0001-AVRO-423.-reuse-existing-http-connections.txt, v1-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Evans updated AVRO-423:
----------------------------
Attachment: (was: v1-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt)
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Attachments: v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835948#action_12835948 ]
Jeff Hammerbacher commented on AVRO-423:
----------------------------------------
httplib can only send a single POST message over an HTTPConnection, so I don't think we can reuse connections without writing our own HTTP client library. The latter option is not crazy, but I'm trying to keep the Python implementation clean and easy to read rather than performant. Which patch am I supposed to look at?
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Fix For: 1.3.0
>
> Attachments: v1-0001-AVRO-423.-reuse-existing-http-connections.txt, v1-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Evans updated AVRO-423:
----------------------------
Attachment: v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
v2-0001-AVRO-423.-reuse-existing-http-connections.txt
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Attachments: v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Hammerbacher updated AVRO-423:
-----------------------------------
Attachment: AVRO-423.patch
Consolidated and updated Eric's patch to use new HTTPTransceiver ctor in tool.py
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Assignee: Eric Evans
> Fix For: 1.3.2
>
> Attachments: AVRO-423.patch, v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12835977#action_12835977 ]
Eric Evans commented on AVRO-423:
---------------------------------
{quote}
httplib can only send a single POST message over an HTTPConnection
{quote}
Are you sure? That doesn't sound right at all.
{quote}
Which patch am I supposed to look at?
{quote}
The first patch (v1-0001-AVRO-423.-reuse-existing-http-connections.txt) is the one that makes it so that you can reuse connections. The important part is the final read() on the response which is necessary to satisfy httplib's state machine (see the little ascii art diagram in the module docstrings).
The second patch merely creates the HTTPConnection instance in the __init__() of the transceiver, which seems like better encapsulation to me, YMMV.
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Fix For: 1.3.0
>
> Attachments: v1-0001-AVRO-423.-reuse-existing-http-connections.txt, v1-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Hammerbacher reassigned AVRO-423:
--------------------------------------
Assignee: Jeff Hammerbacher
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Assignee: Jeff Hammerbacher
> Attachments: v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Hammerbacher reassigned AVRO-423:
--------------------------------------
Assignee: Eric Evans (was: Jeff Hammerbacher)
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Assignee: Eric Evans
> Attachments: v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12836034#action_12836034 ]
Eric Evans commented on AVRO-423:
---------------------------------
test/sample_http_client.py and test/sample_http_server.py can be used to demonstrate multiple request/response cycles python-to-python, will that work?
This new patchset moves things around a bit so that the HTTPResponse.read() occurs in the transceiver instead of FramedReader (which can interchangeably uses blocking sockets :o ). It works with the sample scripts in test/, and with the Cassandra functional tests linked above.
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Attachments: v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-423) HTTPTransceiver does not reuse
connections
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Hammerbacher updated AVRO-423:
-----------------------------------
Issue Type: Improvement (was: Bug)
> HTTPTransceiver does not reuse connections
> ------------------------------------------
>
> Key: AVRO-423
> URL: https://issues.apache.org/jira/browse/AVRO-423
> Project: Avro
> Issue Type: Improvement
> Components: python
> Affects Versions: 1.3.0
> Reporter: Eric Evans
> Assignee: Eric Evans
> Fix For: 1.3.2
>
> Attachments: AVRO-423.patch, v2-0001-AVRO-423.-reuse-existing-http-connections.txt, v2-0002-AVRO-423.-generate-connection-instances-in-transceiver.txt
>
>
> The Python libs HTTPTransceiver creates a new connection for each request. Patch to follow.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.