You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Jeff Hammerbacher (JIRA)" <ji...@apache.org> on 2010/01/14 00:58:54 UTC
[jira] Created: (AVRO-323) Support raw sockets as a transport in
avroj
Support raw sockets as a transport in avroj
-------------------------------------------
Key: AVRO-323
URL: https://issues.apache.org/jira/browse/AVRO-323
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Jeff Hammerbacher
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting reassigned AVRO-323:
---------------------------------
Assignee: Doug Cutting
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Assignee: Doug Cutting
> Attachments: AVRO-323.patch, AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting resolved AVRO-323.
-------------------------------
Resolution: Won't Fix
We should not further embrace a raw socket protocol until it's fully specified.
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Assignee: Doug Cutting
> Attachments: AVRO-323.patch, AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-323:
------------------------------
Attachment: AVRO-323.patch
Here's a version that:
- includes tests
- supports "rpcsend" too
- actually works!
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Attachments: AVRO-323.patch, AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800042#action_12800042 ]
Jeff Hammerbacher commented on AVRO-323:
----------------------------------------
Getting an error when I try this out:
{code}
java -jar build/avro-tools-1.2.0-dev.jar rpcreceive '{"namespace":"example.proto","protocol":"Mail","types":[{"name":"Message","type":"record","fields":[{"name":"to","type":"string"},{"name":"from","type":"string"},{"name":"body","type":"string"}]}],"messages":{"send":{"request":[{"name":"message","type":"Message"}],"response":"string"},"replay":{"request":[],"response":"string"}}}' 9090 send '{"message": {"to":"avro", "from":"jeff", "body":"hey"}}' socket
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 5,Size: 5
{code}
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Attachments: AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800302#action_12800302 ]
Jeff Hammerbacher commented on AVRO-323:
----------------------------------------
When debugging rpcsend, I can see that my Python server is receiving the initial client request, parsing the Handshake properly, and noting that it doesn't know the client hash. It's then returning a handshake response and closing the socket. The Java client doesn't know that the socket has been closed on it, so it tries to write its more verbose handshake request, but gets a broken pipe, like so:
{code}
19 [main] INFO org.apache.avro.ipc.SocketTransceiver - open to localhost/127.0.0.1:9090
Exception in thread "main" java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:75)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at org.apache.avro.ipc.SocketTransceiver.writeBuffers(SocketTransceiver.java:78)
at org.apache.avro.ipc.Transceiver.transceive(Transceiver.java:33)
at org.apache.avro.ipc.Requestor.request(Requestor.java:110)
at org.apache.avro.tool.RpcSendTool.run(RpcSendTool.java:80)
at org.apache.avro.tool.Main.run(Main.java:73)
at org.apache.avro.tool.Main.main(Main.java:62)
{code}
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Assignee: Doug Cutting
> Attachments: AVRO-323.patch, AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800772#action_12800772 ]
Philip Zeyliger commented on AVRO-323:
--------------------------------------
+1 to the patch. In TestRpcReceiveAndSendTools, can you move the JavaDoc, I think the javadoc should be attached to the serveAndSend(protocol) method, instead of one of the test methods.
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Assignee: Doug Cutting
> Attachments: AVRO-323.patch, AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800715#action_12800715 ]
Jeff Hammerbacher commented on AVRO-323:
----------------------------------------
Philip, Doug: Python's out-of-the-box TCPServer implementation will call close() on the socket for each message it handles. There are three lines of logic in the Python client to handle this case (get a new socket if the server has hung up). I'm not sure what the right design decision is here: require all clients to handle the case of a server hanging up mid-handshake, or require all servers to leave the connection open. I think the former makes more sense than the latter, but you guys probably have better taste than me in this regard. Thoughts?
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Assignee: Doug Cutting
> Attachments: AVRO-323.patch, AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800300#action_12800300 ]
Jeff Hammerbacher commented on AVRO-323:
----------------------------------------
In rpcsend with the socket transport, are you opening the socket with the SO_REUSEADDR option? I'm getting "address already in use" errors after killing the client for a few seconds afterwards. Debugging is easier with SO_REUSEADDR.
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Assignee: Doug Cutting
> Attachments: AVRO-323.patch, AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800052#action_12800052 ]
Jeff Hammerbacher commented on AVRO-323:
----------------------------------------
Doug: would be nice to have rpcsend as well.
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Attachments: AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800035#action_12800035 ]
Jeff Hammerbacher commented on AVRO-323:
----------------------------------------
Patch applies from inside of lang/java: is that the new intended behavior for patches post-163? If so, we should update the "How to Contribute" wiki page.
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Attachments: AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-323) Support raw sockets as a transport in
avroj
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-323:
------------------------------
Attachment: AVRO-323.patch
Here's an early-version of an untested patch that permits one to specify the transport on the command line to rpcreceive. Use 'socket' to get the socket transport.
> Support raw sockets as a transport in avroj
> -------------------------------------------
>
> Key: AVRO-323
> URL: https://issues.apache.org/jira/browse/AVRO-323
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Jeff Hammerbacher
> Attachments: AVRO-323.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.