You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Philip Zeyliger (JIRA)" <ji...@apache.org> on 2009/12/30 21:24:30 UTC
[jira] Created: (AVRO-271) InProcessTranceiver: connect RPCs
without going through any sockets
InProcessTranceiver: connect RPCs without going through any sockets
-------------------------------------------------------------------
Key: AVRO-271
URL: https://issues.apache.org/jira/browse/AVRO-271
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Philip Zeyliger
Assignee: Philip Zeyliger
For testing (both Avro itself, and code that uses
Avro servers) it's sometimes handy to connect the RPCs without
even a socket. This implementation of a Transceiver does just that.
(This can, for users, both avoid the overhead of using sockets, and,
in debugging, let users see stack traces that have both client
and server code in one thread.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-271) InProcessTranceiver: connect RPCs
without going through any sockets
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Zeyliger updated AVRO-271:
---------------------------------
Status: Patch Available (was: Open)
> InProcessTranceiver: connect RPCs without going through any sockets
> -------------------------------------------------------------------
>
> Key: AVRO-271
> URL: https://issues.apache.org/jira/browse/AVRO-271
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: AVRO-271.patch.txt
>
>
> For testing (both Avro itself, and code that uses
> Avro servers) it's sometimes handy to connect the RPCs without
> even a socket. This implementation of a Transceiver does just that.
> (This can, for users, both avoid the overhead of using sockets, and,
> in debugging, let users see stack traces that have both client
> and server code in one thread.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-271) InProcessTranceiver: connect RPCs
without going through any sockets
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-271:
------------------------------
Resolution: Fixed
Fix Version/s: 1.3.0
Status: Resolved (was: Patch Available)
I just committed this. Thanks Philip!
> InProcessTranceiver: connect RPCs without going through any sockets
> -------------------------------------------------------------------
>
> Key: AVRO-271
> URL: https://issues.apache.org/jira/browse/AVRO-271
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Fix For: 1.3.0
>
> Attachments: AVRO-271.patch.txt, AVRO-271.patch.txt
>
>
> For testing (both Avro itself, and code that uses
> Avro servers) it's sometimes handy to connect the RPCs without
> even a socket. This implementation of a Transceiver does just that.
> (This can, for users, both avoid the overhead of using sockets, and,
> in debugging, let users see stack traces that have both client
> and server code in one thread.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-271) InProcessTranceiver: connect RPCs
without going through any sockets
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Zeyliger updated AVRO-271:
---------------------------------
Attachment: AVRO-271.patch.txt
> So removing readBuffers() and writeBuffers() altogether might make sense, and declaring that Transceiver is for client-side use only.
Makes sense. Let's do a separate JIRA for that.
> InProcessTransceiver can simply implement transceive() to directly invoke Responder#respond().
Indeed; makes it even simpler.
> Also, what do you think of renaming InProcessTransceiver to be LocalTransceiver?
I like it. Done.
> InProcessTranceiver: connect RPCs without going through any sockets
> -------------------------------------------------------------------
>
> Key: AVRO-271
> URL: https://issues.apache.org/jira/browse/AVRO-271
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: AVRO-271.patch.txt, AVRO-271.patch.txt
>
>
> For testing (both Avro itself, and code that uses
> Avro servers) it's sometimes handy to connect the RPCs without
> even a socket. This implementation of a Transceiver does just that.
> (This can, for users, both avoid the overhead of using sockets, and,
> in debugging, let users see stack traces that have both client
> and server code in one thread.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-271) InProcessTranceiver: connect RPCs
without going through any sockets
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Zeyliger updated AVRO-271:
---------------------------------
Attachment: AVRO-271.patch.txt
Attaching a patch.
The code itself is pretty trivial. I think it's important that this exist in the src/java tree (and not the test/src/java) tree, because it's something that can be used for clients of AVRO testing their own code. For example, when Hadoop sets up a MiniCluster, this could be used.
This was my first foray into the Transceiver initerface (really, abstract class). Am I right that the interface would need to be changed to support a single transceiver being used to process multiple RPCs concurrently? I was thinking of having InProcessTransceiver backed by an Executor, but it didn't make sense. Naively, you could change Transceiver to be an interface with just getRemoteName(), close(), and transceive(), and the fact that many implementations share a synchronized transceive would be an implementation detail. To be clear, I don't wish this JIRA to become the one where we maybe change Transceiver.java.
-- Philip
> InProcessTranceiver: connect RPCs without going through any sockets
> -------------------------------------------------------------------
>
> Key: AVRO-271
> URL: https://issues.apache.org/jira/browse/AVRO-271
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: AVRO-271.patch.txt
>
>
> For testing (both Avro itself, and code that uses
> Avro servers) it's sometimes handy to connect the RPCs without
> even a socket. This implementation of a Transceiver does just that.
> (This can, for users, both avoid the overhead of using sockets, and,
> in debugging, let users see stack traces that have both client
> and server code in one thread.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-271) InProcessTranceiver: connect RPCs
without going through any sockets
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795636#action_12795636 ]
Doug Cutting commented on AVRO-271:
-----------------------------------
> the problem is that readBuffers() and writeBuffers() shouldn't be public.
These were intended to be used by servers to read requests and write responses. But my hope of writing a generic server that's passed a Transceiver hasn't worked out in practice, even when servers process requests synchronously, and it definitely wouldn't work for an async server. So removing readBuffers() and writeBuffers() altogether might make sense, and declaring that Transceiver is for client-side use only.
InProcessTransceiver can simply implement transceive() to directly invoke Responder#respond().
Also, what do you think of renaming InProcessTransceiver to be LocalTransceiver?
> InProcessTranceiver: connect RPCs without going through any sockets
> -------------------------------------------------------------------
>
> Key: AVRO-271
> URL: https://issues.apache.org/jira/browse/AVRO-271
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: AVRO-271.patch.txt
>
>
> For testing (both Avro itself, and code that uses
> Avro servers) it's sometimes handy to connect the RPCs without
> even a socket. This implementation of a Transceiver does just that.
> (This can, for users, both avoid the overhead of using sockets, and,
> in debugging, let users see stack traces that have both client
> and server code in one thread.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-271) InProcessTranceiver: connect RPCs
without going through any sockets
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795469#action_12795469 ]
Philip Zeyliger commented on AVRO-271:
--------------------------------------
bq. A proxy should be thread safe, no?
I agree.
Thinking about this a bit more, I think the problem is that readBuffers() and writeBuffers() shouldn't be public. If they're public, there's no way to correlate two concurrent requests with the correct buffers. tranceive() works fine, because it's a single method, and the Transceiver might be doing the coordination inside.
-- Philip
> InProcessTranceiver: connect RPCs without going through any sockets
> -------------------------------------------------------------------
>
> Key: AVRO-271
> URL: https://issues.apache.org/jira/browse/AVRO-271
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: AVRO-271.patch.txt
>
>
> For testing (both Avro itself, and code that uses
> Avro servers) it's sometimes handy to connect the RPCs without
> even a socket. This implementation of a Transceiver does just that.
> (This can, for users, both avoid the overhead of using sockets, and,
> in debugging, let users see stack traces that have both client
> and server code in one thread.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-271) InProcessTranceiver: connect RPCs
without going through any sockets
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795463#action_12795463 ]
Doug Cutting commented on AVRO-271:
-----------------------------------
> Am I right that the interface would need to be changed to support a single transceiver being used to process multiple RPCs concurrently?
That's not the intent. It's intended to be bound to a particular destination, and proabably (when we get our heads around authentication) a particular user, but that user might reasonably have multiple requests outstanding to that destination using a single transceiver. In particular, a client proxy object has a single transceiver. A proxy should be thread safe, no?
> InProcessTranceiver: connect RPCs without going through any sockets
> -------------------------------------------------------------------
>
> Key: AVRO-271
> URL: https://issues.apache.org/jira/browse/AVRO-271
> Project: Avro
> Issue Type: New Feature
> Components: java
> Reporter: Philip Zeyliger
> Assignee: Philip Zeyliger
> Attachments: AVRO-271.patch.txt
>
>
> For testing (both Avro itself, and code that uses
> Avro servers) it's sometimes handy to connect the RPCs without
> even a socket. This implementation of a Transceiver does just that.
> (This can, for users, both avoid the overhead of using sockets, and,
> in debugging, let users see stack traces that have both client
> and server code in one thread.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.