You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Doug Cutting (JIRA)" <ji...@apache.org> on 2010/01/15 21:00:54 UTC
[jira] Created: (AVRO-340) specify http transport in spec
specify http transport in spec
------------------------------
Key: AVRO-340
URL: https://issues.apache.org/jira/browse/AVRO-340
Project: Avro
Issue Type: Improvement
Components: spec
Reporter: Doug Cutting
Assignee: Doug Cutting
Fix For: 1.3.0
The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-340) specify http transport in spec
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800895#action_12800895 ]
Doug Cutting commented on AVRO-340:
-----------------------------------
The spec should also describe how URIs may be used to name Avro transports. Initially we'll only specify http: uris.
Avro should only specify transports that we intend to support long-term. We may add other transports subsequently, but, in the meantime, we should not encourage implementations of non-standard transports, as they will complicate compatibility: we do not want to have to support them long-term.
Each Avro implementation should implement clients for all transports in the spec, so that applications can talk to any Avro-based service. They need not implement servers for all server transports however.
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-340) specify http transport in spec
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803914#action_12803914 ]
Scott Carey commented on AVRO-340:
----------------------------------
I believe, but could be wrong, that both chunked transfer encoding and standard Content-Length transfer encoding is required (client side) to be HTTP 1.1 compliant, but a server can do whatever it wants.
This is all typically handled at the HTTP library level.
I have seen test HTTP server programs that can purposely send flawed transfer encoding to a client. Its easy to crash IE or Safari this way -- or at least was a couple years ago.
It may be useful to have a "evil server" and "evil client" tool to make sure implementations are robust. The same is true for any transport layer, and is one of the reasons why building your own TCP based transport is not as easy as it seems. You don't want someone to telnet to a port and take down your server.
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-340) specify http transport in spec
Posted by "Philip Zeyliger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829400#action_12829400 ]
Philip Zeyliger commented on AVRO-340:
--------------------------------------
+1 on the patch.
I think that the HttpServer implementation currently in Java maps to any URL, rather than a single specific one. I wouldn't mind the spec suggesting that the url is http://example.host:port/avro, though other URLs are acceptable.
Do we need to explicitly mention that it is acceptable to use the same socket for multiple HTTP requests (keep-alive, I believe, in HTTP terms)?
-- Philip
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
> Attachments: AVRO-340.patch
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-340) specify http transport in spec
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803612#action_12803612 ]
Jeff Hammerbacher commented on AVRO-340:
----------------------------------------
How should we handle framing in the context of HTTP transports? Use an HTTP header for body length?
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-340) specify http transport in spec
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-340:
------------------------------
Attachment: AVRO-340.patch
Here's a patch that fixes this and also addresses AVRO-342 by changing the socket and datagram transport javadoc to state that these are non-standard. It also updates the Java client to explicitly specify POST as the HTTP method.
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
> Attachments: AVRO-340.patch
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-340) specify http transport in spec
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-340:
------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this.
> I wouldn't mind the spec suggesting that the url is http://example.host:port/avro, though other URLs are acceptable.
I think there might be cases where, for security or otherwise, one might wish to use different urls for different protocols. If you feel this would be useful, can you file another issue?
> Do we need to explicitly mention that it is acceptable to use the same socket for multiple HTTP requests (keep-alive, I believe, in HTTP terms)?
I think that's covered in the HTTP spec. We don't mention connections, proxies, redirects or any of that stuff. If folks are confused, then we might add clarifications as needed.
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
> Attachments: AVRO-340.patch
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-340) specify http transport in spec
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803916#action_12803916 ]
Doug Cutting commented on AVRO-340:
-----------------------------------
HTTP chunking should be transparent to Avro. Avro defines the entire content of the request and response. Chunk boundary data should only be visible to the HTTP layer. There may be applications where aligning HTTP chunks with Avro RPC frames will prove useful, and implementations that use a chunked encoding might be wise to align these, but that's not required.
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-340) specify http transport in spec
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803898#action_12803898 ]
Jeff Hammerbacher commented on AVRO-340:
----------------------------------------
So we don't want to do http://en.wikipedia.org/wiki/Chunked_transfer_encoding?
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-340) specify http transport in spec
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803767#action_12803767 ]
Doug Cutting commented on AVRO-340:
-----------------------------------
In HTTP, the Content-Length should, if specified, indicate the length of the entire, framed, request and response. The content of each HTTP request is the framed handshake+messageName+parameters as described in the spec, and the content of each HTTP response is the framed handshake+errorFlag+responseOrError as described in the spec.
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-340) specify http transport in spec
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-340:
------------------------------
Status: Patch Available (was: Open)
> specify http transport in spec
> ------------------------------
>
> Key: AVRO-340
> URL: https://issues.apache.org/jira/browse/AVRO-340
> Project: Avro
> Issue Type: Improvement
> Components: spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 1.3.0
>
> Attachments: AVRO-340.patch
>
>
> The use of HTTP as an Avro RPC transport should be described in the Avro spec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.