You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Sharad Agarwal (JIRA)" <ji...@apache.org> on 2009/06/25 14:00:07 UTC

[jira] Created: (AVRO-62) Canonicalize protocol string before computing MD5

Canonicalize protocol string before computing MD5
-------------------------------------------------

                 Key: AVRO-62
                 URL: https://issues.apache.org/jira/browse/AVRO-62
             Project: Avro
          Issue Type: Improvement
            Reporter: Sharad Agarwal


Creating MD5 from non-canonical protocol string results in extra round trip and requires schema transfer over wire while doing inter language rpc. The protocol string should be canonicalized before computing MD5. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-62) Canonicalize protocol string before computing MD5

Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724427#action_12724427 ] 

Sharad Agarwal commented on AVRO-62:
------------------------------------

bq. For short-lived clients, this cache should be disk-based.
makes sense. Marking as Won't fix.

> Canonicalize protocol string before computing MD5
> -------------------------------------------------
>
>                 Key: AVRO-62
>                 URL: https://issues.apache.org/jira/browse/AVRO-62
>             Project: Avro
>          Issue Type: Improvement
>            Reporter: Sharad Agarwal
>
> Creating MD5 from non-canonical protocol string results in extra round trip and requires schema transfer over wire while doing inter language rpc. The protocol string should be canonicalized before computing MD5. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-62) Canonicalize protocol string before computing MD5

Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724062#action_12724062 ] 

Sharad Agarwal commented on AVRO-62:
------------------------------------

The canonical form could be:
1. Off course no spaces, newlines. tabs etc
2. Type and message definitions in the same order as defined in the avpr file. Types are defined using array, so those are already ordered. Messages are defined in map, so parser have to retain the order. 
If that is too much for some parsers, how about alphabetical ordering by message names ?

> Canonicalize protocol string before computing MD5
> -------------------------------------------------
>
>                 Key: AVRO-62
>                 URL: https://issues.apache.org/jira/browse/AVRO-62
>             Project: Avro
>          Issue Type: Improvement
>            Reporter: Sharad Agarwal
>
> Creating MD5 from non-canonical protocol string results in extra round trip and requires schema transfer over wire while doing inter language rpc. The protocol string should be canonicalized before computing MD5. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (AVRO-62) Canonicalize protocol string before computing MD5

Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AVRO-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724148#action_12724148 ] 

Doug Cutting commented on AVRO-62:
----------------------------------

If the client caches the server's protocol, the extra round-trip is only required the first time a given client connects.  For short-lived clients, this cache should be disk-based.

If folks are using different JSON libraries, canonicalization may be difficult to implement, since, besides whitespace, strings can be escaped in different ways and floating point exponents have multiple forms (e.g., 1E1, 1e1, 1E+1, 1.0E1, 1.0e1, 1.0E+1 are all the same number).

Note that the protocol is careful to ensure that the hash is always computed by the owner of a protocol.  So, in fact, we currently don't even rely on both sides even using the same hash function.  If they do use the same hash function, and they use the same JSON format, then a client's first round-trip can be skipped, but we should not rely on this for performance, since we want clients whose protocols do not match to perform well too.  A long-lived cache is thus a more general route to good performance.


> Canonicalize protocol string before computing MD5
> -------------------------------------------------
>
>                 Key: AVRO-62
>                 URL: https://issues.apache.org/jira/browse/AVRO-62
>             Project: Avro
>          Issue Type: Improvement
>            Reporter: Sharad Agarwal
>
> Creating MD5 from non-canonical protocol string results in extra round trip and requires schema transfer over wire while doing inter language rpc. The protocol string should be canonicalized before computing MD5. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (AVRO-62) Canonicalize protocol string before computing MD5

Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AVRO-62?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sharad Agarwal resolved AVRO-62.
--------------------------------

    Resolution: Won't Fix

> Canonicalize protocol string before computing MD5
> -------------------------------------------------
>
>                 Key: AVRO-62
>                 URL: https://issues.apache.org/jira/browse/AVRO-62
>             Project: Avro
>          Issue Type: Improvement
>            Reporter: Sharad Agarwal
>
> Creating MD5 from non-canonical protocol string results in extra round trip and requires schema transfer over wire while doing inter language rpc. The protocol string should be canonicalized before computing MD5. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.