[jira] Commented: (AVRO-61) Support for blocking arrays/maps in python

["Ravi Gummadi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-61?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742742#action_12742742 ] 

Ravi Gummadi commented on AVRO-61:
----------------------------------

The testcase in this patch passes only with the patch of AVRO-88. So made AVRO-88 as a blocker for this JIRA.

> Support for blocking arrays/maps in python
> ------------------------------------------
>
>                 Key: AVRO-61
>                 URL: https://issues.apache.org/jira/browse/AVRO-61
>             Project: Avro
>          Issue Type: Bug
>          Components: python
>            Reporter: Thiruvalluvan M. G.
>            Assignee: Ravi Gummadi
>         Attachments: AVRO-61.patch
>
>
> We have introduced a blocking version of writer in Java that enables readers to efficiently skip large arrays and maps. The avro format encoded arrays and maps by encoding the number of elements by the the elements themselves. Zero element count indicates that the array/map has ended. The change we introduced is that if the element count is negative, it is followed by the byte-count of the encoded elements that follow. The reader, on seeing a negative element count should flip the sign to get the actual number of elements. In addition, if it is interested in supporting fast skip, it should use the byte count to skip the elements en-bloc instead of decoding them individually. If it does not want to support fast skip, it has to just read the byte count and ignore its value.
> The changes are already made in Java's ValueReader to support this. Similar changes need to done in Python as well.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.