[jira] [Commented] (AVRO-3929) Incompatibility with long serialized data from previous version

["Oscar Westra van Holthe - Kind (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17814740#comment-17814740 ] 

Oscar Westra van Holthe - Kind commented on AVRO-3929:
------------------------------------------------------

Actually, the new limit is also the limit the JVM imposes upon arrays. See https://bugs.openjdk.org/browse/JDK-8246725 for details.

The same JVM limit also applies to Avro arrays, as these are implemented using a JVM array as well.

And lastly, the same limit also applies to Avro maps: these are read as a java.util.HashMap, which uses a JVM array as internal data structure as well.

 

So API wise, this is indeed a breaking change. But in reality, the maximum size that was ever possible to read with the Java library is exactly this limit.

> Incompatibility with long serialized data from previous version
> ---------------------------------------------------------------
>
>                 Key: AVRO-3929
>                 URL: https://issues.apache.org/jira/browse/AVRO-3929
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.11.2
>            Reporter: Leonardo Aleixo da Silva
>            Priority: Major
>
> The new version of 1.11.2 implemented the SystemLimitException with MAX_ARRAY_VM_LIMIT to int type, but in the previous versions it was accepting long values, this make the new versions incompatible with long serialized data. Looking to the implementation this class accept the attribute in application properties that is possible to change max length for bytes (org.apache.avro.limits.bytes.maxLength) but the implementation only accept int conversions.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)