[jira] [Created] (HDFS-12165) getSnapshotDiffReport throws NegativeArraySizeException for very large snapshot diff summary

["Wei-Chiu Chuang (JIRA)" <ji...@apache.org>]

Wei-Chiu Chuang created HDFS-12165:
--------------------------------------

             Summary: getSnapshotDiffReport throws NegativeArraySizeException for very large snapshot diff summary
                 Key: HDFS-12165
                 URL: https://issues.apache.org/jira/browse/HDFS-12165
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Wei-Chiu Chuang


For a really large snapshot diff, getSnapshotDiffReport throws NegativeArraySizeException
{noformat}
2017-07-19 11:14:16,415 WARN org.apache.hadoop.ipc.Server: Error serializing call response for call org.apache.hadoop.hdfs.protocol.ClientProtocol.getSnapshotDiffReport
 from 10.17.211.10:58223 Call#0 Retry#0
java.lang.NegativeArraySizeException
        at com.google.protobuf.CodedOutputStream.newInstance(CodedOutputStream.java:105)
        at com.google.protobuf.AbstractMessageLite.writeDelimitedTo(AbstractMessageLite.java:87)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$RpcResponseWrapper.write(ProtobufRpcEngine.java:468)
        at org.apache.hadoop.ipc.Server.setupResponse(Server.java:2410)
        at org.apache.hadoop.ipc.Server.access$500(Server.java:134)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2182)
{noformat}

This particular snapshot diff contains more than 25 million different file system objects, and which means the serialized response can be more than 2GB, overflowing protobuf length calculation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org