[GitHub] [nifi] markap14 commented on a change in pull request #3801: NIFI-6760: When writing/reading the length of a DataFrame, do so usin…

[GitBox <gi...@apache.org>]

markap14 commented on a change in pull request #3801: NIFI-6760: When writing/reading the length of a DataFrame, do so usin…
URL: https://github.com/apache/nifi/pull/3801#discussion_r333535006
 
 

 ##########
 File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/clustered/client/async/nio/LoadBalanceSession.java
 ##########
 @@ -319,16 +319,15 @@ private ByteBuffer getFlowFileContent() throws IOException {
                 final byte[] compressed = compressDataFrame(byteBuffer, bytesRead);
                 final int compressedMaxLen = compressed.length;
 
-                buffer = ByteBuffer.allocate(3 + compressedMaxLen);
+                buffer = ByteBuffer.allocate(5 + compressedMaxLen);
                 buffer.put((byte) LoadBalanceProtocolConstants.DATA_FRAME_FOLLOWS);
-                buffer.putShort((short) compressedMaxLen);
 
 Review comment:
   The short was used because we knew that the data frame wouldn't be larger than 64 KB, so an unsigned short is sufficient for the length. The problem is that after compressing the content of the frame, if the data in the frame does not compress well, the size can actually exceed 64 KB, so the unsigned short is not large enough to convey the actual size. So it needed to be enlarged to a 32-bit int instead.
   It is true that the wire format changes here. However, that should be okay because NiFi does not currently support running a cluster where different nodes are running different versions of nifi - they must all be the same version. Since this is used only for communication between nodes in the same cluster, we know that they all will be running the same version & hence the same wire format will be used.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services