[GitHub] [arrow] HedgehogCode commented on pull request #8949: ARROW-10880: [Java] Support compressing RecordBatch IPC buffers by LZ4

[GitBox <gi...@apache.org>]

HedgehogCode commented on pull request #8949:
URL: https://github.com/apache/arrow/pull/8949#issuecomment-754727763


   When I use the changes and try to compress and decompress an empty buffer (by using a variable sized vector with only missing values) I get a SIGSEGV:
   ```
   #
   # A fatal error has been detected by the Java Runtime Environment:
   #
   #  SIGSEGV (0xb) at pc=0x00007f1209448fd0, pid=10504, tid=0x00007f1208bc7700
   #
   # JRE version: OpenJDK Runtime Environment (8.0_275-b01) (build 1.8.0_275-b01)
   # Java VM: OpenJDK 64-Bit Server VM (25.275-b01 mixed mode linux-amd64 compressed oops)
   # Problematic frame:
   # V  [libjvm.so+0x6fdfd0]  jni_ThrowNew+0xc0
   #
   # Core dump written. Default location: /workspaces/arrow/java/vector/core or core.10504
   ```
   
   This can be reproduced by adding the following test to `TestCompressionCodec.java`:
   ```java
     @Test
     public void testEmptyBuffer() throws Exception {
       final int vecLength = 10;
       final VarBinaryVector origVec = new VarBinaryVector("vec", allocator);
   
       origVec.allocateNew(vecLength);
   
       // Do not set any values (all missing)
       origVec.setValueCount(vecLength);
   
       final List<ArrowBuf> origBuffers = origVec.getFieldBuffers();
       final List<ArrowBuf> compressedBuffers = compressBuffers(origBuffers);
       final List<ArrowBuf> decompressedBuffers = deCompressBuffers(compressedBuffers);
   
       // TODO assert that the decompressed buffers are correct
       AutoCloseables.close(decompressedBuffers);
     }
   ```
   This looks like an error in the lz4-java library but I am not sure. I thought I should mention it here first.
   (Note that I am using OpenJDK 8 and I haven't tried OpenJDK 11 yet)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org