[GitHub] [arrow-julia] DrChainsaw commented on a diff in pull request #436: Add handling of len = -1 in uncompress

["DrChainsaw (via GitHub)" <gi...@apache.org>]

DrChainsaw commented on code in PR #436:
URL: https://github.com/apache/arrow-julia/pull/436#discussion_r1192989113


##########
src/table.jl:
##########
@@ -521,6 +521,11 @@ function uncompress(ptr::Ptr{UInt8}, buffer, compression)
     len = unsafe_load(convert(Ptr{Int64}, ptr))
     ptr += 8 # skip past uncompressed length as Int64
     encodedbytes = unsafe_wrap(Array, ptr, buffer.length - 8)
+    if len === -1

Review Comment:
   I don't know the logic why the writer would do this. I encountered the issue when trying to read files generated from the official java arrow package. When stepping through the debugger during reading in java I encountered this lines which I just copied over to the Julia implementation: https://github.com/apache/arrow/blob/febd0ff144cfb8b2baffb1cb0be57ca40dc7cc77/java/vector/src/main/java/org/apache/arrow/vector/compression/AbstractCompressionCodec.java#L72-L75
   
   In the linked issue there is an example file which also reads in pyarrow. I haven't made any effort to find the corresponding line in there though.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org