[GitHub] [arrow] mapleFU commented on pull request #35428: GH-35423: [C++][Parquet][BugFix] Parquet Encoding Fixing mismatched buffer size

["mapleFU (via GitHub)" <gi...@apache.org>]

mapleFU commented on PR #35428:
URL: https://github.com/apache/arrow/pull/35428#issuecomment-1534221863

   The real reason is that:
   
   ```c++
   std::shared_ptr<Buffer> SerializedPageReader::DecompressIfNeeded(
       std::shared_ptr<Buffer> page_buffer, int compressed_len, int uncompressed_len,
       int levels_byte_len) {
     if (decompressor_ == nullptr) {
       return page_buffer;
     }
     if (compressed_len < levels_byte_len || uncompressed_len < levels_byte_len) {
       throw ParquetException("Invalid page header");
     }
   
     // Grow the uncompressed buffer if we need to.
     if (uncompressed_len > static_cast<int>(decompression_buffer_->size())) {
       PARQUET_THROW_NOT_OK(
           decompression_buffer_->Resize(uncompressed_len, /*shrink_to_fit=*/false));
     }
   
     if (levels_byte_len > 0) {
       // First copy the levels as-is
       uint8_t* decompressed = decompression_buffer_->mutable_data();
       memcpy(decompressed, page_buffer->data(), levels_byte_len);
     }
   
     // Decompress the values
     PARQUET_THROW_NOT_OK(decompressor_->Decompress(
         compressed_len - levels_byte_len, page_buffer->data() + levels_byte_len,
         uncompressed_len - levels_byte_len,
         decompression_buffer_->mutable_data() + levels_byte_len));
   
     return decompression_buffer_;
   }
   ```
   
   (1) Here the buffer size will only grow up, and never thrink.
   (2) Page.buffer uses this buffer size
   (3) BYTE_STREAM_SPLIT will check this size


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org