You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2025/11/12 09:40:15 UTC

[I] [C++][Parquet][IPC] Use capped memory pool when fuzzing [arrow]

pitrou opened a new issue, #48105:
URL: https://github.com/apache/arrow/issues/48105

   ### Describe the enhancement requested
   
   OSS-Fuzz will trigger an out-of-memory crash if the allocated memory goes beyond a predefined limit (usually 2560 MB, though that can be configured). For Parquet and IPC, it is legitimate to allocate a lot of memory when decompressing data, though, so that can happen on both valid and invalid input files.
   
   Unfortunately, OSS-Fuzz checks for this memory limit *not* by instrumenting `malloc` and having it return NULL when the limit is reached, but by checking allocated memory periodically from a separate thread. This can be solved by implementing our custom allocator with an upper limit, exactly how the mupdf project did in https://github.com/google/oss-fuzz/issues/1830
   
   
   
   ### Component(s)
   
   C++, Continuous Integration, Parquet


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++][Parquet][IPC] Use capped memory pool when fuzzing [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #48105:
URL: https://github.com/apache/arrow/issues/48105#issuecomment-3536319583

   Issue resolved by pull request 48108
   https://github.com/apache/arrow/pull/48108


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++][Parquet][IPC] Use capped memory pool when fuzzing [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #48105: [C++][Parquet][IPC] Use capped memory pool when fuzzing
URL: https://github.com/apache/arrow/issues/48105


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org