[jira] [Commented] (ARROW-2222) [C++] Add option to validate Flatbuffers messages

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-2222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403422#comment-16403422 ] 

ASF GitHub Bot commented on ARROW-2222:
---------------------------------------

crepererum opened a new pull request #1763: ARROW-2222: handle untrusted inputs (POC)
URL: https://github.com/apache/arrow/pull/1763
 
 
   This is a proof of concept of handling untrusted inputs. The error messages aren't filled yet and tests aren't there, but this is everything the fuzzer found after running over an hour. It basically includes the following parts:
   
   ## Flatbuffer validation
   This may be the only thing that could be optional and where we should test the performance impact. It's a single, simple change in `Message::ReadFrom`.
   
   ## nullptr validation
   It seems, that even when we validate the flatbuffer, nullptrs are still legit in many places and we just ignore them. I've added a bunch of checks. We may want to introduce a simple macro for that, that includes error message generation and the return w/ an `IOError`.
   
   ## Integer handling in `BufferReader`
   The `nbytes` argument can be negative since it's a signed integer. I've seen at least one case where the fuzzer tried to produce a negative body size of a message that led to a negative `nbytes` argument, a subsequent negative `position_` and a out of bounds read. I think we should catch the negative body length somewhere else and produce a proper error message, but I'm not sure if that's the only way of provoking "negative reads". For robustness, the checks should be there. If someone is to worried about performance, we should rather implement a template-based, header-based, function-inlined interface that reads exactly the number of bytes that are required for certain types.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> [C++] Add option to validate Flatbuffers messages
> -------------------------------------------------
>
>                 Key: ARROW-2222
>                 URL: https://issues.apache.org/jira/browse/ARROW-2222
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>            Reporter: Wes McKinney
>            Priority: Major
>              Labels: pull-request-available
>
> This is follow up work to ARROW-1589, ARROW-2023, and can be validated by the {{ipc-fuzzer-test}}. Users receiving untrusted input streams can prevent segfaults this way
> As part of this, we should quantify the overhead associated with message validation in regular use



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)