You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Andrew Lamb <al...@influxdata.com> on 2021/11/30 22:10:56 UTC
Re: [Rust] Heads up: RUSTSEC security advisory against arrow-rs
Well, I finally have a PR[1] that adds full ArrayData validation ready for
review Any thoughts or review comments would be most helpful.
Thank you,
Andrew
[1] https://github.com/apache/arrow-rs/pull/921
On Fri, Oct 29, 2021 at 6:23 AM Andrew Lamb <al...@influxdata.com> wrote:
> There is more discussion about the RUSTSEC process here:
> https://github.com/rustsec/advisory-db/issues/1092
>
> On Wed, Oct 6, 2021 at 10:52 AM Andrew Lamb <al...@influxdata.com> wrote:
>
>> I have incorporated feedback into a proposal [1] of how to handle
>> validation of arguments to ArrayData::new, and would appreciate further
>> review
>>
>> [1] https://github.com/apache/arrow-rs/issues/817
>>
>> On Fri, Oct 1, 2021 at 6:44 AM Andrew Lamb <al...@influxdata.com> wrote:
>>
>>> Thank you Antoine,
>>>
>>> The C++ validation routine is super helpful, especially with respect to
>>> validating nested structures. I will follow its lead
>>>
>>> Andrew
>>>
>>> On Fri, Oct 1, 2021 at 3:25 AM Antoine Pitrou <an...@python.org>
>>> wrote:
>>>
>>>>
>>>> In C++ we have dedicated validation routines that can be used against
>>>> untrusted input. (*) We also have fuzzing set up using OSS-Fuzz to
>>>> validate that invalid input cannot crash the IPC reader.
>>>>
>>>> (*)
>>>>
>>>> https://github.com/apache/arrow/blob/master/cpp/src/arrow/array/validate.h
>>>>
>>>>
>>>> Le 01/10/2021 à 00:13, Andrew Lamb a écrit :
>>>> > I have created a WIP PR for initial feedback on the approach of
>>>> validating
>>>> > ArrayData upon creation[1]. If there are no objections to the
>>>> approach I
>>>> > will complete the implementation over the next few days
>>>> >
>>>> > The approach that Sergey describes of `get` and `unsafe get_unchecked`
>>>> > sounds like a good one to me if performance testing shows we need a
>>>> bypass.
>>>> >
>>>> > Andrew
>>>> >
>>>> > [1] https://github.com/apache/arrow-rs/pull/810
>>>>
>>>
Re: [Rust] Heads up: RUSTSEC security advisory against arrow-rs
Posted by Andrew Lamb <al...@influxdata.com>.
To finish up this thread, the rustsec advisory has been updated:
https://github.com/rustsec/advisory-db/pull/1131
On Tue, Nov 30, 2021 at 5:10 PM Andrew Lamb <al...@influxdata.com> wrote:
> Well, I finally have a PR[1] that adds full ArrayData validation ready
> for review Any thoughts or review comments would be most helpful.
>
> Thank you,
> Andrew
>
> [1] https://github.com/apache/arrow-rs/pull/921
>
> On Fri, Oct 29, 2021 at 6:23 AM Andrew Lamb <al...@influxdata.com> wrote:
>
>> There is more discussion about the RUSTSEC process here:
>> https://github.com/rustsec/advisory-db/issues/1092
>>
>> On Wed, Oct 6, 2021 at 10:52 AM Andrew Lamb <al...@influxdata.com> wrote:
>>
>>> I have incorporated feedback into a proposal [1] of how to handle
>>> validation of arguments to ArrayData::new, and would appreciate further
>>> review
>>>
>>> [1] https://github.com/apache/arrow-rs/issues/817
>>>
>>> On Fri, Oct 1, 2021 at 6:44 AM Andrew Lamb <al...@influxdata.com> wrote:
>>>
>>>> Thank you Antoine,
>>>>
>>>> The C++ validation routine is super helpful, especially with respect to
>>>> validating nested structures. I will follow its lead
>>>>
>>>> Andrew
>>>>
>>>> On Fri, Oct 1, 2021 at 3:25 AM Antoine Pitrou <an...@python.org>
>>>> wrote:
>>>>
>>>>>
>>>>> In C++ we have dedicated validation routines that can be used against
>>>>> untrusted input. (*) We also have fuzzing set up using OSS-Fuzz to
>>>>> validate that invalid input cannot crash the IPC reader.
>>>>>
>>>>> (*)
>>>>>
>>>>> https://github.com/apache/arrow/blob/master/cpp/src/arrow/array/validate.h
>>>>>
>>>>>
>>>>> Le 01/10/2021 à 00:13, Andrew Lamb a écrit :
>>>>> > I have created a WIP PR for initial feedback on the approach of
>>>>> validating
>>>>> > ArrayData upon creation[1]. If there are no objections to the
>>>>> approach I
>>>>> > will complete the implementation over the next few days
>>>>> >
>>>>> > The approach that Sergey describes of `get` and `unsafe
>>>>> get_unchecked`
>>>>> > sounds like a good one to me if performance testing shows we need a
>>>>> bypass.
>>>>> >
>>>>> > Andrew
>>>>> >
>>>>> > [1] https://github.com/apache/arrow-rs/pull/810
>>>>>
>>>>