You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Roberto Diaz <rd...@gmail.com> on 2022/08/31 13:25:48 UTC
Flatbuffers vulnerability and arrow
Hello dear Arrow user members.
I am writting to you due to this reported flatbuffers vulnerability:
https://github.com/advisories/GHSA-3jch-9qgp-4844
It seems there is not fix up yo now.
I wonder if this is affecting apache arrow somehow.. i can read the
following:
https://arrow.apache.org/faq/
The Arrow file format does use Flatbuffers under the hood to serialize
schemas and other metadata needed to implement the Arrow binary IPC
protocol, but the Arrow data format uses its own representation for optimal
access and computation.
I wonder if someone has analysed this vulnerability for apache arrow.
Thank you very much and best regards
Roberto.
Re: Flatbuffers vulnerability and arrow
Posted by Antoine Pitrou <an...@python.org>.
On Wed, 31 Aug 2022 17:33:24 +0200
Roberto Diaz <rd...@gmail.com> wrote:
> Ok thank you very much.. I guess that if Arrow uses flatbuffers to
> serialize schemas an metadata for IPC that is going to come always from a
> trusted source (from Arrow itself I guess) so no security risks here.
It shouldn't need to be trusted.
Arrow C++ has fuzzing in place to ensure that invalid IPC input cannot
produce malicious behaviour. I cannot speak for the Arrow Rust
implementation, though.
Regards
Antoine.
>
> Thank you very much!
>
> Roberto.
>
> El mié., 31 ago. 2022 16:04, Andrew Lamb <al...@influxdata.com> escribió:
>
> > This advisory is related to the Rust implementation of Arrow. I do not
> > think there are any exploitable vulnerabilities in arrow due to the
> > underlying flatbuffers dependency.
> >
> > The TLDR is that if an application accepts data that claims to be in the
> > Arrow in memory format from an untrusted source, it is a security risk: The
> > arrow implementation does not do additional validation (including on the
> > embedded flatbuffers) for performance reasons.
> >
> > The way one signals this "the input data must be trusted" property in Rust
> > is to mark the API as `unsafe`. The arrow-rs API marks APIs for reading
> > Arrow from external formats as unsafe (and please let us know if you find
> > any examples to the contrary)
> >
> > The specific vulnerability reported basically says that the underlying
> > flatbuffers APIs are not correctly marked as unsafe thus learning to
> > potentially masked vulnerabilities if users take input from trusted sources.
> >
> > Andrew
> >
> > On Wed, Aug 31, 2022 at 9:26 AM Roberto Diaz <rd...@gmail.com>
> > wrote:
> >
> >> Hello dear Arrow user members.
> >>
> >> I am writting to you due to this reported flatbuffers vulnerability:
> >>
> >>
> >> https://github.com/advisories/GHSA-3jch-9qgp-4844
> >>
> >> It seems there is not fix up yo now.
> >>
> >> I wonder if this is affecting apache arrow somehow.. i can read the
> >> following:
> >>
> >> https://arrow.apache.org/faq/
> >>
> >> The Arrow file format does use Flatbuffers under the hood to serialize
> >> schemas and other metadata needed to implement the Arrow binary IPC
> >> protocol, but the Arrow data format uses its own representation for optimal
> >> access and computation.
> >>
> >> I wonder if someone has analysed this vulnerability for apache arrow.
> >>
> >> Thank you very much and best regards
> >>
> >> Roberto.
> >>
> >
>
Re: Flatbuffers vulnerability and arrow
Posted by Roberto Diaz <rd...@gmail.com>.
In fact I am using spark-sql which is
using Arrow so I am not quite sure since I am not using It directly.
Thank you!
Roberto
El mié., 31 ago. 2022 17:45, Andrew Lamb <al...@influxdata.com> escribió:
> There are some cases (e.g. arrow flight, and the .arrow / IPC file format)
> that can come from other sources, but that depends on your usecase
>
> On Wed, Aug 31, 2022 at 11:35 AM Roberto Diaz <rd...@gmail.com>
> wrote:
>
>> Ok thank you very much.. I guess that if Arrow uses flatbuffers to
>> serialize schemas an metadata for IPC that is going to come always from a
>> trusted source (from Arrow itself I guess) so no security risks here.
>>
>> Thank you very much!
>>
>> Roberto.
>>
>> El mié., 31 ago. 2022 16:04, Andrew Lamb <al...@influxdata.com> escribió:
>>
>>> This advisory is related to the Rust implementation of Arrow. I do not
>>> think there are any exploitable vulnerabilities in arrow due to the
>>> underlying flatbuffers dependency.
>>>
>>> The TLDR is that if an application accepts data that claims to be in the
>>> Arrow in memory format from an untrusted source, it is a security risk: The
>>> arrow implementation does not do additional validation (including on the
>>> embedded flatbuffers) for performance reasons.
>>>
>>> The way one signals this "the input data must be trusted" property in
>>> Rust is to mark the API as `unsafe`. The arrow-rs API marks APIs for
>>> reading Arrow from external formats as unsafe (and please let us know if
>>> you find any examples to the contrary)
>>>
>>> The specific vulnerability reported basically says that the underlying
>>> flatbuffers APIs are not correctly marked as unsafe thus learning to
>>> potentially masked vulnerabilities if users take input from trusted sources.
>>>
>>> Andrew
>>>
>>> On Wed, Aug 31, 2022 at 9:26 AM Roberto Diaz <rd...@gmail.com>
>>> wrote:
>>>
>>>> Hello dear Arrow user members.
>>>>
>>>> I am writting to you due to this reported flatbuffers vulnerability:
>>>>
>>>>
>>>> https://github.com/advisories/GHSA-3jch-9qgp-4844
>>>>
>>>> It seems there is not fix up yo now.
>>>>
>>>> I wonder if this is affecting apache arrow somehow.. i can read the
>>>> following:
>>>>
>>>> https://arrow.apache.org/faq/
>>>>
>>>> The Arrow file format does use Flatbuffers under the hood to serialize
>>>> schemas and other metadata needed to implement the Arrow binary IPC
>>>> protocol, but the Arrow data format uses its own representation for optimal
>>>> access and computation.
>>>>
>>>> I wonder if someone has analysed this vulnerability for apache arrow.
>>>>
>>>> Thank you very much and best regards
>>>>
>>>> Roberto.
>>>>
>>>
Re: Flatbuffers vulnerability and arrow
Posted by Andrew Lamb <al...@influxdata.com>.
There are some cases (e.g. arrow flight, and the .arrow / IPC file format)
that can come from other sources, but that depends on your usecase
On Wed, Aug 31, 2022 at 11:35 AM Roberto Diaz <rd...@gmail.com> wrote:
> Ok thank you very much.. I guess that if Arrow uses flatbuffers to
> serialize schemas an metadata for IPC that is going to come always from a
> trusted source (from Arrow itself I guess) so no security risks here.
>
> Thank you very much!
>
> Roberto.
>
> El mié., 31 ago. 2022 16:04, Andrew Lamb <al...@influxdata.com> escribió:
>
>> This advisory is related to the Rust implementation of Arrow. I do not
>> think there are any exploitable vulnerabilities in arrow due to the
>> underlying flatbuffers dependency.
>>
>> The TLDR is that if an application accepts data that claims to be in the
>> Arrow in memory format from an untrusted source, it is a security risk: The
>> arrow implementation does not do additional validation (including on the
>> embedded flatbuffers) for performance reasons.
>>
>> The way one signals this "the input data must be trusted" property in
>> Rust is to mark the API as `unsafe`. The arrow-rs API marks APIs for
>> reading Arrow from external formats as unsafe (and please let us know if
>> you find any examples to the contrary)
>>
>> The specific vulnerability reported basically says that the underlying
>> flatbuffers APIs are not correctly marked as unsafe thus learning to
>> potentially masked vulnerabilities if users take input from trusted sources.
>>
>> Andrew
>>
>> On Wed, Aug 31, 2022 at 9:26 AM Roberto Diaz <rd...@gmail.com>
>> wrote:
>>
>>> Hello dear Arrow user members.
>>>
>>> I am writting to you due to this reported flatbuffers vulnerability:
>>>
>>>
>>> https://github.com/advisories/GHSA-3jch-9qgp-4844
>>>
>>> It seems there is not fix up yo now.
>>>
>>> I wonder if this is affecting apache arrow somehow.. i can read the
>>> following:
>>>
>>> https://arrow.apache.org/faq/
>>>
>>> The Arrow file format does use Flatbuffers under the hood to serialize
>>> schemas and other metadata needed to implement the Arrow binary IPC
>>> protocol, but the Arrow data format uses its own representation for optimal
>>> access and computation.
>>>
>>> I wonder if someone has analysed this vulnerability for apache arrow.
>>>
>>> Thank you very much and best regards
>>>
>>> Roberto.
>>>
>>
Re: Flatbuffers vulnerability and arrow
Posted by Roberto Diaz <rd...@gmail.com>.
Ok thank you very much.. I guess that if Arrow uses flatbuffers to
serialize schemas an metadata for IPC that is going to come always from a
trusted source (from Arrow itself I guess) so no security risks here.
Thank you very much!
Roberto.
El mié., 31 ago. 2022 16:04, Andrew Lamb <al...@influxdata.com> escribió:
> This advisory is related to the Rust implementation of Arrow. I do not
> think there are any exploitable vulnerabilities in arrow due to the
> underlying flatbuffers dependency.
>
> The TLDR is that if an application accepts data that claims to be in the
> Arrow in memory format from an untrusted source, it is a security risk: The
> arrow implementation does not do additional validation (including on the
> embedded flatbuffers) for performance reasons.
>
> The way one signals this "the input data must be trusted" property in Rust
> is to mark the API as `unsafe`. The arrow-rs API marks APIs for reading
> Arrow from external formats as unsafe (and please let us know if you find
> any examples to the contrary)
>
> The specific vulnerability reported basically says that the underlying
> flatbuffers APIs are not correctly marked as unsafe thus learning to
> potentially masked vulnerabilities if users take input from trusted sources.
>
> Andrew
>
> On Wed, Aug 31, 2022 at 9:26 AM Roberto Diaz <rd...@gmail.com>
> wrote:
>
>> Hello dear Arrow user members.
>>
>> I am writting to you due to this reported flatbuffers vulnerability:
>>
>>
>> https://github.com/advisories/GHSA-3jch-9qgp-4844
>>
>> It seems there is not fix up yo now.
>>
>> I wonder if this is affecting apache arrow somehow.. i can read the
>> following:
>>
>> https://arrow.apache.org/faq/
>>
>> The Arrow file format does use Flatbuffers under the hood to serialize
>> schemas and other metadata needed to implement the Arrow binary IPC
>> protocol, but the Arrow data format uses its own representation for optimal
>> access and computation.
>>
>> I wonder if someone has analysed this vulnerability for apache arrow.
>>
>> Thank you very much and best regards
>>
>> Roberto.
>>
>
Re: Flatbuffers vulnerability and arrow
Posted by Andrew Lamb <al...@influxdata.com>.
This advisory is related to the Rust implementation of Arrow. I do not
think there are any exploitable vulnerabilities in arrow due to the
underlying flatbuffers dependency.
The TLDR is that if an application accepts data that claims to be in the
Arrow in memory format from an untrusted source, it is a security risk: The
arrow implementation does not do additional validation (including on the
embedded flatbuffers) for performance reasons.
The way one signals this "the input data must be trusted" property in Rust
is to mark the API as `unsafe`. The arrow-rs API marks APIs for reading
Arrow from external formats as unsafe (and please let us know if you find
any examples to the contrary)
The specific vulnerability reported basically says that the underlying
flatbuffers APIs are not correctly marked as unsafe thus learning to
potentially masked vulnerabilities if users take input from trusted sources.
Andrew
On Wed, Aug 31, 2022 at 9:26 AM Roberto Diaz <rd...@gmail.com> wrote:
> Hello dear Arrow user members.
>
> I am writting to you due to this reported flatbuffers vulnerability:
>
>
> https://github.com/advisories/GHSA-3jch-9qgp-4844
>
> It seems there is not fix up yo now.
>
> I wonder if this is affecting apache arrow somehow.. i can read the
> following:
>
> https://arrow.apache.org/faq/
>
> The Arrow file format does use Flatbuffers under the hood to serialize
> schemas and other metadata needed to implement the Arrow binary IPC
> protocol, but the Arrow data format uses its own representation for optimal
> access and computation.
>
> I wonder if someone has analysed this vulnerability for apache arrow.
>
> Thank you very much and best regards
>
> Roberto.
>