You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Antoine Pitrou <an...@python.org> on 2020/01/15 18:59:24 UTC
[C++] Arrow added to OSS-Fuzz
Hello,
I would like to announce that Arrow has been accepted on the OSS-Fuzz
infrastructure (a continuous fuzzing infrastructure operated by Google):
https://github.com/google/oss-fuzz/pull/3233
Right now the only fuzz targets are the C++ stream and file IPC readers.
The first build results haven't appeared yet. They will appear on
https://oss-fuzz.com/ . Access needs a Google account, and you also
need to be listed in the "auto_ccs" here:
https://github.com/google/oss-fuzz/blob/master/projects/arrow/project.yaml
(if you are a PMC or core developer and want to be listed, just open a
PR to the oss-fuzz repository)
Once we confirm the first builds succeed on OSS-Fuzz, we should probably
add more fuzz targets (for example for reading Parquet files).
Regards
Antoine.
Re: [C++] Arrow added to OSS-Fuzz
Posted by Antoine Pitrou <so...@pitrou.net>.
Hello,
A quick update: since Arrow C++ started being fuzzed in OSS-Fuzz, 41
issues (usually crashes) on invalid input have been found, 35 of which
have already been corrected.
We plan to expand the fuzzed areas to cover Parquet files, as well as
serialized Tensor and SparseTensor data.
Regards
Antoine.
On Wed, 15 Jan 2020 19:59:24 +0100
Antoine Pitrou <an...@python.org> wrote:
> Hello,
>
> I would like to announce that Arrow has been accepted on the OSS-Fuzz
> infrastructure (a continuous fuzzing infrastructure operated by Google):
> https://github.com/google/oss-fuzz/pull/3233
>
> Right now the only fuzz targets are the C++ stream and file IPC readers.
> The first build results haven't appeared yet. They will appear on
> https://oss-fuzz.com/ . Access needs a Google account, and you also
> need to be listed in the "auto_ccs" here:
> https://github.com/google/oss-fuzz/blob/master/projects/arrow/project.yaml
>
> (if you are a PMC or core developer and want to be listed, just open a
> PR to the oss-fuzz repository)
>
> Once we confirm the first builds succeed on OSS-Fuzz, we should probably
> add more fuzz targets (for example for reading Parquet files).
>
> Regards
>
> Antoine.
>
Re: [C++] Arrow added to OSS-Fuzz
Posted by Antoine Pitrou <an...@python.org>.
Le 17/01/2020 à 07:33, Marco Neumann a écrit :
> Hey Antoine,
>
> Thanks a lot also from my side.
>
> The build is likely currently succeeding due to the Fuzzing work done by fuzzit.
Actually, around ~30 crashes were found on the first run or so.
I think one of the important things here was to supply a seed corpus to
help the fuzzer explore the space of interesting inputs more efficiently.
Regards
Antoine.
Re: [C++] Arrow added to OSS-Fuzz
Posted by Marco Neumann <ma...@crepererum.net.INVALID>.
Hey Antoine,
Thanks a lot also from my side.
The build is likely currently succeeding due to the Fuzzing work done by fuzzit. We had loads of crashes in the beginning and fixed tons of edge cases, especially around null pointer handling.
I also have some code locally for a Parquet fuzzing stub, which I can publish (there is a ticket for that task I think). Sadly, this doesn't result in a long fuzzer run as the arrow IPC reader fuzzer and crashes after some seconds (was some issue in the thrift deserialization last time I've tried it). So I would probably not add it directly to fuzzit and OSS-fuzz to not be bombarded by error messages.
Cheers,
Marco Neumann
Jan 16, 2020, 02:23 by liya.fan03@gmail.com:
> Hi Antoine,
>
> Good job! And thanks for sharing the great news!
>
> Best,
> Liya Fan
>
> On Thu, Jan 16, 2020 at 2:59 AM Antoine Pitrou <an...@python.org> wrote:
>
>>
>> Hello,
>>
>> I would like to announce that Arrow has been accepted on the OSS-Fuzz
>> infrastructure (a continuous fuzzing infrastructure operated by Google):
>> https://github.com/google/oss-fuzz/pull/3233
>>
>> Right now the only fuzz targets are the C++ stream and file IPC readers.
>> The first build results haven't appeared yet. They will appear on
>> https://oss-fuzz.com/ . Access needs a Google account, and you also
>> need to be listed in the "auto_ccs" here:
>> https://github.com/google/oss-fuzz/blob/master/projects/arrow/project.yaml
>>
>> (if you are a PMC or core developer and want to be listed, just open a
>> PR to the oss-fuzz repository)
>>
>> Once we confirm the first builds succeed on OSS-Fuzz, we should probably
>> add more fuzz targets (for example for reading Parquet files).
>>
>> Regards
>>
>> Antoine.
>>
Re: [C++] Arrow added to OSS-Fuzz
Posted by Fan Liya <li...@gmail.com>.
Hi Antoine,
Good job! And thanks for sharing the great news!
Best,
Liya Fan
On Thu, Jan 16, 2020 at 2:59 AM Antoine Pitrou <an...@python.org> wrote:
>
> Hello,
>
> I would like to announce that Arrow has been accepted on the OSS-Fuzz
> infrastructure (a continuous fuzzing infrastructure operated by Google):
> https://github.com/google/oss-fuzz/pull/3233
>
> Right now the only fuzz targets are the C++ stream and file IPC readers.
> The first build results haven't appeared yet. They will appear on
> https://oss-fuzz.com/ . Access needs a Google account, and you also
> need to be listed in the "auto_ccs" here:
> https://github.com/google/oss-fuzz/blob/master/projects/arrow/project.yaml
>
> (if you are a PMC or core developer and want to be listed, just open a
> PR to the oss-fuzz repository)
>
> Once we confirm the first builds succeed on OSS-Fuzz, we should probably
> add more fuzz targets (for example for reading Parquet files).
>
> Regards
>
> Antoine.
>