OSS-Fuzz

[Julian Hyde <jh...@apache.org>]

Someone from Google logged a case offering to add Calcite to the
OSS-Fuzz program. (I work for Google but was not aware that we were
being considered.)

https://issues.apache.org/jira/browse/CALCITE-5781

How do people feel about participating in this program?

I think that it could improve our security significantly, but it will
take work. The fuzzer might generate a lot of false negatives. It
might also generate quite a few genuine security issues that we will
need to respond to appropriately. As an all-volunteer project it might
put a strain on us.

Julian

Re: OSS-Fuzz

[Stamatis Zampetakis <za...@gmail.com>]

I had a quick look at the OSS-Fuzz project [1] and I get the
impression that it is not only security oriented but a general
framework for fuzzy testing components.

I am sure that fuzzy testing can uncover many bugs (especially small
ones) so it's worth having I guess. However, receiving notifications
or creating tickets for every problem might be too much. Currently,
it's hard to keep up with JIRAs and PRs created by humans so not sure
if getting more bug reports will really improve the quality of the
project.

For the record, we have some basic fuzzy testing in Calcite already
[2]. Currently it is mostly disabled and not used much but I remember
that it was pretty efficient in identifying problems in Rex-land.

All-in-all good I like the idea but I will probably not have time to
look into every single bug report that comes in from the automation
tool. If it could be configured to run on PRs and "attack" the new
code that is getting in, that would be really helpful and the load
would be more evenly distributed.

Best,
Stamatis

[1] https://github.com/google/oss-fuzz
[2] https://github.com/apache/calcite/blob/3f2ae2f4dd2d6b1fab7c3a91e67a6a6d28523298/core/src/test/java/org/apache/calcite/test/fuzzer/RexProgramFuzzyTest.java#L356

On Fri, Jun 16, 2023 at 8:37 PM Michael Mior <mm...@apache.org> wrote:
>
> Thanks for sharing Julian!
>
> Do we *need* to respond to security issues that are uncovered? I certainly
> agree that we *should* if at all possible. But by choosing not to
> participate, we would be choosing not to respond to *all* security issues
> that might only be uncovered via fuzzing. It seems reasonable to me
> (assuming any discovered vulnerabilities can be kept private), that we
> should be free to ignore issues that are uncovered.
>
> --
> Michael Mior
> mmior@apache.org
>
>
> On Fri, Jun 16, 2023 at 2:31 PM Julian Hyde <jh...@apache.org> wrote:
>
> > Someone from Google logged a case offering to add Calcite to the
> > OSS-Fuzz program. (I work for Google but was not aware that we were
> > being considered.)
> >
> > https://issues.apache.org/jira/browse/CALCITE-5781
> >
> > How do people feel about participating in this program?
> >
> > I think that it could improve our security significantly, but it will
> > take work. The fuzzer might generate a lot of false negatives. It
> > might also generate quite a few genuine security issues that we will
> > need to respond to appropriately. As an all-volunteer project it might
> > put a strain on us.
> >
> > Julian
> >

Re: OSS-Fuzz

[Michael Mior <mm...@apache.org>]

Thanks for sharing Julian!

Do we *need* to respond to security issues that are uncovered? I certainly
agree that we *should* if at all possible. But by choosing not to
participate, we would be choosing not to respond to *all* security issues
that might only be uncovered via fuzzing. It seems reasonable to me
(assuming any discovered vulnerabilities can be kept private), that we
should be free to ignore issues that are uncovered.

--
Michael Mior
mmior@apache.org


On Fri, Jun 16, 2023 at 2:31 PM Julian Hyde <jh...@apache.org> wrote:

> Someone from Google logged a case offering to add Calcite to the
> OSS-Fuzz program. (I work for Google but was not aware that we were
> being considered.)
>
> https://issues.apache.org/jira/browse/CALCITE-5781
>
> How do people feel about participating in this program?
>
> I think that it could improve our security significantly, but it will
> take work. The fuzzer might generate a lot of false negatives. It
> might also generate quite a few genuine security issues that we will
> need to respond to appropriately. As an all-volunteer project it might
> put a strain on us.
>
> Julian
>