You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "LucaCappelletti94 (via GitHub)" <gi...@apache.org> on 2026/09/18 17:21:36 UTC
[PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
LucaCappelletti94 opened a new pull request, #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527
Step 2 of #2519, modelled on diesel-rs/diesel#5212.
Adds the [ClusterFuzzLite](https://google.github.io/clusterfuzzlite/) configuration and four workflows for the existing `fuzz_parse_sql` target.
Every pull request fuzzes for 10 minutes in code-change mode, pushes to main refresh a runner-cached build, and a daily batch at 06:00 UTC plus a corpus prune at 05:00 keep the shared corpus on the clusterfuzzlite branch of this repository using the default GITHUB_TOKEN.
Given this is the first time we actually start to fuzz the code automatically and the various PRs, I expect some blood bath of bugs will ensue, but that is sort of the point. We will add more fuzz harnesses over time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on code in PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#discussion_r4061587344
##########
.github/workflows/cflite_build.yml:
##########
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: ClusterFuzzLite continuous builds
Review Comment:
This particular workflow is solely focused on ensuring the fuzzer harnesses keep building. It does not run the harnesses themselves.
It additionally works alongside the PR fuzzing job as it is the harness built that is used for the differential fuzzing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 merged PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5765377845
Basically here if I understood how it works, you have:
1) the [.github/workflows/cflite_build.yml](https://github.com/apache/datafusion-sqlparser-rs/pull/2527/files/e85e2c50668af825c5f528ee1c71405f319d0962#diff-6a380eaf4eb765b7f8021b58c945424b0cef11356e166e1a5b223e6792572bac) builds the main fuzzer harness, our `g`
2) a PR comes along, and triggers the [.github/workflows/cflite_pr.yml](https://github.com/apache/datafusion-sqlparser-rs/pull/2527/files/e85e2c50668af825c5f528ee1c71405f319d0962#diff-901d28e78554ed027d6845dc00bca07d9fa96a70159d9d5db422de24de389f46) workflow, and builds the PR branch fuzzer harness, our `f`
3) When, as in https://github.com/apache/datafusion-sqlparser-rs/pull/2530 there is no `g` compatible to `f`, `g` collapses to be an infallible function
4) The fuzzer starts generating inputs, feeds `f(x)`, if `f(x)` crashes it checks `g(x)`. If the latter crashes, the bug is old, otherwise it is new.
So for any given fuzzer iteration, `f`, `g`, and `x` are given by that iteration of the fuzzer `h`, then it will produce `x'` and iterate onward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 closed pull request #2527: Fuzz: Wire up ClusterFuzzLite CI fuzzing
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5759786313
I will set up analogous workflows in a test repo to verify whether the PR workflow behaves as I described, just in case I either (a) copied the wrong documentation examples / configured them poorly (b) misunderstood anything of the documentation.
I believe it works as advertised but given it is a relatively new tool for me (I generally just fuzz locally) I want to make extra sure.
If it goes red only when the bug is new, that would be exactly what we want and should put the issue of having random red CIs in PR at rest, I suppose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5765614811
Thank you for the lesson @LucaCappelletti94 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5762665359
> If it goes red only when the bug is new, that would be exactly what we want and should put the issue of having random red CIs in PR at rest, I suppose.
I think the gate should also be about false positives (not just true negatives)
So in theory I would hope our existing process / new unit tests would cover regressions and then the fuzzing is a net new set of tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5765048520
> > > I think the gate should also be about false positives (not just true negatives)
> >
> >
> > What do you mean by false positives?
>
> I mean the chance of the fuzz test failing on a PR that did not cause whatever problem the fuzzer found.
>
> I think if we run non deterministic tests on PRs we are going to inject some noise into the process -- I would recommend against it if at all possible (at least until we have some confidence that the fuzz test is mostly successful -- e.g. run it for a week or more every day and not find anything new)
I am not sure how that would occur, suppose the PR compiles to `f` and the main is compiled to `g`. For a given fuzzer input `x`, if `f(s)` crashes and `g(x)` does not, there is no non-determinism. The only non-deterministic aspect is that a failing input `x` might not be found in the limited 10 minutes window.
The scenario you describe where `f(x)` crashes and `g(x)` does not but it should can only happen if I configured ClusterFuzzLite poorly, if I understood it correctly.
To be clear, I **perfectly agree** that if the CI starts to randomly give out bugs to PRs that have nothing to do with that bug it would be something extremely problematic and to be absolutely avoided. If it happens to be the case, I would suggest to only keep the periodic fuzz job.
That being said, again if I understood correctly its documentation, that should not be the case as it would make ClusterFuzzLite principal use case be nigh useless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on code in PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#discussion_r4061557373
##########
.github/workflows/cflite_pr.yml:
##########
@@ -0,0 +1,48 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: ClusterFuzzLite PR fuzzing
Review Comment:
My hope was that, as the fuzzer stabilizes over time, the bugs that get raised by a PR fuzzing run are exactly bugs of the PR, and this should simplify a lot the job of reviewers.
I do agree that at this time most bugs raised by the fuzzer are pre-existing things that need handling before it can be deployed on PRs as at this time yes they would be confusing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5759449686
> I expect some blood bath of bugs will ensue, but that is sort of the point.
Yes I expect this too -- but I would recommend shielding other contributors from the pain (and yourself from having to explain it to them)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on code in PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#discussion_r4061529016
##########
.github/workflows/cflite_prune.yml:
##########
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: ClusterFuzzLite corpus pruning
Review Comment:
TLDR of it, the corpus entries over time become overlapping/redundant, and we want to keep the minimum set of entries that seed the maximum coverage. It is generally done less frequently than fuzzing itself.
I will add a definition and link the documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on code in PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#discussion_r4061529016
##########
.github/workflows/cflite_prune.yml:
##########
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: ClusterFuzzLite corpus pruning
Review Comment:
TLDR of it, the corpus entries over time become overlapping/redundant, and we want to keep the minimum set of entries that seed the maximum coverage. This minimization procedure trims down the corpus by assigning to each entry a fingerprint of the coverage it produces, and then keeps sufficiently distinct ones. It is generally done less frequently than fuzzing itself.
I will add a definition and link the documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on code in PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#discussion_r4061587344
##########
.github/workflows/cflite_build.yml:
##########
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: ClusterFuzzLite continuous builds
Review Comment:
This particular workflow is solely focused on ensuring the fuzzer harnesses keep building. It does not run the harnesses themselves.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5762686281
> I think the gate should also be about false positives (not just true negatives)
What do you mean by false positives?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#discussion_r4061495676
##########
.github/workflows/cflite_pr.yml:
##########
@@ -0,0 +1,48 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: ClusterFuzzLite PR fuzzing
Review Comment:
likewise here I think PR fuzzing will just generate more confusion to contributors than it is worth -- if the fuzz test fails on their PR they probably will either
1. Rerun it
2. Get confused and think it is related to something in their PR
I think both of these are confusing outcomes and recommend against it
##########
.github/workflows/cflite_build.yml:
##########
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: ClusterFuzzLite continuous builds
Review Comment:
I am not sure about the value of running fuzzing on main pushes as well -- my rationale is that if the fuzz test hits an issue on main then there will be a red x recorded on that commit and we will perhaps be mislead into thinking that a bug was introduced by that commit (or at least will have to rule it out)
Also it seems strange to me to make the amount of effort devoted to fuzz testing a function of the number of PRs merged to main (which is what this will do)
I personally suggest just running the fuzz testing in batch mode to begin with. If you want more fuzz time, then just crank it up there
##########
.github/workflows/cflite_prune.yml:
##########
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: ClusterFuzzLite corpus pruning
Review Comment:
I am not familiar with corpus pruning -- maybe a definition to the link would be helpful
Also I don't understand why this is a separate job from fuzz testing -- why not make the daily schedule a single action (perhaps with two jobs)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5735473092
No crash on the first fuzz runs is the expected outcome. The gate ran from an empty corpus, so libFuzzer had to rediscover the SQL grammar from random bytes, and a cold ten-minute window spends nearly all of its effort on grammar surface rather than deep paths. The parser also has five years of upstream fuzzing history, so the shallow crash surface was swept long ago. The detection power arrives in two steps, the nightly batch growing a shared corpus on the clusterfuzzlite branch and the planned parse -> Display -> re-parse target with a committed seed corpus turning that corpus productive from day one. What the first runs prove is the part that mattered, the ASF allowlist gate is open and the build-and-fuzz pipeline runs green on every pull request.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on code in PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#discussion_r4061659216
##########
.github/workflows/cflite_pr.yml:
##########
@@ -0,0 +1,48 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: ClusterFuzzLite PR fuzzing
Review Comment:
I am a tad new to ClusterFuzzLite specifically, so I double checked. In theory, IF I configured this correctly (and I primarely copied the documentation examples) if a bug exists on main and survives in the PR, it is reported as pre-existing, and if it exists in the PR and not in the main it is noted as new bug. Only the latter case should be marked as a blocking error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 closed pull request #2527: Fuzz: Wire up ClusterFuzzLite CI fuzzing
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5764982038
> > I think the gate should also be about false positives (not just true negatives)
>
> What do you mean by false positives?
I mean the chance of the fuzz test failing on a PR that did not cause whatever problem the fuzzer found.
I think if we run non deterministic tests on PRs we are going to inject some noise into the process -- I would recommend against it if at all possible (at least until we have some confidence that the fuzz test is mostly successful -- e.g. run it for a week or more every day and not find anything new)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fuzz: Wire up ClusterFuzzLite CI fuzzing [datafusion-sqlparser-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #2527:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2527#issuecomment-5765307589
> The scenario you describe where f(x) crashes and g(x) does not but it should can only happen if I configured ClusterFuzzLite poorly, if I understood it correctly.
Maybe it is different in ClusterFuzz but with most fuzzers, if you run them twice they test two different sets of queries.
So in that case you don't actually compare `f(x)` and `g(x)` -- instead you are basically testing `f(x)` and `g(x')` where the queries tested in `x'` are different than `x` because they are randomly generated
Perhaps I am misunderstanding 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org