You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "korowa (via GitHub)" <gi...@apache.org> on 2024/09/06 18:00:14 UTC

[PR] tests: enable fuzz for filtered anti-semi NLJoin [datafusion]

korowa opened a new pull request, #12360:
URL: https://github.com/apache/datafusion/pull/12360

   ## Which issue does this PR close?
   
   <!--
   We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
   -->
   
   Closes #11537.
   
   ## Rationale for this change
   
   <!--
    Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.  
   -->
   
   It turned out to be not a NLJoin issue, but fuzz tests -- during NLjoin construction, join filter was completely ignored, and all NLjoin tests were running with only equijoin conditions in filter.
   
   In the same time filtered inner/left/right/full join fuzz tests were passing even without the filter -- the reason is that they've been using `less_than_100_join_filter`, which always returns True, as generation of input data for column, used in this filter, always returns values less than 100 -- `make_staggered_batches` comment:
   ```
   /// two sorted int32 columns 'a', 'b' ranged from 0..99 as join columns
   ```
   so it seems to be more useful to place `col_lt_col_filter` in all tests, since it has non 100% selectivity.
   
   Finally, after replacing filter in all test cases, some of HjSmj tests turned out to be flaky, so they are temporarily disabled, until #12359 fix.
   
   ## What changes are included in this PR?
   
   <!--
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   
   - NLJoin construction in fuzz tests now takes into account join_filter, and constructs composite filter as a union of join_filter and equijoin conditions, all fuzz tests are enabled for NljHj case
   - all `_filtered` test cases now use `col_lt_col_filter`
   - some `_filtered` testcases disabled for SortMergeJoin due to their flakiness
   
   ## Are these changes tested?
   
   <!--
   We typically require tests for all PRs in order to:
   1. Prevent the code from being accidentally broken by subsequent changes
   2. Serve as another way to document the expected behavior of the code
   
   If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
   -->
   
   Yes
   
   ## Are there any user-facing changes?
   
   <!--
   If there are user-facing changes then we may require documentation to be updated before approving the PR.
   -->
   
   No, it's related exclusively to the tests
   
   <!--
   If there are any breaking changes to public APIs, please add the `api change` label.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] tests: enable fuzz for filtered anti-semi NLJoin [datafusion]

Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on code in PR #12360:
URL: https://github.com/apache/datafusion/pull/12360#discussion_r1749326396


##########
datafusion/core/tests/fuzz_cases/join_fuzz.rs:
##########
@@ -54,33 +55,6 @@ enum JoinTestType {
     // because if existing variants both passed that means SortMergeJoin and NestedLoopJoin also passes
     HjSmj,
 }
-#[tokio::test]
-async fn test_inner_join_1k() {
-    JoinFuzzTestCase::new(
-        make_staggered_batches(1000),
-        make_staggered_batches(1000),
-        JoinType::Inner,
-        None,
-    )
-    .run_test(&[JoinTestType::HjSmj, JoinTestType::NljHj], false)
-    .await
-}
-
-fn less_than_100_join_filter(schema1: Arc<Schema>, _schema2: Arc<Schema>) -> JoinFilter {

Review Comment:
   yeah, I think I introduced the `col_lt_col_filter` when testing flaky semi join.
   We probably should have used this filter for other filtered use cases.
   
   But if the tests becoming flaky, it would be dangerous to merge the PR without ignoring tests



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] tests: enable fuzz for filtered anti-semi NLJoin [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #12360:
URL: https://github.com/apache/datafusion/pull/12360#discussion_r1748041204


##########
datafusion/core/tests/fuzz_cases/join_fuzz.rs:
##########
@@ -54,33 +55,6 @@ enum JoinTestType {
     // because if existing variants both passed that means SortMergeJoin and NestedLoopJoin also passes
     HjSmj,
 }
-#[tokio::test]
-async fn test_inner_join_1k() {
-    JoinFuzzTestCase::new(
-        make_staggered_batches(1000),
-        make_staggered_batches(1000),
-        JoinType::Inner,
-        None,
-    )
-    .run_test(&[JoinTestType::HjSmj, JoinTestType::NljHj], false)
-    .await
-}
-
-fn less_than_100_join_filter(schema1: Arc<Schema>, _schema2: Arc<Schema>) -> JoinFilter {

Review Comment:
   > during NLjoin construction, join filter was completely ignored, and all NLjoin tests were running with only equijoin conditions in filter.
   
   
   Just to double check my understanding, your analysis is that this test is invalid because `NestedLoopJoinExec` doesn't appy the `less_than_100_join_filter` correctly?
   
   I reread the docs for `NestedLoopJoinExec` and it seems to imply the point is to run joins without a equality predicate 🤔 
   
   https://docs.rs/datafusion/latest/datafusion/physical_plan/joins/struct.NestedLoopJoinExec.html
   
   So is the difference that `less_than_100_join_filter` only referred to one join input (aka only refers to `a`) where NLJ needs filters that refer to both sides ?
   
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] tests: enable fuzz for filtered anti-semi NLJoin [datafusion]

Posted by "korowa (via GitHub)" <gi...@apache.org>.
korowa commented on PR #12360:
URL: https://github.com/apache/datafusion/pull/12360#issuecomment-2337132517

   > To find if test is flaky I followed the approach of running the test 1000 times in the loop. Like for AntiJoin
   
   I've run all them in a loop with hundred iterations, and all currently present tests are passing. The only failed tests after filter chenge were SMJ related -- they are disabled in f9d2a907f48cffdb1bf2d6ba601f8104d0dd8eeb


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] tests: enable fuzz for filtered anti-semi NLJoin [datafusion]

Posted by "korowa (via GitHub)" <gi...@apache.org>.
korowa commented on code in PR #12360:
URL: https://github.com/apache/datafusion/pull/12360#discussion_r1749196754


##########
datafusion/core/tests/fuzz_cases/join_fuzz.rs:
##########
@@ -54,33 +55,6 @@ enum JoinTestType {
     // because if existing variants both passed that means SortMergeJoin and NestedLoopJoin also passes
     HjSmj,
 }
-#[tokio::test]
-async fn test_inner_join_1k() {
-    JoinFuzzTestCase::new(
-        make_staggered_batches(1000),
-        make_staggered_batches(1000),
-        JoinType::Inner,
-        None,
-    )
-    .run_test(&[JoinTestType::HjSmj, JoinTestType::NljHj], false)
-    .await
-}
-
-fn less_than_100_join_filter(schema1: Arc<Schema>, _schema2: Arc<Schema>) -> JoinFilter {

Review Comment:
   > it seems to imply the point is to run joins without a equality predicate
   
   That's true, but in order to somehow perform fuzz testing against other join implementations, NLJoin runs (intended to run) in fuzz tests with the following filter `<on / equijoin conditions> and <optional join filter>`
   
   > Just to double check my understanding, your analysis is that this test is invalid because NestedLoopJoinExec doesn't appy the `less_than_100_join_filter` correctly?
   
   The test is invalid because of how NLJoin was built in these tests -- if `optional join filter` was passed as a test case argument, it was not included into NLJoin filter expression, and the test compared `join on <equijoin conditions> and <filter>` for HashJoin vs `join on <equijoin condition>` for NLJoin -- this problem is fixed by the first commit.
   
   > So is the difference that `less_than_100_join_filter` only referred to one join input (aka only refers to a) where NLJ needs filters that refer to both sides ?
   
   `less_than_100_join_filter` is removed because it's expression (`t.a < 100`) doesn't make much sense, since `a` produced by `make_staggered_batches` is always less than 100 -- that's why only semi/anti_filtered joins were failing (they used different filter expression -- `col_lt_col_filter`, which is not always true), and all other filtered test cases (inner/left/right/full) were passing even with missing filter for NLJoin (from p.1).
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] tests: enable fuzz for filtered anti-semi NLJoin [datafusion]

Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead merged PR #12360:
URL: https://github.com/apache/datafusion/pull/12360


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org