[GitHub] [spark] agrawaldevesh commented on pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

[GitBox <gi...@apache.org>]

agrawaldevesh commented on pull request #29104:
URL: https://github.com/apache/spark/pull/29104#issuecomment-661598557


   > @agrawaldevesh @cloud-fan
   > for BroadcasthashJoinExec, how about this
   > 
   > ```
   > if (isNullAwareAntiJoin) {
   >   require(leftKeys.length == 1, "leftKeys length should be 1")
   >   require(rightKeys.length == 1, "rightKeys length should be 1")
   >   require(right.output.length == 1, "null aware anti join only supports single column.")
   >   require(joinType == LeftAnti, "joinType must be LeftAnti.")
   >   require(buildSide == BuildRight, "buildSide must be BuildRight.")
   >   require(SQLConf.get.nullAwareAntiJoinOptimizeEnabled,
   >     "nullAwareAntiJoinOptimizeEnabled must be on for null aware anti join optimize.")
   >   require(checkCondition == "", "null aware anti join optimize condition should be empty.")
   > 
   >   if (broadcastRelation.value.inputEmpty) {
   >     return s"""
   >        |// singleColumn NAAJ inputEmpty(true) accept all
   >        |$numOutput.add(1);
   >        |${consume(ctx, input)}
   >      """.stripMargin
   >   } else if (broadcastRelation.value.anyNullKeyExists) {
   >     return s"""
   >        |// singleColumn NAAJ inputEmpty(false) anyNullKeyExists(true) reject all
   >      """.stripMargin
   >   }
   > }
   > 
   > if (uniqueKeyCodePath) {
   >   val found = ctx.freshName("found")
   >   s"""
   >      |boolean $found = false;
   >      |// generate join key for stream side
   >      |${keyEv.code}
   >      |// Check if the key has nulls.
   >      |if (!($anyNull)) {
   >      |  // Check if the HashedRelation exists.
   >      |  UnsafeRow $matched = (UnsafeRow)$relationTerm.getValue(${keyEv.value});
   >      |  if ($matched != null) {
   >      |    // Evaluate the condition.
   >      |    $checkCondition {
   >      |      $found = true;
   >      |    }
   >      |  }
   >      |} ${ if (isNullAwareAntiJoin) "else { $found = true }" else ""}
   >      |if (!$found) {
   >      |  $numOutput.add(1);
   >      |  ${consume(ctx, input)}
   >      |}
   >    """.stripMargin
   > } else {
   >   val matches = ctx.freshName("matches")
   >   val iteratorCls = classOf[Iterator[UnsafeRow]].getName
   >   val found = ctx.freshName("found")
   >   s"""
   >      |boolean $found = false;
   >      |// generate join key for stream side
   >      |${keyEv.code}
   >      |// Check if the key has nulls.
   >      |if (!($anyNull)) {
   >      |  // Check if the HashedRelation exists.
   >      |  $iteratorCls $matches = ($iteratorCls)$relationTerm.get(${keyEv.value});
   >      |  if ($matches != null) {
   >      |    // Evaluate the condition.
   >      |    while (!$found && $matches.hasNext()) {
   >      |      UnsafeRow $matched = (UnsafeRow) $matches.next();
   >      |      $checkCondition {
   >      |        $found = true;
   >      |      }
   >      |    }
   >      |  }
   >      |} ${ if (isNullAwareAntiJoin) "else { $found = true }" else ""}
   >      |if (!$found) {
   >      |  $numOutput.add(1);
   >      |  ${consume(ctx, input)}
   >      |}
   >    """.stripMargin
   > }
   > ```
   
   Looks great. Can you add a comment to:      ```|} ${ if (isNullAwareAntiJoin) "else { $found = true }" else ""}``` part ?
   
   Should also lead to a shorter diff.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org