[I] Optimize checkGlutenOperatorMatch [incubator-gluten]

["xumingming (via GitHub)" <gi...@apache.org>]

xumingming opened a new issue, #5759:
URL: https://github.com/apache/incubator-gluten/issues/5759

   ### Backend
   
   VL (Velox)
   
   ### Bug description
   
   With the following test case(which will fail, not important here, we want to see what the error message is):
   
   ```scala
     test("test explode1") {
       withTempView("t1") {
         sql(
           """select * from values (array(1)), (array(2)), (array(3)), (array(4))
             |as tbl(a)
        """.stripMargin).createOrReplaceTempView("t1")
         runQueryAndCompare(
           s"""
              |SELECT explode(a) from t1;
              |""".stripMargin) {
           checkGlutenOperatorMatch[BatchScanExecTransformer]
         }
       }
     }
   ```
   
   The error message is:
   
   ```bash
   org.scalatest.exceptions.TestFailedException: executedPlan.exists(((plan: org.apache.spark.sql.execution.SparkPlan) => tag.runtimeClass.isInstance(plan))) was false Expect Class exists in executedPlan:
    List(LocalTableScan [a#122]
   , RowToVeloxColumnar
   +- LocalTableScan [a#122]
   , InputAdapter
   +- RowToVeloxColumnar
      +- LocalTableScan [a#122]
   , InputIteratorTransformer[a#122]
   +- InputAdapter
      +- RowToVeloxColumnar
         +- LocalTableScan [a#122]
   , FilterExecTransformer (size(a#122, true) > 0)
   +- InputIteratorTransformer[a#122]
      +- InputAdapter
         +- RowToVeloxColumnar
            +- LocalTableScan [a#122]
   , ProjectExecTransformer [a#122, a#122]
   +- FilterExecTransformer (size(a#122, true) > 0)
      +- InputIteratorTransformer[a#122]
         +- InputAdapter
            +- RowToVeloxColumnar
               +- LocalTableScan [a#122]
   , GenerateExecTransformer explode(a#122), false, [col#128]
   +- ProjectExecTransformer [a#122, a#122]
      +- FilterExecTransformer (size(a#122, true) > 0)
         +- InputIteratorTransformer[a#122]
            +- InputAdapter
               +- RowToVeloxColumnar
                  +- LocalTableScan [a#122]
   , VeloxColumnarToRowExec
   +- ^(1) GenerateExecTransformer explode(a#122), false, [col#128]
      +- ^(1) ProjectExecTransformer [a#122, a#122]
         +- ^(1) FilterExecTransformer (size(a#122, true) > 0)
            +- ^(1) InputIteratorTransformer[a#122]
               +- ^(1) InputAdapter
                  +- ^(1) RowToVeloxColumnar
                     +- ^(1) LocalTableScan [a#122]
   ```
   
   
   Two issues here:
   
   1. The expected Operator name is show as `Class`, we expect actual operator name `BatchScanExecTransformer`  to be shown.
   2. All the Operators structure are shown, a better way is to show only the last one, which contains the whole plan structure here, i.e.
   
   ```bash
   VeloxColumnarToRowExec
   +- ^(1) GenerateExecTransformer explode(a#122), false, [col#128]
      +- ^(1) ProjectExecTransformer [a#122, a#122]
         +- ^(1) FilterExecTransformer (size(a#122, true) > 0)
            +- ^(1) InputIteratorTransformer[a#122]
               +- ^(1) InputAdapter
                  +- ^(1) RowToVeloxColumnar
                     +- ^(1) LocalTableScan [a#122]
   ```
   
   ### Spark version
   
   None
   
   ### Spark configurations
   
   N/A
   
   ### System information
   
   N/A
   
   ### Relevant logs
   
   ```bash
   N/A
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] Optimize checkGlutenOperatorMatch [incubator-gluten]

["Yohahaha (via GitHub)" <gi...@apache.org>]

Yohahaha closed issue #5759: Optimize checkGlutenOperatorMatch
URL: https://github.com/apache/incubator-gluten/issues/5759


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org