[GitHub] [carbondata] QiangCai commented on a change in pull request #4097: [CARBONDATA-4137] Refactor CarbonDataSourceScan without the soruces.Filter of Spark 3

[GitBox <gi...@apache.org>]

QiangCai commented on a change in pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#discussion_r588018676



##########
File path: integration/spark/src/main/scala/org/apache/carbondata/geo/InPolygonUDF.scala
##########
@@ -59,23 +58,3 @@ class InPolygonRangeListUDF extends ((String, String) => Boolean) with Serializa
     true // Carbon applies the filter. So, Spark do not have to apply filter.
   }
 }

Review comment:
       1. it will still keep the udf and expression for Spatial index
   
   2. In the old flow,  carbon converts spark expression to spark filter, and then converts spark filter to carbon expression.
   So, after creating carbon expression, this filter will be removed from memory.
   Now in the new flow, it will convert spark expression to carbon expression directly, and not need this filter.
   
   
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org