[GitHub] [arrow-datafusion] pjmore commented on issue #237: Extend & generalize constant folding / evaluation in logical optimizer

[GitBox <gi...@apache.org>]

pjmore commented on issue #237:
URL: https://github.com/apache/arrow-datafusion/issues/237#issuecomment-845477448


   So as far as evaluating literal expressions, such as ```abs(lit(12) - lit(3.5))``` goes another approach that can be taken is breaking out the core logic of pure physical expressions that don't read data from the ```RecordBatch``` from the evaluate function and putting it into a associated function that takes the ```ColumnarValue``` from sub expression evaluations and operates on that. For example for a ```BinaryExpr``` the signature would be:
     ``` fn evaluate_binop(left_value: ColumnarValue, op: &Operator, right_value: ColumnarValue, num_rows: usize)->Result<ColumnarValue>```
   
   The ```PhysicalExpr```'s evaluate function becomes:
   ``` 
   fn evaluate(&self, batch: &RecordBatch) -> Result<ColumnarValue> {
           let left_value = self.left.evaluate(batch)?;
           let right_value = self.right.evaluate(batch)?;
           BinaryExpr::evaluate_binop(left_value, &self.op, right_value, batch.num_rows())
   }
   ```
   The split allows ```Expr```s that are made up of literal values to be evaluated directly using mostly the same logic.  There is still  some implementation divergence, mainly due to expressions that are inserted during the creation of a physical expression, such as ```BinaryExpr``` inserting a ```TryCastExpr``` to coerce types. I've implemented a proof of concept [here](https://github.com/pjmore/arrow-datafusion/tree/master/datafusion). Word of warning, the organization of the code and the expression rewriting are both quite messy, and I ignored all expression renaming issues.  There is also potentially an issue with how I handled array return values, which was to convert them to a scalar if it has one element and a scalar list otherwise.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org