Re: [I] Simple Functions [datafusion]

["findepi (via GitHub)" <gi...@apache.org>]

findepi commented on issue #12635:
URL: https://github.com/apache/datafusion/issues/12635#issuecomment-2401892724

   I experimented with this on the way from DataFusion meetup in Belgrade.
   
   i came up with something like this
   
   #### function author would write this
   
   row-oriented function logic
   
   ```rust
   // hand-written
   pub struct MyGcd {}
   impl MyGcd {
       pub fn call(x: i64, y: i64) -> Result<i64> {
           datafusion::functions::math::gcd::compute_gcd(x, y)
       }
   }
   ```
   
   #### a macro would generate
   
   generated (syntactically derived)
   ```rust
   impl datafusion::logical_expr::ScalarUDFImpl for MyGcd {
       fn as_any(&self) -> &dyn Any {
           self
       }
   
       fn name(&self) -> &str {
           "MyGcd"
       }
   
       fn signature(&self) -> &Signature {
           todo!() // TODO here we should use logical typyes
       }
   
       fn return_type(&self, _arg_types: &[DataType]) -> Result<DataType> {
           todo!() // TODO here we should use logical typyes
       }
   
       fn invoke(&self, args: &[ColumnarValue]) -> Result<ColumnarValue> {
           lifted_invoke::<MyGcd>(self, args)
       }
   }
   
   impl Invocable for MyGcd {
       const ARGUMENT_COUNT: u8 = 2;
       type ArgumentTypeList = (i64, (i64, ()));
       type OutArgType = ();
       type FormalReturnType = Result<i64>;
   
       fn invoke(&self, regular_args: Self::ArgumentTypeList, _out_arg: &mut Self::OutArgType) -> Self::FormalReturnType {
           let (a, (b, ())) = regular_args;
           MyGcd::call(a, b)
       }
   }
   ```
   
   All the heavy lifting happens inside `lifted_invoke` (source https://gist.github.com/findepi/f1b0b84fc5f2002cb3d31b19a5bb983e#file-library-rs)
   
   Notes on API design:
   
   - `type ArgumentTypeList = (i64, (i64, ()));` -- this  form of list presentation yields to template programming
   - `OutArgType`  + `FormalReturnType` -- we need both to support plain returning functions (numerics) and out args (for returning allocatable stuff like strings or structs)
   - `FormalReturnType = Result<i64>;` -- `lifted_invoke` supports functions fallible and infallible functions, as well as nullable return values (via `Option<T>`)
   
   
   Note:
   
   - for continuing work on this would be best to have logical types defined (https://github.com/apache/datafusion/issues/12622). @notfilippo could we perhaps have a logical types stub in `main` at some point?
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org