You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by "Bowden, Chris" <ch...@hpe.com> on 2017/01/20 18:35:44 UTC

FunctionRegistry

Thoughts on exposing FunctionRegistry via ExperimentalMethods?


I have functionality which can not be expressed efficiently via UDFs, consequently I implement my own Expressions. Currently I have to lift access to FunctionRegistry in my project(s) within org.apache.spark.sql.*. I also have to duplicate a number of functions to reconstruct FunctionRegistry.expression's behavior.


If we allow public modification of strategies and optimizations is there much risk in allowing direct access to register Expressions that can't be succinctly or efficiently represented as high level UDFs?