[jira] [Commented] (DRILL-4963) Issues when overloading Drill native functions with dynamic UDFs

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-4963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891718#comment-15891718 ] 

ASF GitHub Bot commented on DRILL-4963:
---------------------------------------

Github user jinfengni commented on the issue:

    https://github.com/apache/drill/pull/701
  
    +1
    
    I have one naive question. Before we dive into MVCC, or try to improve the current ZK-based approach, have we consider adding a command "refresh function registry", so that users enforce that function registry across the entire cluster is consistent? I remember someone (probably @amansinha100 ) once brought up this idea.
    
    1. How often would user run into the issue caused by inconsistency of functions in different drillbit?  Sounds like we are adding non-negligible overhead for other queries which do not use dynamic UDF or overloaded functions, just for the sake of resolving issues of function inconsistency.   
    2. Drill's function resolution logic not only happens in planning but also execution time (due to the fact schema may be known in execution time). This means the function registry could be checked multiple times during the query lifetime.  Adding overhead to each check would slow down the entire query.
    3. With the "refresh function registry" command, use has a way to ensure function registry is consistent after issuing such command. We could tell user that drillbit could run into various of problems, before run "refresh function registry", therefore it's always recommend to run such command after user creates a new UDF.
    



> Issues when overloading Drill native functions with dynamic UDFs
> ----------------------------------------------------------------
>
>                 Key: DRILL-4963
>                 URL: https://issues.apache.org/jira/browse/DRILL-4963
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Functions - Drill
>    Affects Versions: 1.9.0
>            Reporter: Roman
>            Assignee: Arina Ielchiieva
>              Labels: ready-to-commit
>             Fix For: Future
>
>         Attachments: subquery_udf-1.0.jar, subquery_udf-1.0-sources.jar, test_overloading-1.0.jar, test_overloading-1.0-sources.jar
>
>
> I created jar file which overloads 3 DRILL native functions (LOG(VARCHAR-REQUIRED), CURRENT_DATE(VARCHAR-REQUIRED) and ABS(VARCHAR-REQUIRED,VARCHAR-REQUIRED)) and registered it as dynamic UDF.
> If I try to use my functions I will get errors:
> {code:xml}
> SELECT CURRENT_DATE('test') FROM (VALUES(1));
> {code}
> Error: FUNCTION ERROR: CURRENT_DATE does not support operand types (CHAR)
> SQL Query null
> {code:xml}
> SELECT ABS('test','test') FROM (VALUES(1));
> {code}
> Error: FUNCTION ERROR: ABS does not support operand types (CHAR,CHAR)
> SQL Query null
> {code:xml}
> SELECT LOG('test') FROM (VALUES(1));
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Failure while materializing expression in constant expression evaluator LOG('test').  Errors: 
> Error in expression at index -1.  Error: Missing function implementation: castTINYINT(VARCHAR-REQUIRED).  Full expression: UNKNOWN EXPRESSION.
> But if I rerun all this queries after "DrillRuntimeException", they will run correctly. It seems that Drill have not updated the function signature before that error. Also if I add jar as usual UDF (copy jar to /drill_home/jars/3rdparty and restart drillbits), all queries will run correctly without errors.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)