[GitHub] [hive] kgyrtkirk commented on a change in pull request #1876: [WIP] HIVE-24645: Call configure for UDFs after fetch task conversion

[GitBox <gi...@apache.org>]

kgyrtkirk commented on a change in pull request #1876:
URL: https://github.com/apache/hive/pull/1876#discussion_r565476709



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java
##########
@@ -140,8 +141,23 @@ public ObjectInspector initialize(ObjectInspector rowInspector) throws HiveExcep
       childrenOIs[i] = children[i].initialize(rowInspector);
     }
     MapredContext context = MapredContext.get();
+    // It is possible that there is no context at this point. For example a context is not created
+    // when "hive.fetch.task.conversion" occurs.
     if (context != null) {
       context.setup(genericUDF);
+    } else {
+      // It is a bit unfortunate that currently the UDF configuration signature expects a
+      // MapredContext (even if execution is tez or another engine) - this causes an
+      // impedence mismatch. For example: MapredContext has Reporter objects that may or
+      // may not make sense for the current engine.
+      //
+      // We attempt to create a dummyContext that has at least access to the currently set
+      // configuration. The other unfortunate issue is that some paths set the configuration
+      // when creating ExprNodeGenericFuncEvaluator while others do not, so we fallback to a
+      // "default" new HiveConf object which might be missing configuration that changed during
+      // runtime.
+      MapredContext dummyContext = MapredContext.createDummy(getConf() != null ? getConf() : new HiveConf());

Review comment:
       you might have access to the session conf via `SessionsState.get().getConf()` - which should be much closer to the actual configuration than `new HiveConf()`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org