[GitHub] [hive] ganlute commented on a diff in pull request #3485: HIVE-26436: Hive on MR NullPointerException When initializeOp has not…

[GitBox <gi...@apache.org>]

ganlute commented on code in PR #3485:
URL: https://github.com/apache/hive/pull/3485#discussion_r944327963


##########
ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java:
##########
@@ -675,6 +678,13 @@ public void close(boolean abort) throws HiveException {
       return;
     }
 
+    // if the operator has not been initialized, there is no need to close
+    if (!this.rootInitializeCalled) {
+      LOG.debug("No need to close operator {}", this);
+      return;
+    }
+
+

Review Comment:
   > why this is specific to MR and doesn't happen with Tez? And what is the actual scenario/query where this happens, the extended test doesn't seems to be very indicative
   
   @anishek I am so sorry for the slow reply. 
   
   In fact, I can not reproduce the problem. It happened in one of my online clusters and I can not reproduce it in my new cluster. Some of configs may be different but I am not sure which one cause the problem.
   
   Here is the map file printed when exception happened:
   <MAP>Id =0
     <Children>
       <TS>Id =156
         <Children>
           <UNION>Id =26
             <Children>
               <SEL>Id =40
                 <Children>
                   <FS>Id =146
                     <Children>
                     <\Children>
                     <Parent>Id = 40 null<\Parent>
                   <\FS>
                 <\Children>
                 <Parent>Id = 26 null<\Parent>
               <\SEL>
             <\Children>
             <Parent>Id = 156 nullId = 162 
           <TS>Id =162
             <Children>null
             <\Children>
             <Parent><\Parent>
           <\TS>Id = 166 
           <TS>Id =166
             <Children>null
             <\Children>
             <Parent><\Parent>
           <\TS><\Parent>
           <\UNION>
         <\Children>
         <Parent>Id = 0 null<\Parent>
       <\TS>
     <\Children>
     <Parent><\Parent>
   <\MAP>
   
   However, I think for whatever reason the operator is not initialized, close should not throw an exception.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org