[jira] [Created] (HIVE-21280) Null pointer exception on runnning compaction against a MM table.

["Aditya Shah (JIRA)" <ji...@apache.org>]

Aditya Shah created HIVE-21280:
----------------------------------

             Summary: Null pointer exception on runnning compaction against a MM table.
                 Key: HIVE-21280
                 URL: https://issues.apache.org/jira/browse/HIVE-21280
             Project: Hive
          Issue Type: Bug
    Affects Versions: 3.1.1, 3.0.0
            Reporter: Aditya Shah


On running compaction on MM table, got a null pointer exception while getting HDFS session path. The error seemed to me that the session state was not started for these queries. Even after making it start it further fails in running a Teztask for insert overwrite on temp table with the contents of the original table. The cause for this is Tezsession state is not able to initialize due to Illegal Argument exception being thrown at the time of setting up caller context in Tez task due to caller id which uses queryid being an empty string. 
I do think session state needs to be started and each of the queries running for compaction (I'm also doubtful for stats updater thread's queries) should have a query id. Some details are as follows:


Steps to reproduce:
1) Using beeline with HS2 and HMS
2) create an MM table
3) Insert a few values in the table
4) alter table mm_table compact 'major'; 

Stack trace on HMS:
{code:java}
compactor.Worker: Caught exception while trying to compact id:8,dbname:default,tableName:acid_mm_orc,partName:null,state:^@,type:MAJOR,properties:null,runAs:null,tooManyAborts:false,highestWriteId:0. Marking failed to avoid repeated failures, java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to run create temporary table default.tmp_compactor_acid_mm_orc_1550222367257(`a` int, `b` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'WITH SERDEPROPERTIES (
'serialization.format'='1')STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' LOCATION 'hdfs://localhost:9000/user/hive/warehouse/acid_mm_orc/_tmp_2d8a096c-2db5-4ed8-921c-b3f6d31e079e/_base' TBLPROPERTIES ('transactional'='false')
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.runMmCompaction(CompactorMR.java:373)
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.run(CompactorMR.java:241)
at org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:174)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to run create temporary table default.tmp_compactor_acid_mm_orc_1550222367257(`a` int, `b` string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'WITH SERDEPROPERTIES (
'serialization.format'='1')STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' LOCATION 'hdfs://localhost:9000/user/hive/warehouse/acid_mm_orc/_tmp_2d8a096c-2db5-4ed8-921c-b3f6d31e079e/_base' TBLPROPERTIES ('transactional'='false')
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.runOnDriver(CompactorMR.java:525)
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.runMmCompaction(CompactorMR.java:365)
... 2 more
Caused by: java.lang.NullPointerException: Non-local session path expected to be non-null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:228)
at org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:815)
at org.apache.hadoop.hive.ql.Context.<init>(Context.java:309)
at org.apache.hadoop.hive.ql.Context.<init>(Context.java:295)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:591)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1684)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1807)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1567)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1556)
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.runOnDriver(CompactorMR.java:522)
... 3 more
{code}
cc: [~ekoifman] [~vgumashta] [~sershe]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)