You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Siddharth Tiwari <si...@live.com> on 2014/03/01 10:02:23 UTC

Seeing strange error in Hive

Hi Team,
I am seeing following error in hive in reduce phase,can you guide me on its cause and possible solution ?
java.lang.RuntimeException: Hive Runtime Error while closing operators: Unable to rename output from: hdfs://tlvcluster/tmp/hive-hadoop/hive_2014-03-01_03-14-36_812_8390586541316719852-1/_task_tmp.-ext-10001/_tmp.000003_0 to: hdfs://tlvcluster/tmp/hive-hadoop/hive_2014-03-01_03-14-36_812_8390586541316719852-1/_tmp.-ext-10001/000003_0
	at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:313)
	at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:516)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:447)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://tlvcluster/tmp/hive-hadoop/hive_2014-03-01_03-14-36_812
I am using hive-10.x , hadoop-2.0.0,. Appreciate any help in understanding the issue.

*------------------------*

Cheers !!!

Siddharth Tiwari

Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of God.” 

"Maybe other people will try to limit me but I don't limit myself"

Re: Amazon EMR error

Posted by Sanjay Subramanian <sa...@yahoo.com>.
ok so  I spun up another cluster with a previous version and it worked successfully 

This Amazon Hive version WORKS SUCCESSFULLY 
=====================================
AMI version:2.4.1
Hadoop distribution:Amazon 1.0.3
Applications:Hive 0.11.0.1



________________________________
 From: Sanjay Subramanian <sa...@yahoo.com>
To: "user@hive.apache.org" <us...@hive.apache.org>; USers Hadoop <us...@hadoop.apache.org> 
Sent: Saturday, March 1, 2014 4:57 PM
Subject: Amazon EMR error
 


Sorry guys , not sure if I should request help with this error here because  its an error on Amazon EMR Hive
But you guys have been my Hive fraternity for about 2 years now  and I thought it best to turn to u for help first 

Amazon Hive version 
===============
AMI version:2.4.2
Hadoop distribution:Amazon 1.0.3
Applications:Hive 0.11.0.1

HIVE TABLE
=======
DROP TABLE  solar_installs;
CREATE EXTERNAL TABLE  IF NOT EXISTS solar_installs(
   zipcode STRING,
   state STRING,
   sizekw DOUBLE,
   cost   DOUBLE,
   date_  STRING,
   lat     DOUBLE,
   lon     DOUBLE) ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
;
ALTER TABLE solar_installs SET LOCATION 's3://solarinstallations/data'
;

DATA SET SAMPLE 
==============
19801,DE,230.6,,12/30/2014,39.7402405,-75.5521456
80005,CO,7.1,12000.0,01/28/2014,39.851157,-105.132533
19968,DE,5.2,,01/27/2014,38.7776124,-75.3099136
52314,IA,30.0,116900.0,01/17/2014,41.901451,-91.48409
33414,FL,9.28,34000.0,01/16/2014,26.667321,-80.215585
19808,DE,5.2,,01/16/2014,39.721215,-75.668342
19702,DE,7.28,,01/16/2014,39.649586,-75.6998009
19958,DE,5.5,,01/16/2014,38.7638163,-75.1778853
19958,DE,10.56,,01/15/2014,38.7638163,-75.1778853
19958,DE,4.5,,01/15/2014,38.708142,-75.214294
06410,CT,860.0,,01/14/2014,41.5589919,-72.912401
05701,VT,140.4,,01/14/2014,43.598277,-72.955117
19971,DE,7.8,,01/13/2014,38.734048,-75.126563
19971,DE,4.06,,01/13/2014,38.7020005,-75.0961549

QUERY 
=====
hive -e "select zipcode, count(*) from solar_installs group by zipcode"

ERROR
=====
java.lang.RuntimeException: java.lang.IllegalArgumentException: Can not create a Path from an empty string at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:167) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:441) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:377) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132) at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.IllegalArgumentException: Can not create a Path from an empty string at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82) at org.apache.hadoop.fs.Path.<init>(Path.java:90) at org.apache.hadoop.hive.ql.exec.MapOperator.cleanUpInputFileChangedOp(MapOperator.java:588) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1377) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:611) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:149) ... 8 more

Amazon EMR error

Posted by Sanjay Subramanian <sa...@yahoo.com>.
Sorry guys , not sure if I should request help with this error here because  its an error on Amazon EMR Hive
But you guys have been my Hive fraternity for about 2 years now  and I thought it best to turn to u for help first 

Amazon Hive version 
===============
AMI version:2.4.2
Hadoop distribution:Amazon 1.0.3
Applications:Hive 0.11.0.1

HIVE TABLE
=======
DROP TABLE  solar_installs;
CREATE EXTERNAL TABLE  IF NOT EXISTS solar_installs(
   zipcode STRING,
   state STRING,
   sizekw DOUBLE,
   cost   DOUBLE,
   date_  STRING,
   lat     DOUBLE,
   lon     DOUBLE) ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
;
ALTER TABLE solar_installs SET LOCATION 's3://solarinstallations/data'
;

DATA SET SAMPLE 
==============
19801,DE,230.6,,12/30/2014,39.7402405,-75.5521456
80005,CO,7.1,12000.0,01/28/2014,39.851157,-105.132533
19968,DE,5.2,,01/27/2014,38.7776124,-75.3099136
52314,IA,30.0,116900.0,01/17/2014,41.901451,-91.48409
33414,FL,9.28,34000.0,01/16/2014,26.667321,-80.215585
19808,DE,5.2,,01/16/2014,39.721215,-75.668342
19702,DE,7.28,,01/16/2014,39.649586,-75.6998009
19958,DE,5.5,,01/16/2014,38.7638163,-75.1778853
19958,DE,10.56,,01/15/2014,38.7638163,-75.1778853
19958,DE,4.5,,01/15/2014,38.708142,-75.214294
06410,CT,860.0,,01/14/2014,41.5589919,-72.912401
05701,VT,140.4,,01/14/2014,43.598277,-72.955117
19971,DE,7.8,,01/13/2014,38.734048,-75.126563
19971,DE,4.06,,01/13/2014,38.7020005,-75.0961549

QUERY 
=====
hive -e "select zipcode, count(*) from solar_installs group by zipcode"

ERROR
=====
java.lang.RuntimeException: java.lang.IllegalArgumentException: Can not create a Path from an empty string at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:167) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:441) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:377) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132) at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.IllegalArgumentException: Can not create a Path from an empty string at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82) at org.apache.hadoop.fs.Path.<init>(Path.java:90) at org.apache.hadoop.hive.ql.exec.MapOperator.cleanUpInputFileChangedOp(MapOperator.java:588) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1377) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:611) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:149) ... 8 more