You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "Arthur.hk.chan@gmail.com" <ar...@gmail.com> on 2014/12/17 10:23:40 UTC

java.io.FileNotFoundException: File does not exist (nexr-hive-udf-0.2-SNAPSHOT.jar)

Hi,

Please help!

I am using hiveserver2 on HIVE 0.13 on Hadoop 2.4.1, also nexr-hive-udf-0.2-SNAPSHOT.jar

I can run query from CLI, e.g.
hive> SELECT add_months(sysdate(), +12) FROM DUAL;
Execution completed successfully
MapredLocal task succeeded
OK
2015-12-17
Time taken: 7.393 seconds, Fetched: 1 row(s)


hive-site.xml (added)
 <property>
  <name>hive.aux.jars.path</name>
  <value>$HIVE_HOME/nexr-hive-udf-0.2-SNAPSHOT.jar,$HIVE_HOME/csv-serde-1.1.2-0.11.0-all.jar</value>
 </property>

hive-env.sh (added)
export HIVE_AUX_JARS_PATH=$HIVE_HOME/lib/csv-serde-1.1.2-0.11.0-all.jar:$HIVE_HOME/lib/nexr-hive-udf-0.2-SNAPSHOT.jar


However, if it is accessed via hiveserver2, I got the following error, please help.

Regards
Arthur




14/12/17 16:47:52 WARN conf.Configuration: file:/tmp/hive_2014-12-17_16-47-51_096_5821374687950910377-1/-local-10003/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
Execution log at: /tmp/hduser_20141217164747_80b15b85-7820-4e3a-88ea-afffa131ff5a.log
java.io.FileNotFoundException: File does not exist: hdfs://mycluster/hadoop_data/hadoop_data/tmp/mapred/staging/hduser1962118853/.staging/job_local1962118853_0001/libjars/nexr-hive-udf-0.2-SNAPSHOT.jar
	at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
	at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)
	at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
	at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
	at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:93)
	at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
	at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:265)
	at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
	at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
	at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
	at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
	at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:740)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Job Submission failed with exception 'java.io.FileNotFoundException(File does not exist: hdfs://mycluster/hadoop_data/hadoop_data/tmp/mapred/staging/hduser1962118853/.staging/job_local1962118853_0001/libjars/nexr-hive-udf-0.2-SNAPSHOT.jar)'
Execution failed with exit status: 1
Obtaining error information

Re: java.io.FileNotFoundException: File does not exist (nexr-hive-udf-0.2-SNAPSHOT.jar)

Posted by Muthu Pandi <mu...@gmail.com>.
Hi Arthur

    Have a look at here
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-PermanentFunctions



*RegardsMuthupandi.K*

 Think before you print.



On Wed, Dec 17, 2014 at 2:53 PM, Arthur.hk.chan@gmail.com <
arthur.hk.chan@gmail.com> wrote:
>
> Hi,
>
> Please help!
>
> I am using *hiveserver2 *on HIVE 0.13 on Hadoop 2.4.1, also
> nexr-hive-udf-0.2-SNAPSHOT.jar
>
> I can run query from CLI, e.g.
> hive> SELECT add_months(sysdate(), +12) FROM DUAL;
> Execution completed successfully
> MapredLocal task succeeded
> OK
> 2015-12-17
> Time taken: 7.393 seconds, Fetched: 1 row(s)
>
>
> hive-site.xml (added)
>  <property>
>   <name>hive.aux.jars.path</name>
>
> <value>$HIVE_HOME/nexr-hive-udf-0.2-SNAPSHOT.jar,$HIVE_HOME/csv-serde-1.1.2-0.11.0-all.jar</value>
>  </property>
>
> hive-env.sh (added)
> export
> HIVE_AUX_JARS_PATH=$HIVE_HOME/lib/csv-serde-1.1.2-0.11.0-all.jar:$HIVE_HOME/lib/nexr-hive-udf-0.2-SNAPSHOT.jar
>
>
> However, if it is accessed via *hiveserver2, *I got the following error,
> please help.
>
> Regards
> Arthur
>
>
>
>
> 14/12/17 16:47:52 WARN conf.Configuration:
> file:/tmp/hive_2014-12-17_16-47-51_096_5821374687950910377-1/-local-10003/jobconf.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts;  Ignoring.
> Execution log at:
> /tmp/hduser_20141217164747_80b15b85-7820-4e3a-88ea-afffa131ff5a.log
> java.io.FileNotFoundException: File does not exist:
> hdfs://mycluster/hadoop_data/hadoop_data/tmp/mapred/staging/hduser1962118853/.staging/job_local1962118853_0001/libjars/nexr-hive-udf-0.2-SNAPSHOT.jar
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1128)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1120)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1120)
> at
> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
> at
> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
> at
> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:93)
> at
> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:265)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:420)
> at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:740)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Job Submission failed with exception 'java.io.FileNotFoundException(File
> does not exist:
> hdfs://mycluster/hadoop_data/hadoop_data/tmp/mapred/staging/hduser1962118853/.staging/job_local1962118853_0001/libjars/nexr-hive-udf-0.2-SNAPSHOT.jar
> )'
> Execution failed with exit status: 1
> Obtaining error information
>
>