You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Lenin Raj <em...@gmail.com> on 2013/06/02 12:39:53 UTC
Re: Able to ADD JAR but FileNotFoundException while querying
CC'ing the dev list.
Thanks,
Lenin
On Sat, Jun 1, 2013 at 5:47 PM, Lenin Raj <em...@gmail.com> wrote:
> Hello,
>
> I created a 'tweets' table as mentioned here:
> https://github.com/cloudera/cdh-twitter-example
>
> After populating data into HDFS directory, When I tried to query the
> table, I am getting FileNotFoundException for the Hive serdes file.
> ADD JAR command successfully works. But still throws that exception while
> querying the table.
>
>
> *hive> add jar
> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar;*
> Added /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar to
> class path
> Added resource:
> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar
> *hive> select count(*) from tweets;*
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
> set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
> set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
> set mapred.reduce.tasks=<number>
> *java.io.FileNotFoundException: File does not exist:
> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar
> * at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:517)
> at
> org.apache.hadoop.filecache.DistributedCache.getFileStatus(DistributedCache.java:185)
> at
> org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestamps(TrackerDistributedCacheManager.java:707)
> at
> org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:763)
> at
> org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:655)
> at
> org.apache.hadoop.mapred.JobClient.access$300(JobClient.java:174)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:865)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:416)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
> at
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447)
> at
> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:138)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Job Submission failed with exception 'java.io.FileNotFoundException(File
> does not exist:
> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar)'
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
> hive>
>
> Help is much appreciated.
>
> Thanks,
> Lenin
>
Re: Able to ADD JAR but FileNotFoundException while querying
Posted by Lenin Raj <em...@gmail.com>.
Hello All,
Bcc'ing Dev list
The PATH of the JAR file refers a HDFS path. Moved the file to HDFS and the
issue is resolved.
Though, ADD JAR command should valid whether file exists before saying
"Added resource"
Please correct me if I'm wrong.
Thanks,
Lenin
On Sun, Jun 2, 2013 at 4:09 PM, Lenin Raj <em...@gmail.com> wrote:
> CC'ing the dev list.
>
>
> Thanks,
> Lenin
>
>
> On Sat, Jun 1, 2013 at 5:47 PM, Lenin Raj <em...@gmail.com> wrote:
>
>> Hello,
>>
>> I created a 'tweets' table as mentioned here:
>> https://github.com/cloudera/cdh-twitter-example
>>
>> After populating data into HDFS directory, When I tried to query the
>> table, I am getting FileNotFoundException for the Hive serdes file.
>> ADD JAR command successfully works. But still throws that exception while
>> querying the table.
>>
>>
>> *hive> add jar
>> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar;*
>> Added /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar to
>> class path
>> Added resource:
>> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar
>> *hive> select count(*) from tweets;*
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks determined at compile time: 1
>> In order to change the average load for a reducer (in bytes):
>> set hive.exec.reducers.bytes.per.reducer=<number>
>> In order to limit the maximum number of reducers:
>> set hive.exec.reducers.max=<number>
>> In order to set a constant number of reducers:
>> set mapred.reduce.tasks=<number>
>> *java.io.FileNotFoundException: File does not exist:
>> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar
>> * at
>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:517)
>> at
>> org.apache.hadoop.filecache.DistributedCache.getFileStatus(DistributedCache.java:185)
>> at
>> org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestamps(TrackerDistributedCacheManager.java:707)
>> at
>> org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:763)
>> at
>> org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:655)
>> at
>> org.apache.hadoop.mapred.JobClient.access$300(JobClient.java:174)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:865)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:416)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
>> at
>> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447)
>> at
>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:138)
>> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144)
>> at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> Job Submission failed with exception 'java.io.FileNotFoundException(File
>> does not exist:
>> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar)'
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>> hive>
>>
>> Help is much appreciated.
>>
>> Thanks,
>> Lenin
>>
>
>
Re: Able to ADD JAR but FileNotFoundException while querying
Posted by Lenin Raj <em...@gmail.com>.
Hello All,
Bcc'ing Dev list
The PATH of the JAR file refers a HDFS path. Moved the file to HDFS and the
issue is resolved.
Though, ADD JAR command should valid whether file exists before saying
"Added resource"
Please correct me if I'm wrong.
Thanks,
Lenin
On Sun, Jun 2, 2013 at 4:09 PM, Lenin Raj <em...@gmail.com> wrote:
> CC'ing the dev list.
>
>
> Thanks,
> Lenin
>
>
> On Sat, Jun 1, 2013 at 5:47 PM, Lenin Raj <em...@gmail.com> wrote:
>
>> Hello,
>>
>> I created a 'tweets' table as mentioned here:
>> https://github.com/cloudera/cdh-twitter-example
>>
>> After populating data into HDFS directory, When I tried to query the
>> table, I am getting FileNotFoundException for the Hive serdes file.
>> ADD JAR command successfully works. But still throws that exception while
>> querying the table.
>>
>>
>> *hive> add jar
>> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar;*
>> Added /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar to
>> class path
>> Added resource:
>> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar
>> *hive> select count(*) from tweets;*
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks determined at compile time: 1
>> In order to change the average load for a reducer (in bytes):
>> set hive.exec.reducers.bytes.per.reducer=<number>
>> In order to limit the maximum number of reducers:
>> set hive.exec.reducers.max=<number>
>> In order to set a constant number of reducers:
>> set mapred.reduce.tasks=<number>
>> *java.io.FileNotFoundException: File does not exist:
>> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar
>> * at
>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:517)
>> at
>> org.apache.hadoop.filecache.DistributedCache.getFileStatus(DistributedCache.java:185)
>> at
>> org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestamps(TrackerDistributedCacheManager.java:707)
>> at
>> org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:763)
>> at
>> org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:655)
>> at
>> org.apache.hadoop.mapred.JobClient.access$300(JobClient.java:174)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:865)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:416)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
>> at
>> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
>> at
>> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:447)
>> at
>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:138)
>> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144)
>> at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
>> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> Job Submission failed with exception 'java.io.FileNotFoundException(File
>> does not exist:
>> /usr/local/hive/hive/hive-serdes/hive-serdes-1.0-SNAPSHOT.jar)'
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>> hive>
>>
>> Help is much appreciated.
>>
>> Thanks,
>> Lenin
>>
>
>