You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Ia...@barclayscapital.com on 2012/01/06 16:55:22 UTC
Problem with Hive->Hadoop
Hello,
Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining and loading tables worked fine, and I can query them with select *. However, a summary query against any table results in the following exception. I've see other documented issues like this regarding HBase and the Cloudera stack, but nothing specific to what I'm seeing.
Any thoughts?
java.io.IOException: Call to **********/**********:50030 failed on local exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
at org.apache.hadoop.ipc.Client.call(Client.java:1033)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
Job Submission failed with exception 'java.io.IOException(Call to **********/**********:50030 failed on local exception: java.io.EOFException)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
Ian Meyers
Barclays Capital
FICC Solutions Architecture
* Direct : +44 (0) 20 777 37437 * Extension: 37437
* ian.meyers@barclayscapital.com<ma...@barclayscapital.com>
_______________________________________________
This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing. Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________
RE: Problem with Hive->Hadoop
Posted by Ia...@barclayscapital.com.
No, no compression is used on any definition of input or output.
Ian
-----Original Message-----
From: alo.alt [mailto:wget.null@googlemail.com]
Sent: Friday, January 06, 2012 5:14 PM
To: user@hive.apache.org
Subject: Re: Problem with Hive->Hadoop
Hi Ian,
did you use compressed output or compressed files for input?
- Alex
Alexander Lorenz
http://mapredit.blogspot.com
On Jan 6, 2012, at 7:55 AM, <Ia...@barclayscapital.com> <Ia...@barclayscapital.com> wrote:
> Hello,
>
> Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining and loading tables worked fine, and I can query them with select *. However, a summary query against any table results in the following exception. I've see other documented issues like this regarding HBase and the Cloudera stack, but nothing specific to what I'm seeing.
>
> Any thoughts?
>
> java.io.IOException: Call to **********/**********:50030 failed on local exception: java.io.EOFException
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown Source)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
> at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
> at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
> at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
> Job Submission failed with exception 'java.io.IOException(Call to **********/**********:50030 failed on local exception: java.io.EOFException)'
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
>
> Ian Meyers
>
> Barclays Capital
> FICC Solutions Architecture
>
> ( Direct : +44 (0) 20 777 37437 ( Extension: 37437
> * ian.meyers@barclayscapital.com
>
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing. Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from other members of the Barclays Group.
> _______________________________________________
Re: Problem with Hive->Hadoop
Posted by Ankit Jain <an...@gmail.com>.
Hi,
It will use hadoop lib by default. Requirement is HADOOP_HOME must be set.
Thanks,
Ankit
On Mon, Jan 9, 2012 at 3:30 PM, <Ia...@barclayscapital.com> wrote:
> I did see that, and to try and ensure the right Hadoop libs are being
> used, I’ve created the following entry in hive-site.xml:****
>
> ** **
>
> <property>****
>
> <name>hive.aux.jars.path</name>****
>
>
> <value>/apps/hadoop/hadoop-0.20.203.0/hadoop-core-0.20.203.0.jar,/apps/hadoop/hadoop-0.20.203.0/hadoop-tools-0.20.203.0.jar</value>
> ****
>
> <description>Force override jar locations</description>***
> *
>
> </property>****
>
> ** **
>
> Is that correct? Does it try to use $HADOOP_HOME/lib by default? I don’t
> see any Hadoop libs in the hive/lib directory at all.****
>
> ** **
>
> Ian****
>
> ** **
>
> *From:* shashwat shriparv [mailto:dwivedishashwat@gmail.com]
> *Sent:* Friday, January 06, 2012 7:28 PM
>
> *To:* user@hive.apache.org
> *Subject:* Re: Problem with Hive->Hadoop****
>
> ** **
>
> Check out this post may be will be useful for you****
>
> ** **
>
>
> http://mail-archives.apache.org/mod_mbox/hive-user/201101.mbox/%3CAANLkTi=vTTVpd3Z5De24BjA8LKNwpvocHpnRTB7eOarK@mail.gmail.com%3E
> ****
>
> ** **
>
> https://issues.apache.org/jira/browse/HADOOP-4262****
>
> ** **
>
> ** **
>
> http://search-hadoop.com/m/hrAvDrf0Vu/v=threaded****
>
> Regards ****
>
> Shashwat****
>
> ** **
>
> ** **
>
> On Fri, Jan 6, 2012 at 10:43 PM, alo.alt <wg...@googlemail.com> wrote:
> ****
>
> Hi Ian,
>
> did you use compressed output or compressed files for input?
>
> - Alex
>
> Alexander Lorenz
> http://mapredit.blogspot.com****
>
>
>
> On Jan 6, 2012, at 7:55 AM, <Ia...@barclayscapital.com> <
> Ian.Meyers@barclayscapital.com> wrote:
>
> > Hello,
> >
> > Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining and
> loading tables worked fine, and I can query them with select *. However, a
> summary query against any table results in the following exception. I’ve
> see other documented issues like this regarding HBase and the Cloudera
> stack, but nothing specific to what I’m seeing.
> >
> > Any thoughts?
> >
> > java.io.IOException: Call to **********/**********:50030 failed on local
> exception: java.io.EOFException
> > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> > at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> > at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown
> Source)
> > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
> > at
> org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
> > at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
> > at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
> > at
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
> > at
> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
> > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
> > at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
> > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
> > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
> > at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> > at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> > at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > Caused by: java.io.EOFException
> > at java.io.DataInputStream.readInt(DataInputStream.java:375)
> > at
> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
> > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
> > Job Submission failed with exception 'java.io.IOException(Call to
> **********/**********:50030 failed on local exception:
> java.io.EOFException)'
> > FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
> >
> > Ian Meyers
> >
> > Barclays Capital
> > FICC Solutions Architecture
> >
> > ( Direct : +44 (0) 20 777 37437 ( Extension: 37437
> > * ian.meyers@barclayscapital.com
> >
> > _______________________________________________
> >
> > This e-mail may contain information that is confidential, privileged or
> otherwise protected from disclosure. If you are not an intended recipient
> of this e-mail, do not duplicate or redistribute it by any means. Please
> delete it and any attachments and notify the sender that you have received
> it in error. Unless specifically indicated, this e-mail is not an offer to
> buy or sell or a solicitation to buy or sell any securities, investment
> products or other financial product or service, an official confirmation of
> any transaction, or an official statement of Barclays. Any views or
> opinions presented are solely those of the author and do not necessarily
> represent those of Barclays. This e-mail is subject to terms available at
> the following link: www.barcap.com/emaildisclaimer. By messaging with
> Barclays you consent to the foregoing. Barclays Capital is the investment
> banking division of Barclays Bank PLC, a company registered in England
> (number 1026167) with its registered office at 1 Churchill Place, London,
> E14 5HP. This email may relate to or be sent from other members of the
> Barclays Group.
> > _______________________________________________****
>
>
>
>
> --
> Shashwat Shriparv
> 09900059620
> 09663531241
>
>
>
> <iframe src="
> http://rcm.amazon.com/e/cm?t=shriparv-20&o=1&p=48&l=ur1&category=kindlerotating&f=ifr"
> width="728" height="90" scrolling="no" border="0" marginwidth="0"
> style="border:none;" frameborder="0"></iframe>
>
> ****
>
RE: Problem with Hive->Hadoop
Posted by Ia...@barclayscapital.com.
After downgrading the hive references to Hadoop 0.20.2 rather than 0.20.203.0, I was unable to run any Hive queries. I'm still unable to use any aggregate functions whatever from Hive. Very strange behaviour. Again, the error message is as follows:
java.io.IOException: Call to ***/***:50030 failed on local exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
at org.apache.hadoop.ipc.Client.call(Client.java:1033)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
Job Submission failed with exception 'java.io.IOException(Call to ***/***:50030 failed on local exception: java.io.EOFException)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
Ian
From: Bhavesh Shah [mailto:bhavesh25shah@gmail.com]
Sent: Monday, January 09, 2012 10:09 AM
To: user@hive.apache.org
Subject: Re: Problem with Hive->Hadoop
Hello Ian,
Refer this link:
https://groups.google.com/a/cloudera.org/group/cdh-user/browse_thread/thread/e804ebbe13a2f6fc
--
Regards,
Bhavesh Shah
On Mon, Jan 9, 2012 at 3:35 PM, Ankit Jain <an...@gmail.com>> wrote:
It will use hadoop lib by default.. requirement is hadoop must be in class path
On Mon, Jan 9, 2012 at 3:30 PM, <Ia...@barclayscapital.com>> wrote:
I did see that, and to try and ensure the right Hadoop libs are being used, I've created the following entry in hive-site.xml:
<property>
<name>hive.aux.jars.path</name>
<value>/apps/hadoop/hadoop-0.20.203.0/hadoop-core-0.20.203.0.jar,/apps/hadoop/hadoop-0.20.203.0/hadoop-tools-0.20.203.0.jar</value>
<description>Force override jar locations</description>
</property>
Is that correct? Does it try to use $HADOOP_HOME/lib by default? I don't see any Hadoop libs in the hive/lib directory at all.
Ian
From: shashwat shriparv [mailto:dwivedishashwat@gmail.com<ma...@gmail.com>]
Sent: Friday, January 06, 2012 7:28 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: Problem with Hive->Hadoop
Check out this post may be will be useful for you
http://mail-archives.apache.org/mod_mbox/hive-user/201101.mbox/%3CAANLkTi=vTTVpd3Z5De24BjA8LKNwpvocHpnRTB7eOarK@mail.gmail.com%3E
https://issues.apache.org/jira/browse/HADOOP-4262
http://search-hadoop.com/m/hrAvDrf0Vu/v=threaded
Regards
Shashwat
On Fri, Jan 6, 2012 at 10:43 PM, alo.alt <wg...@googlemail.com>> wrote:
Hi Ian,
did you use compressed output or compressed files for input?
- Alex
Alexander Lorenz
http://mapredit.blogspot.com
On Jan 6, 2012, at 7:55 AM, <Ia...@barclayscapital.com>> <Ia...@barclayscapital.com>> wrote:
> Hello,
>
> Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining and loading tables worked fine, and I can query them with select *. However, a summary query against any table results in the following exception. I've see other documented issues like this regarding HBase and the Cloudera stack, but nothing specific to what I'm seeing.
>
> Any thoughts?
>
> java.io.IOException: Call to **********/**********:50030 failed on local exception: java.io.EOFException
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown Source)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
> at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
> at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
> at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
> Job Submission failed with exception 'java.io.IOException(Call to **********/**********:50030 failed on local exception: java.io.EOFException)'
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
>
> Ian Meyers
>
> Barclays Capital
> FICC Solutions Architecture
>
> ( Direct : +44 (0) 20 777 37437 ( Extension: 37437
> * ian.meyers@barclayscapital.com<ma...@barclayscapital.com>
>
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer<http://www.barcap.com/emaildisclaimer>. By messaging with Barclays you consent to the foregoing. Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from other members of the Barclays Group.
> _______________________________________________
--
Shashwat Shriparv
09900059620
09663531241
<iframe src="http://rcm.amazon.com/e/cm?t=shriparv-20&o=1&p=48&l=ur1&category=kindlerotating&f=ifr" width="728" height="90" scrolling="no" border="0" marginwidth="0" style="border:none;" frameborder="0"></iframe>
Re: Problem with Hive->Hadoop
Posted by Bhavesh Shah <bh...@gmail.com>.
Hello Ian,
Refer this link:
https://groups.google.com/a/cloudera.org/group/cdh-user/browse_thread/thread/e804ebbe13a2f6fc
--
Regards,
Bhavesh Shah
On Mon, Jan 9, 2012 at 3:35 PM, Ankit Jain <an...@gmail.com> wrote:
> It will use hadoop lib by default.. requirement is hadoop must be in class
> path
>
> On Mon, Jan 9, 2012 at 3:30 PM, <Ia...@barclayscapital.com> wrote:
>
>> I did see that, and to try and ensure the right Hadoop libs are being
>> used, I’ve created the following entry in hive-site.xml:****
>>
>> ** **
>>
>> <property>****
>>
>> <name>hive.aux.jars.path</name>****
>>
>>
>> <value>/apps/hadoop/hadoop-0.20.203.0/hadoop-core-0.20.203.0.jar,/apps/hadoop/hadoop-0.20.203.0/hadoop-tools-0.20.203.0.jar</value>
>> ****
>>
>> <description>Force override jar locations</description>**
>> **
>>
>> </property>****
>>
>> ** **
>>
>> Is that correct? Does it try to use $HADOOP_HOME/lib by default? I don’t
>> see any Hadoop libs in the hive/lib directory at all.****
>>
>> ** **
>>
>> Ian****
>>
>> ** **
>>
>> *From:* shashwat shriparv [mailto:dwivedishashwat@gmail.com]
>> *Sent:* Friday, January 06, 2012 7:28 PM
>>
>> *To:* user@hive.apache.org
>> *Subject:* Re: Problem with Hive->Hadoop****
>>
>> ** **
>>
>> Check out this post may be will be useful for you****
>>
>> ** **
>>
>>
>> http://mail-archives.apache.org/mod_mbox/hive-user/201101.mbox/%3CAANLkTi=vTTVpd3Z5De24BjA8LKNwpvocHpnRTB7eOarK@mail.gmail.com%3E
>> ****
>>
>> ** **
>>
>> https://issues.apache.org/jira/browse/HADOOP-4262****
>>
>> ** **
>>
>> ** **
>>
>> http://search-hadoop.com/m/hrAvDrf0Vu/v=threaded****
>>
>> Regards ****
>>
>> Shashwat****
>>
>> ** **
>>
>> ** **
>>
>> On Fri, Jan 6, 2012 at 10:43 PM, alo.alt <wg...@googlemail.com>
>> wrote:****
>>
>> Hi Ian,
>>
>> did you use compressed output or compressed files for input?
>>
>> - Alex
>>
>> Alexander Lorenz
>> http://mapredit.blogspot.com****
>>
>>
>>
>> On Jan 6, 2012, at 7:55 AM, <Ia...@barclayscapital.com> <
>> Ian.Meyers@barclayscapital.com> wrote:
>>
>> > Hello,
>> >
>> > Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining
>> and loading tables worked fine, and I can query them with select *.
>> However, a summary query against any table results in the following
>> exception. I’ve see other documented issues like this regarding HBase and
>> the Cloudera stack, but nothing specific to what I’m seeing.
>> >
>> > Any thoughts?
>> >
>> > java.io.IOException: Call to **********/**********:50030 failed on
>> local exception: java.io.EOFException
>> > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
>> > at org.apache.hadoop.ipc.Client.call(Client.java:1033)
>> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
>> > at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown
>> Source)
>> > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
>> > at
>> org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
>> > at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
>> > at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
>> > at
>> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
>> > at
>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
>> > at
>> org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
>> > at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>> > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
>> > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
>> > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
>> > at
>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
>> > at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
>> > at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
>> > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
>> > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>> > Caused by: java.io.EOFException
>> > at java.io.DataInputStream.readInt(DataInputStream.java:375)
>> > at
>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
>> > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
>> > Job Submission failed with exception 'java.io.IOException(Call to
>> **********/**********:50030 failed on local exception:
>> java.io.EOFException)'
>> > FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>> >
>> > Ian Meyers
>> >
>> > Barclays Capital
>> > FICC Solutions Architecture
>> >
>> > ( Direct : +44 (0) 20 777 37437 ( Extension: 37437
>> > * ian.meyers@barclayscapital.com
>> >
>> > _______________________________________________
>> >
>> > This e-mail may contain information that is confidential, privileged or
>> otherwise protected from disclosure. If you are not an intended recipient
>> of this e-mail, do not duplicate or redistribute it by any means. Please
>> delete it and any attachments and notify the sender that you have received
>> it in error. Unless specifically indicated, this e-mail is not an offer to
>> buy or sell or a solicitation to buy or sell any securities, investment
>> products or other financial product or service, an official confirmation of
>> any transaction, or an official statement of Barclays. Any views or
>> opinions presented are solely those of the author and do not necessarily
>> represent those of Barclays. This e-mail is subject to terms available at
>> the following link: www.barcap.com/emaildisclaimer. By messaging with
>> Barclays you consent to the foregoing. Barclays Capital is the investment
>> banking division of Barclays Bank PLC, a company registered in England
>> (number 1026167) with its registered office at 1 Churchill Place, London,
>> E14 5HP. This email may relate to or be sent from other members of the
>> Barclays Group.
>> > _______________________________________________****
>>
>>
>>
>>
>> --
>> Shashwat Shriparv
>> 09900059620
>> 09663531241
>>
>>
>>
>> <iframe src="
>> http://rcm.amazon.com/e/cm?t=shriparv-20&o=1&p=48&l=ur1&category=kindlerotating&f=ifr"
>> width="728" height="90" scrolling="no" border="0" marginwidth="0"
>> style="border:none;" frameborder="0"></iframe>
>>
>> ****
>>
>
>
Re: Problem with Hive->Hadoop
Posted by Ankit Jain <an...@gmail.com>.
It will use hadoop lib by default.. requirement is hadoop must be in class
path
On Mon, Jan 9, 2012 at 3:30 PM, <Ia...@barclayscapital.com> wrote:
> I did see that, and to try and ensure the right Hadoop libs are being
> used, I’ve created the following entry in hive-site.xml:****
>
> ** **
>
> <property>****
>
> <name>hive.aux.jars.path</name>****
>
>
> <value>/apps/hadoop/hadoop-0.20.203.0/hadoop-core-0.20.203.0.jar,/apps/hadoop/hadoop-0.20.203.0/hadoop-tools-0.20.203.0.jar</value>
> ****
>
> <description>Force override jar locations</description>***
> *
>
> </property>****
>
> ** **
>
> Is that correct? Does it try to use $HADOOP_HOME/lib by default? I don’t
> see any Hadoop libs in the hive/lib directory at all.****
>
> ** **
>
> Ian****
>
> ** **
>
> *From:* shashwat shriparv [mailto:dwivedishashwat@gmail.com]
> *Sent:* Friday, January 06, 2012 7:28 PM
>
> *To:* user@hive.apache.org
> *Subject:* Re: Problem with Hive->Hadoop****
>
> ** **
>
> Check out this post may be will be useful for you****
>
> ** **
>
>
> http://mail-archives.apache.org/mod_mbox/hive-user/201101.mbox/%3CAANLkTi=vTTVpd3Z5De24BjA8LKNwpvocHpnRTB7eOarK@mail.gmail.com%3E
> ****
>
> ** **
>
> https://issues.apache.org/jira/browse/HADOOP-4262****
>
> ** **
>
> ** **
>
> http://search-hadoop.com/m/hrAvDrf0Vu/v=threaded****
>
> Regards ****
>
> Shashwat****
>
> ** **
>
> ** **
>
> On Fri, Jan 6, 2012 at 10:43 PM, alo.alt <wg...@googlemail.com> wrote:
> ****
>
> Hi Ian,
>
> did you use compressed output or compressed files for input?
>
> - Alex
>
> Alexander Lorenz
> http://mapredit.blogspot.com****
>
>
>
> On Jan 6, 2012, at 7:55 AM, <Ia...@barclayscapital.com> <
> Ian.Meyers@barclayscapital.com> wrote:
>
> > Hello,
> >
> > Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining and
> loading tables worked fine, and I can query them with select *. However, a
> summary query against any table results in the following exception. I’ve
> see other documented issues like this regarding HBase and the Cloudera
> stack, but nothing specific to what I’m seeing.
> >
> > Any thoughts?
> >
> > java.io.IOException: Call to **********/**********:50030 failed on local
> exception: java.io.EOFException
> > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> > at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> > at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown
> Source)
> > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
> > at
> org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
> > at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
> > at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
> > at
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
> > at
> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
> > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
> > at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
> > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
> > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
> > at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> > at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> > at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > Caused by: java.io.EOFException
> > at java.io.DataInputStream.readInt(DataInputStream.java:375)
> > at
> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
> > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
> > Job Submission failed with exception 'java.io.IOException(Call to
> **********/**********:50030 failed on local exception:
> java.io.EOFException)'
> > FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
> >
> > Ian Meyers
> >
> > Barclays Capital
> > FICC Solutions Architecture
> >
> > ( Direct : +44 (0) 20 777 37437 ( Extension: 37437
> > * ian.meyers@barclayscapital.com
> >
> > _______________________________________________
> >
> > This e-mail may contain information that is confidential, privileged or
> otherwise protected from disclosure. If you are not an intended recipient
> of this e-mail, do not duplicate or redistribute it by any means. Please
> delete it and any attachments and notify the sender that you have received
> it in error. Unless specifically indicated, this e-mail is not an offer to
> buy or sell or a solicitation to buy or sell any securities, investment
> products or other financial product or service, an official confirmation of
> any transaction, or an official statement of Barclays. Any views or
> opinions presented are solely those of the author and do not necessarily
> represent those of Barclays. This e-mail is subject to terms available at
> the following link: www.barcap.com/emaildisclaimer. By messaging with
> Barclays you consent to the foregoing. Barclays Capital is the investment
> banking division of Barclays Bank PLC, a company registered in England
> (number 1026167) with its registered office at 1 Churchill Place, London,
> E14 5HP. This email may relate to or be sent from other members of the
> Barclays Group.
> > _______________________________________________****
>
>
>
>
> --
> Shashwat Shriparv
> 09900059620
> 09663531241
>
>
>
> <iframe src="
> http://rcm.amazon.com/e/cm?t=shriparv-20&o=1&p=48&l=ur1&category=kindlerotating&f=ifr"
> width="728" height="90" scrolling="no" border="0" marginwidth="0"
> style="border:none;" frameborder="0"></iframe>
>
> ****
>
RE: Problem with Hive->Hadoop
Posted by Ia...@barclayscapital.com.
I will attempt to downgrade to 0.20.2 of Hadoop libraries for hive to use - can anyone please advise if the below syntax is the correct method to change the Hadoop jar path? As an alternative I will change the Hadoop home for the session...
Ian
From: Meyers, Ian: IT (LDN)
Sent: Monday, January 09, 2012 10:00 AM
To: user@hive.apache.org
Subject: RE: Problem with Hive->Hadoop
I did see that, and to try and ensure the right Hadoop libs are being used, I've created the following entry in hive-site.xml:
<property>
<name>hive.aux.jars.path</name>
<value>/apps/hadoop/hadoop-0.20.203.0/hadoop-core-0.20.203.0.jar,/apps/hadoop/hadoop-0.20.203.0/hadoop-tools-0.20.203.0.jar</value>
<description>Force override jar locations</description>
</property>
Is that correct? Does it try to use $HADOOP_HOME/lib by default? I don't see any Hadoop libs in the hive/lib directory at all.
Ian
From: shashwat shriparv [mailto:dwivedishashwat@gmail.com]
Sent: Friday, January 06, 2012 7:28 PM
To: user@hive.apache.org
Subject: Re: Problem with Hive->Hadoop
Check out this post may be will be useful for you
http://mail-archives.apache.org/mod_mbox/hive-user/201101.mbox/%3CAANLkTi=vTTVpd3Z5De24BjA8LKNwpvocHpnRTB7eOarK@mail.gmail.com%3E
https://issues.apache.org/jira/browse/HADOOP-4262
http://search-hadoop.com/m/hrAvDrf0Vu/v=threaded
Regards
Shashwat
On Fri, Jan 6, 2012 at 10:43 PM, alo.alt <wg...@googlemail.com>> wrote:
Hi Ian,
did you use compressed output or compressed files for input?
- Alex
Alexander Lorenz
http://mapredit.blogspot.com
On Jan 6, 2012, at 7:55 AM, <Ia...@barclayscapital.com>> <Ia...@barclayscapital.com>> wrote:
> Hello,
>
> Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining and loading tables worked fine, and I can query them with select *. However, a summary query against any table results in the following exception. I've see other documented issues like this regarding HBase and the Cloudera stack, but nothing specific to what I'm seeing.
>
> Any thoughts?
>
> java.io.IOException: Call to **********/**********:50030 failed on local exception: java.io.EOFException
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown Source)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
> at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
> at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
> at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
> Job Submission failed with exception 'java.io.IOException(Call to **********/**********:50030 failed on local exception: java.io.EOFException)'
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
>
> Ian Meyers
>
> Barclays Capital
> FICC Solutions Architecture
>
> ( Direct : +44 (0) 20 777 37437 ( Extension: 37437
> * ian.meyers@barclayscapital.com<ma...@barclayscapital.com>
>
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer<http://www.barcap.com/emaildisclaimer>. By messaging with Barclays you consent to the foregoing. Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from other members of the Barclays Group.
> _______________________________________________
--
Shashwat Shriparv
09900059620
09663531241
<iframe src="http://rcm.amazon.com/e/cm?t=shriparv-20&o=1&p=48&l=ur1&category=kindlerotating&f=ifr" width="728" height="90" scrolling="no" border="0" marginwidth="0" style="border:none;" frameborder="0"></iframe>
RE: Problem with Hive->Hadoop
Posted by Ia...@barclayscapital.com.
I did see that, and to try and ensure the right Hadoop libs are being used, I've created the following entry in hive-site.xml:
<property>
<name>hive.aux.jars.path</name>
<value>/apps/hadoop/hadoop-0.20.203.0/hadoop-core-0.20.203.0.jar,/apps/hadoop/hadoop-0.20.203.0/hadoop-tools-0.20.203.0.jar</value>
<description>Force override jar locations</description>
</property>
Is that correct? Does it try to use $HADOOP_HOME/lib by default? I don't see any Hadoop libs in the hive/lib directory at all.
Ian
From: shashwat shriparv [mailto:dwivedishashwat@gmail.com]
Sent: Friday, January 06, 2012 7:28 PM
To: user@hive.apache.org
Subject: Re: Problem with Hive->Hadoop
Check out this post may be will be useful for you
http://mail-archives.apache.org/mod_mbox/hive-user/201101.mbox/%3CAANLkTi=vTTVpd3Z5De24BjA8LKNwpvocHpnRTB7eOarK@mail.gmail.com%3E
https://issues.apache.org/jira/browse/HADOOP-4262
http://search-hadoop.com/m/hrAvDrf0Vu/v=threaded
Regards
Shashwat
On Fri, Jan 6, 2012 at 10:43 PM, alo.alt <wg...@googlemail.com>> wrote:
Hi Ian,
did you use compressed output or compressed files for input?
- Alex
Alexander Lorenz
http://mapredit.blogspot.com
On Jan 6, 2012, at 7:55 AM, <Ia...@barclayscapital.com>> <Ia...@barclayscapital.com>> wrote:
> Hello,
>
> Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining and loading tables worked fine, and I can query them with select *. However, a summary query against any table results in the following exception. I've see other documented issues like this regarding HBase and the Cloudera stack, but nothing specific to what I'm seeing.
>
> Any thoughts?
>
> java.io.IOException: Call to **********/**********:50030 failed on local exception: java.io.EOFException
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown Source)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
> at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
> at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
> at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
> Job Submission failed with exception 'java.io.IOException(Call to **********/**********:50030 failed on local exception: java.io.EOFException)'
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
>
> Ian Meyers
>
> Barclays Capital
> FICC Solutions Architecture
>
> ( Direct : +44 (0) 20 777 37437 ( Extension: 37437
> * ian.meyers@barclayscapital.com<ma...@barclayscapital.com>
>
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer<http://www.barcap.com/emaildisclaimer>. By messaging with Barclays you consent to the foregoing. Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from other members of the Barclays Group.
> _______________________________________________
--
Shashwat Shriparv
09900059620
09663531241
<iframe src="http://rcm.amazon.com/e/cm?t=shriparv-20&o=1&p=48&l=ur1&category=kindlerotating&f=ifr" width="728" height="90" scrolling="no" border="0" marginwidth="0" style="border:none;" frameborder="0"></iframe>
Re: Problem with Hive->Hadoop
Posted by shashwat shriparv <dw...@gmail.com>.
Check out this post may be will be useful for you
http://mail-archives.apache.org/mod_mbox/hive-user/201101.mbox/%3CAANLkTi=vTTVpd3Z5De24BjA8LKNwpvocHpnRTB7eOarK@mail.gmail.com%3E
https://issues.apache.org/jira/browse/HADOOP-4262
http://search-hadoop.com/m/hrAvDrf0Vu/v=threaded
Regards
Shashwat
On Fri, Jan 6, 2012 at 10:43 PM, alo.alt <wg...@googlemail.com> wrote:
> Hi Ian,
>
> did you use compressed output or compressed files for input?
>
> - Alex
>
> Alexander Lorenz
> http://mapredit.blogspot.com
>
>
> On Jan 6, 2012, at 7:55 AM, <Ia...@barclayscapital.com> <
> Ian.Meyers@barclayscapital.com> wrote:
>
> > Hello,
> >
> > Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining and
> loading tables worked fine, and I can query them with select *. However, a
> summary query against any table results in the following exception. I’ve
> see other documented issues like this regarding HBase and the Cloudera
> stack, but nothing specific to what I’m seeing.
> >
> > Any thoughts?
> >
> > java.io.IOException: Call to **********/**********:50030 failed on local
> exception: java.io.EOFException
> > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> > at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> > at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown
> Source)
> > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
> > at
> org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
> > at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
> > at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
> > at
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
> > at
> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
> > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
> > at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
> > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
> > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
> > at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> > at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> > at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > Caused by: java.io.EOFException
> > at java.io.DataInputStream.readInt(DataInputStream.java:375)
> > at
> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
> > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
> > Job Submission failed with exception 'java.io.IOException(Call to
> **********/**********:50030 failed on local exception:
> java.io.EOFException)'
> > FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
> >
> > Ian Meyers
> >
> > Barclays Capital
> > FICC Solutions Architecture
> >
> > ( Direct : +44 (0) 20 777 37437 ( Extension: 37437
> > * ian.meyers@barclayscapital.com
> >
> > _______________________________________________
> >
> > This e-mail may contain information that is confidential, privileged or
> otherwise protected from disclosure. If you are not an intended recipient
> of this e-mail, do not duplicate or redistribute it by any means. Please
> delete it and any attachments and notify the sender that you have received
> it in error. Unless specifically indicated, this e-mail is not an offer to
> buy or sell or a solicitation to buy or sell any securities, investment
> products or other financial product or service, an official confirmation of
> any transaction, or an official statement of Barclays. Any views or
> opinions presented are solely those of the author and do not necessarily
> represent those of Barclays. This e-mail is subject to terms available at
> the following link: www.barcap.com/emaildisclaimer. By messaging with
> Barclays you consent to the foregoing. Barclays Capital is the investment
> banking division of Barclays Bank PLC, a company registered in England
> (number 1026167) with its registered office at 1 Churchill Place, London,
> E14 5HP. This email may relate to or be sent from other members of the
> Barclays Group.
> > _______________________________________________
>
>
--
Shashwat Shriparv
09900059620
09663531241
<iframe src="
http://rcm.amazon.com/e/cm?t=shriparv-20&o=1&p=48&l=ur1&category=kindlerotating&f=ifr"
width="728" height="90" scrolling="no" border="0" marginwidth="0"
style="border:none;" frameborder="0"></iframe>
Re: Problem with Hive->Hadoop
Posted by "alo.alt" <wg...@googlemail.com>.
Hi Ian,
did you use compressed output or compressed files for input?
- Alex
Alexander Lorenz
http://mapredit.blogspot.com
On Jan 6, 2012, at 7:55 AM, <Ia...@barclayscapital.com> <Ia...@barclayscapital.com> wrote:
> Hello,
>
> Using version .8 of Hive with version 0.20.203.0 of Hadoop. Defining and loading tables worked fine, and I can query them with select *. However, a summary query against any table results in the following exception. I’ve see other documented issues like this regarding HBase and the Cloudera stack, but nothing specific to what I’m seeing.
>
> Any thoughts?
>
> java.io.IOException: Call to **********/**********:50030 failed on local exception: java.io.EOFException
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> at org.apache.hadoop.mapred.$Proxy8.getProtocolVersion(Unknown Source)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364)
> at org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:460)
> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:454)
> at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:437)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:435)
> at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:774)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
> Job Submission failed with exception 'java.io.IOException(Call to **********/**********:50030 failed on local exception: java.io.EOFException)'
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
>
> Ian Meyers
>
> Barclays Capital
> FICC Solutions Architecture
>
> ( Direct : +44 (0) 20 777 37437 ( Extension: 37437
> * ian.meyers@barclayscapital.com
>
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing. Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from other members of the Barclays Group.
> _______________________________________________