You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Cyrille Djoko <cd...@agnik.com> on 2013/02/15 19:44:45 UTC
Hive Queries
I am looking for a relatively efficient way of transferring data between a
remote server and Hive without going through the hassle of storing the
data first on memory before loading it to Hive.
>From what I have read so far there is no such command but it would not
hurt to ask.
Is it possible to insert data through an insert query in hive? (The
equivalent to insert into table_name
values (...) in xSQLx)
Thank you in advance for an answer.
Cyrille Djoko
Data Mining Developer Intern
Re: Hive Queries
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
[-user@hive, +user@sqoop]
Hi Cyrille,
this seems to me more a Sqoop issue than Hive issue, so I've moved this email to user@sqoop mailing list. I'm keeping user@hive in Bcc so that the mailing list will get the memo. Please join the user@sqoop mailing list [1] to receive additional feedback.
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> class org.apache.hadoop.mapreduce.JobContext, but interface was expected
Exception that you're getting is typical when one is running code compiled against Hadoop 2.0 on Hadoop 1.0 or vice versa. You've specified that you're running Sqoop 1.4.2 --hadoop1.0.0.0, but that do not seem to be the case. Would you mind downloading it again from our mirror [2] and retrying it?
Jarcec
Links:
1: http://sqoop.apache.org/mail-lists.html
2: http://www.apache.org/dist/sqoop/1.4.2/
On Fri, Feb 15, 2013 at 03:20:09PM -0500, Cyrille Djoko wrote:
> Hi Jarcec,
> I did try Sqoop. I am running sqoop 1.4.2 --hadoop1.0.0 along with hadoop
> 1.0.4 But I keep running on the following exception.
>
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> class org.apache.hadoop.mapreduce.JobContext, but interface was expected
>
> So I wrote a small program but all I can do is send queries to the server.
> > Hi Cyrille,
> > I'm not exactly sure what exactly you mean, so I'm more or less blindly
> > shooting, but maybe Apache Sqoop [1] might help you?
> >
> > Jarcec
> >
> > Links:
> > 1: http://sqoop.apache.org/
> >
> > On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote:
> >> I am looking for a relatively efficient way of transferring data between
> >> a
> >> remote server and Hive without going through the hassle of storing the
> >> data first on memory before loading it to Hive.
> >> From what I have read so far there is no such command but it would not
> >> hurt to ask.
> >> Is it possible to insert data through an insert query in hive? (The
> >> equivalent to insert into table_name
> >> values (...) in xSQLx)
> >>
> >> Thank you in advance for an answer.
> >>
> >>
> >> Cyrille Djoko
> >> Data Mining Developer Intern
> >>
> >
>
>
> Cyrille Djoko
>
> Agnik LLC
> Data Mining Developer Intern
>
Re: Hive Queries
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
[-user@hive, +user@sqoop]
Hi Cyrille,
this seems to me more a Sqoop issue than Hive issue, so I've moved this email to user@sqoop mailing list. I'm keeping user@hive in Bcc so that the mailing list will get the memo. Please join the user@sqoop mailing list [1] to receive additional feedback.
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> class org.apache.hadoop.mapreduce.JobContext, but interface was expected
Exception that you're getting is typical when one is running code compiled against Hadoop 2.0 on Hadoop 1.0 or vice versa. You've specified that you're running Sqoop 1.4.2 --hadoop1.0.0.0, but that do not seem to be the case. Would you mind downloading it again from our mirror [2] and retrying it?
Jarcec
Links:
1: http://sqoop.apache.org/mail-lists.html
2: http://www.apache.org/dist/sqoop/1.4.2/
On Fri, Feb 15, 2013 at 03:20:09PM -0500, Cyrille Djoko wrote:
> Hi Jarcec,
> I did try Sqoop. I am running sqoop 1.4.2 --hadoop1.0.0 along with hadoop
> 1.0.4 But I keep running on the following exception.
>
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> class org.apache.hadoop.mapreduce.JobContext, but interface was expected
>
> So I wrote a small program but all I can do is send queries to the server.
> > Hi Cyrille,
> > I'm not exactly sure what exactly you mean, so I'm more or less blindly
> > shooting, but maybe Apache Sqoop [1] might help you?
> >
> > Jarcec
> >
> > Links:
> > 1: http://sqoop.apache.org/
> >
> > On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote:
> >> I am looking for a relatively efficient way of transferring data between
> >> a
> >> remote server and Hive without going through the hassle of storing the
> >> data first on memory before loading it to Hive.
> >> From what I have read so far there is no such command but it would not
> >> hurt to ask.
> >> Is it possible to insert data through an insert query in hive? (The
> >> equivalent to insert into table_name
> >> values (...) in xSQLx)
> >>
> >> Thank you in advance for an answer.
> >>
> >>
> >> Cyrille Djoko
> >> Data Mining Developer Intern
> >>
> >
>
>
> Cyrille Djoko
>
> Agnik LLC
> Data Mining Developer Intern
>
Re: Hive Queries
Posted by Cyrille Djoko <cd...@agnik.com>.
Hi Jarcec,
I did try Sqoop. I am running sqoop 1.4.2 --hadoop1.0.0 along with hadoop
1.0.4 But I keep running on the following exception.
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
class org.apache.hadoop.mapreduce.JobContext, but interface was expected
So I wrote a small program but all I can do is send queries to the server.
> Hi Cyrille,
> I'm not exactly sure what exactly you mean, so I'm more or less blindly
> shooting, but maybe Apache Sqoop [1] might help you?
>
> Jarcec
>
> Links:
> 1: http://sqoop.apache.org/
>
> On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote:
>> I am looking for a relatively efficient way of transferring data between
>> a
>> remote server and Hive without going through the hassle of storing the
>> data first on memory before loading it to Hive.
>> From what I have read so far there is no such command but it would not
>> hurt to ask.
>> Is it possible to insert data through an insert query in hive? (The
>> equivalent to insert into table_name
>> values (...) in xSQLx)
>>
>> Thank you in advance for an answer.
>>
>>
>> Cyrille Djoko
>> Data Mining Developer Intern
>>
>
Cyrille Djoko
Agnik LLC
Data Mining Developer Intern
Re: NULLable STRUCTs
Posted by Michael Malak <mi...@yahoo.com>.
If no one has any objection, I'm going to update HIVE-4022, which I entered a week ago when I thought the behavior was Avro-specific, to indicate it actually affects even native Hive tables.
https://issues.apache.org/jira/browse/HIVE-4022
--- On Fri, 2/15/13, Michael Malak <mi...@yahoo.com> wrote:
> From: Michael Malak <mi...@yahoo.com>
> Subject: NULLable STRUCTs
> To: user@hive.apache.org
> Date: Friday, February 15, 2013, 5:03 PM
> It seems that all Hive columns (at
> least those of primitive types) are always NULLable?
> What about columns of type STRUCT?
>
> The following:
>
> echo 1,2 >twovalues.csv
> hive
> CREATE TABLE tc (x INT, y INT) ROW FORMAT DELIMITED FIELDS
> TERMINATED BY ',';
> LOAD DATA LOCAL INPATH 'twovalues.csv' INTO TABLE tc;
> CREATE TABLE oc (z STRUCT<a: int, b: int>);
> INSERT INTO TABLE oc SELECT null FROM tc;
>
> produces the error
>
> FAILED: SemanticException [Error 10044]: Line 1:18 Cannot
> insert into target table because column number/types are
> different 'oc': Cannot convert column 0 from void to
> struct<a:int,b:int>.
>
> I initially discovered such behavior with Avro-backed
> tables, and even entered a JIRA
> https://issues.apache.org/jira/browse/HIVE-4022
> but now I realized it happens with CSV-backed tables as
> well.
>
> Perhaps related, perhaps not, it seems that all members of a
> STRUCT are always non-NULLable. The following:
>
> INSERT INTO TABLE oc SELECT named_struct('a', null, 'b',
> null) FROM tc;
>
> produces the error:
>
> FAILED: SemanticException [Error 10044]: Line 1:18 Cannot
> insert into target table because column number/types are
> different 'oc': Cannot convert column 0 from
> struct<a:void,b:void> to struct<a:int,b:int>.
>
>
NULLable STRUCTs
Posted by Michael Malak <mi...@yahoo.com>.
It seems that all Hive columns (at least those of primitive types) are always NULLable? What about columns of type STRUCT?
The following:
echo 1,2 >twovalues.csv
hive
CREATE TABLE tc (x INT, y INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
LOAD DATA LOCAL INPATH 'twovalues.csv' INTO TABLE tc;
CREATE TABLE oc (z STRUCT<a: int, b: int>);
INSERT INTO TABLE oc SELECT null FROM tc;
produces the error
FAILED: SemanticException [Error 10044]: Line 1:18 Cannot insert into target table because column number/types are different 'oc': Cannot convert column 0 from void to struct<a:int,b:int>.
I initially discovered such behavior with Avro-backed tables, and even entered a JIRA
https://issues.apache.org/jira/browse/HIVE-4022
but now I realized it happens with CSV-backed tables as well.
Perhaps related, perhaps not, it seems that all members of a STRUCT are always non-NULLable. The following:
INSERT INTO TABLE oc SELECT named_struct('a', null, 'b', null) FROM tc;
produces the error:
FAILED: SemanticException [Error 10044]: Line 1:18 Cannot insert into target table because column number/types are different 'oc': Cannot convert column 0 from struct<a:void,b:void> to struct<a:int,b:int>.
Re: Hive Queries
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Cyrille,
I'm not exactly sure what exactly you mean, so I'm more or less blindly shooting, but maybe Apache Sqoop [1] might help you?
Jarcec
Links:
1: http://sqoop.apache.org/
On Fri, Feb 15, 2013 at 01:44:45PM -0500, Cyrille Djoko wrote:
> I am looking for a relatively efficient way of transferring data between a
> remote server and Hive without going through the hassle of storing the
> data first on memory before loading it to Hive.
> From what I have read so far there is no such command but it would not
> hurt to ask.
> Is it possible to insert data through an insert query in hive? (The
> equivalent to insert into table_name
> values (...) in xSQLx)
>
> Thank you in advance for an answer.
>
>
> Cyrille Djoko
> Data Mining Developer Intern
>