You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hive.apache.org by Krishna Rao <kr...@gmail.com> on 2012/12/04 18:04:56 UTC

Problems Sqoop importing columns with NULLs

Hi all,

I'm haivng trouble transfering NULLs in a VARCHAR column in a table in
PostgresQL into Hive. A null value ends up as an empty value in Hive,
rather than NULL.

I'm running the following command:

sqoop import --username <username> -P --hive-import --hive-overwrite
--null-string='\\N' --null-non-string '\\N' --direct --compression-codec
org.apache.hadoop.io.compress.SnappyCodec

I'm using Sqoop version 1.4.1 & Hive 0.9.0

Cheers

Re: Problems Sqoop importing columns with NULLs

Posted by Krishna Rao <kr...@gmail.com>.

Thanks Jarek. Good to hear it's at least a known issue.

On 4 December 2012 17:20, Jarek Jarcec Cecho <ja...@apache.org> wrote:

> Hi Krishna,
> I'm afraid that this is known limitation of current PostgreSQL direct
> connector. We already have a JIRA to address this - SQOOP-654 [1].
>
> Currently suggested workaround is to use JDBC based import by dropping the
> "--direct" argument.
>
> Links:
> 1: https://issues.apache.org/jira/browse/SQOOP-654
>
> On Tue, Dec 04, 2012 at 05:04:56PM +0000, Krishna Rao wrote:
> > Hi all,
> >
> > I'm haivng trouble transfering NULLs in a VARCHAR column in a table in
> > PostgresQL into Hive. A null value ends up as an empty value in Hive,
> > rather than NULL.
> >
> > I'm running the following command:
> >
> > sqoop import --username <username> -P --hive-import --hive-overwrite
> > --null-string='\\N' --null-non-string '\\N' --direct --compression-codec
> > org.apache.hadoop.io.compress.SnappyCodec
> >
> > I'm using Sqoop version 1.4.1 & Hive 0.9.0
> >
> > Cheers
>

Re: Problems Sqoop importing columns with NULLs

Posted by Krishna Rao <kr...@gmail.com>.

Thanks Jarek. Good to hear it's at least a known issue.

On 4 December 2012 17:20, Jarek Jarcec Cecho <ja...@apache.org> wrote:

> Hi Krishna,
> I'm afraid that this is known limitation of current PostgreSQL direct
> connector. We already have a JIRA to address this - SQOOP-654 [1].
>
> Currently suggested workaround is to use JDBC based import by dropping the
> "--direct" argument.
>
> Links:
> 1: https://issues.apache.org/jira/browse/SQOOP-654
>
> On Tue, Dec 04, 2012 at 05:04:56PM +0000, Krishna Rao wrote:
> > Hi all,
> >
> > I'm haivng trouble transfering NULLs in a VARCHAR column in a table in
> > PostgresQL into Hive. A null value ends up as an empty value in Hive,
> > rather than NULL.
> >
> > I'm running the following command:
> >
> > sqoop import --username <username> -P --hive-import --hive-overwrite
> > --null-string='\\N' --null-non-string '\\N' --direct --compression-codec
> > org.apache.hadoop.io.compress.SnappyCodec
> >
> > I'm using Sqoop version 1.4.1 & Hive 0.9.0
> >
> > Cheers
>

Re: Problems Sqoop importing columns with NULLs

Posted by Jarek Jarcec Cecho <ja...@apache.org>.

Hi Krishna,
I'm afraid that this is known limitation of current PostgreSQL direct connector. We already have a JIRA to address this - SQOOP-654 [1].

Currently suggested workaround is to use JDBC based import by dropping the "--direct" argument.

Links:
1: https://issues.apache.org/jira/browse/SQOOP-654

On Tue, Dec 04, 2012 at 05:04:56PM +0000, Krishna Rao wrote:
> Hi all,
> 
> I'm haivng trouble transfering NULLs in a VARCHAR column in a table in
> PostgresQL into Hive. A null value ends up as an empty value in Hive,
> rather than NULL.
> 
> I'm running the following command:
> 
> sqoop import --username <username> -P --hive-import --hive-overwrite
> --null-string='\\N' --null-non-string '\\N' --direct --compression-codec
> org.apache.hadoop.io.compress.SnappyCodec
> 
> I'm using Sqoop version 1.4.1 & Hive 0.9.0
> 
> Cheers