You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Alessandra Tiengo <at...@daisylabs.it> on 2014/07/25 17:05:22 UTC
Import NULL value HBase
Hi,
I imported a table in HDFS from a SQL database using SQOOP. Null values have been replaced by "null" string, so a row is (for instance): rowkey, value1, value2, null, null
After that I imported the data from HDFS to a HBase table using first org.apache.hadoop.hbase.mapreduce.ImportTsv and then org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.
How can I manage these "null" values so that they are not imported in the HBase table?
Currently they are present as "null" string.
I also tried to use --null-string '\\N' --null-non-string '\\N', but HBase stored \N.
Thanks,
Alessandra
Alessandra Tiengo
DaisyLabs
Polo Tecnologico
Via Fratelli Cuzio 42
Pavia (PV), 27100 Italy
Mobile: +39.392 9631888
desk: +39.0382 1726245
fax: + 39.0382 049387
e-mail: atiengo@daisylabs.it<ma...@daisylabs.it>
www: www.daisylabs.it<https://mail.daisylabs.it/exchweb/bin/redir.asp?URL=https://mail.daisylabs.it/exchweb/bin/redir.asp?URL=http://www.daisylabs.it>
P Rispettiamo l'ambiente. Se non è necessario, non stampare questa mail
Re: Import NULL value HBase
Posted by Gwen Shapira <gs...@cloudera.com>.
Let us know how it goes. If there are performance issues, we may be
able to fix them :)
On Tue, Jul 29, 2014 at 7:34 AM, Alessandra Tiengo <at...@daisylabs.it> wrote:
> Thanks for your reply!
> I read that using Sqoop to import from MySQL directly to HBase is not very performant, but I will try it.
>
> Alessandra
>
> -----Original Message-----
> From: Gwen Shapira [mailto:gshapira@cloudera.com]
> Sent: venerdì 25 luglio 2014 19:24
> To: user@sqoop.apache.org
> Subject: Re: Import NULL value HBase
>
> Any reason you are not using Sqoop to import from MySQL directly to HBase? This seems like the easiest solution.
>
> Other possible solutions:
> - Use the empty string as null value
> - Modify ImportTSV so it will skip the value "null" in a text file.
>
> Gwen
>
> On Fri, Jul 25, 2014 at 8:05 AM, Alessandra Tiengo <at...@daisylabs.it> wrote:
>> Hi,
>>
>> I imported a table in HDFS from a SQL database using SQOOP. Null
>> values have been replaced by “null” string, so a row is (for
>> instance): rowkey, value1, value2, null, null
>>
>>
>>
>> After that I imported the data from HDFS to a HBase table using first
>> org.apache.hadoop.hbase.mapreduce.ImportTsv and then
>> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.
>>
>> How can I manage these “null” values so that they are not imported in
>> the HBase table?
>>
>> Currently they are present as “null” string.
>>
>>
>>
>> I also tried to use --null-string ‘\\N’ --null-non-string ‘\\N’, but
>> HBase stored \N.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Alessandra
>>
>>
>>
RE: Import NULL value HBase
Posted by Alessandra Tiengo <at...@daisylabs.it>.
Thanks for your reply!
I read that using Sqoop to import from MySQL directly to HBase is not very performant, but I will try it.
Alessandra
-----Original Message-----
From: Gwen Shapira [mailto:gshapira@cloudera.com]
Sent: venerdì 25 luglio 2014 19:24
To: user@sqoop.apache.org
Subject: Re: Import NULL value HBase
Any reason you are not using Sqoop to import from MySQL directly to HBase? This seems like the easiest solution.
Other possible solutions:
- Use the empty string as null value
- Modify ImportTSV so it will skip the value "null" in a text file.
Gwen
On Fri, Jul 25, 2014 at 8:05 AM, Alessandra Tiengo <at...@daisylabs.it> wrote:
> Hi,
>
> I imported a table in HDFS from a SQL database using SQOOP. Null
> values have been replaced by “null” string, so a row is (for
> instance): rowkey, value1, value2, null, null
>
>
>
> After that I imported the data from HDFS to a HBase table using first
> org.apache.hadoop.hbase.mapreduce.ImportTsv and then
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.
>
> How can I manage these “null” values so that they are not imported in
> the HBase table?
>
> Currently they are present as “null” string.
>
>
>
> I also tried to use --null-string ‘\\N’ --null-non-string ‘\\N’, but
> HBase stored \N.
>
>
>
> Thanks,
>
>
>
> Alessandra
>
>
>
Re: Import NULL value HBase
Posted by Gwen Shapira <gs...@cloudera.com>.
Any reason you are not using Sqoop to import from MySQL directly to
HBase? This seems like the easiest solution.
Other possible solutions:
- Use the empty string as null value
- Modify ImportTSV so it will skip the value "null" in a text file.
Gwen
On Fri, Jul 25, 2014 at 8:05 AM, Alessandra Tiengo <at...@daisylabs.it> wrote:
> Hi,
>
> I imported a table in HDFS from a SQL database using SQOOP. Null values have
> been replaced by “null” string, so a row is (for instance): rowkey, value1,
> value2, null, null
>
>
>
> After that I imported the data from HDFS to a HBase table using first
> org.apache.hadoop.hbase.mapreduce.ImportTsv and then
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.
>
> How can I manage these “null” values so that they are not imported in the
> HBase table?
>
> Currently they are present as “null” string.
>
>
>
> I also tried to use --null-string ‘\\N’ --null-non-string ‘\\N’, but HBase
> stored \N.
>
>
>
> Thanks,
>
>
>
> Alessandra
>
>
>
>
>
> Alessandra Tiengo
>
> DaisyLabs
>
> Polo Tecnologico
>
> Via Fratelli Cuzio 42
> Pavia (PV), 27100 Italy
>
>
>
> Mobile: +39.392 9631888
>
> desk: +39.0382 1726245
>
> fax: + 39.0382 049387
>
> e-mail: atiengo@daisylabs.it
>
> www: www.daisylabs.it
>
> P Rispettiamo l'ambiente. Se non è necessario, non stampare questa mail
>
>