Can we use Sqoop to load BLOB into HIVE

[王国栋 <wa...@gmail.com>]

HI guys,

I am using sqoop 1.4.2 to import data into HIVE warehosue.

In my understanding, BLOB and CLOB can not map to any HIVE data type. But I
am not sure if we can do some hack to import BLOB into HIVE.

*My questions are as follow*
1. Can we use "--map-column-hive" to map blob to HIVE string ?
2. Usually, BLOB may contain '\n', '\A', these are HIVE field separator and
record separator, obviously, we can not save BLOB as TextFile. Can we
import blob as SequenceFile or AvroFile which can be loaded with HIVE ?


Thanks.

Guodong

Re: Can we use Sqoop to load BLOB into HIVE

[Jarek Jarcec Cecho <ja...@apache.org>]

Hi Guodong,
CLOB stands for "character large object" and thus it should be equivalent with Hive's STRING. The BLOB ("binary large object") can generally contain any binary data (such as pictures or PDFs) that are not generally suitable for Hive and thus their encoding might be quite tricky.

What exactly are you storing in the BLOB columns? If it's just a string, you can take advantage of the --map-column-hive and --hive-drop-input-delms/--hive-delims-replacement for your particular use case.

Jarcec

On Mon, Oct 28, 2013 at 06:13:35PM +0800, 王国栋 wrote:
> HI guys,
> 
> I am using sqoop 1.4.2 to import data into HIVE warehosue.
> 
> In my understanding, BLOB and CLOB can not map to any HIVE data type. But I
> am not sure if we can do some hack to import BLOB into HIVE.
> 
> *My questions are as follow*
> 1. Can we use "--map-column-hive" to map blob to HIVE string ?
> 2. Usually, BLOB may contain '\n', '\A', these are HIVE field separator and
> record separator, obviously, we can not save BLOB as TextFile. Can we
> import blob as SequenceFile or AvroFile which can be loaded with HIVE ?
> 
> 
> Thanks.
> 
> Guodong