Re: Sqoop + HCatalog + Oracle escape new line character from column data

[Suraj Nayak <sn...@gmail.com>]

Hi Everyone,

To solve the *\n* problem mentioned in the previous email, use following
solution:

Use --map-column-java option to explicitly state the column is of type
String. Then --hive-drop-import-delims works as expected (to remove \n from
data).

Changed Sqoop Command :

sqoop import --connect jdbc:oracle:thin:@ORA_IP:ORA_PORT:ORA_SID
\--username user123 --password passwd123 -table SCHEMA.TBL_2 \
--hcatalog-table tbl2 --hcatalog-database testdb --num-mappers 1 \
--split-by SOME_ID --columns col1,col2,col3,col4
--hive-drop-import-delims \--outdir /tmp/temp_table_loc --class-name
"SqoopWithHCAT" \--null-string "" --map-column-java col4=String

But Sqoop can maintain a mapping for predefined SQL datatypes so that it
can infer whether to use String or not. Thus making
--hive-drop-import-delims  work out of the box.

Can Developers help me identify whether am missing something of can I raise
a JIRA for this? (I am willing to contribute for this issue).

Answered in stackoverflow :
http://stackoverflow.com/questions/28076200/hive-drop-import-delims-not-removing-newline-while-using-hcatalog-in-sqoop/28424184#28424184

Thanks!

On Wed, Jan 21, 2015 at 12:32 PM, Suraj Nayak <sn...@gmail.com> wrote:

> Hi All,
>
> Sqoop while used with HCatalog import not able to remove new line (\n)
> from column data even after using --hive-drop-import-delims option in the
> command when running Apache Sqoop with Oracle.
>
> Sqoop Query:
>
>     sqoop import --connect jdbc:oracle:thin:@ORA_IP:ORA_PORT:ORA_SID \--username user123 --password passwd123 -table SCHEMA.TBL_2 \ --hcatalog-table tbl2 --hcatalog-database testdb --num-mappers 1 \ --split-by SOME_ID --columns col1,col2,col3,col4 --hive-drop-import-delims \--outdir /tmp/temp_table_loc --class-name "Sqoop with HCAT" \--null-string ""
>
> Data in Oracle Column *col4* as below: (Data has control characters such
> as ^M)
>
> <li>Details:^M
>     <ul>^M
>         <li>
>
> Does Control character causing this problem?
>
> Am I missing anything ? Is there any workaround or solution for this
> problem?
>
>
> Stackoverflow Link:-
> http://stackoverflow.com/questions/28076200/hive-drop-import-delims-not-removing-newline-while-using-hcatalog-in-sqoop
>
> --
> Thanks
> Suraj Nayak M
>



-- 
Thanks
Suraj Nayak M