[jira] [Created] (SQOOP-3227) Sqoop Avro import with decimal mapping issue

["liviu (JIRA)" <ji...@apache.org>]

liviu created SQOOP-3227:
----------------------------

             Summary: Sqoop Avro import with decimal mapping issue
                 Key: SQOOP-3227
                 URL: https://issues.apache.org/jira/browse/SQOOP-3227
             Project: Sqoop
          Issue Type: Bug
            Reporter: liviu
            Priority: Blocker


Hi,

We are using Sqoop version 1.4.6-cdh5.8.5 for importing numeric data types from Oracle to Avro Decimal Logical type (Bytes as Decimal with precision and scale)
The import works ok except for cases in which the values stored in Oracle have the scale lower than the one defined in table ddl.
Ex. the column is defined as NUMERIC(20,2) and the value stored is "3.2" (only one digit after decimal point); in this case we receive the error message:
*_Error: org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroTypeException: Cannot encode decimal with scale 1 as scale 2_*

This error was discussed in [https://issues.apache.org/jira/browse/AVRO-1864] with the resolution that this sqoop behavior is correct (it cannot add by default an extra info for transforming "3.2" to "3.20")

We tried below methods for conversion to AVRO Decimal(20,2) Logical datatype:

1). use   *--map-column-java "COLNAME=java.math.BigDecimal"* in sqoop import command

error received:
_*Error: org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroTypeException: Cannot encode decimal with scale 1 as scale 2*_

2). use _*--map-column-java "COL1=Decimal(20%2C2)"*_ in sqoop command

error received:
 *_ERROR tool.ImportTool: Import failed: No ResultSet method for Java type Decimal(20,2)_*

3). made the column as varchar2(100) in Oracle database, stored the value as "3.20" and use _*--map-column-java "COLNAME=java.math.BigDecimal"* _in sqoop command

error received: 

*_ERROR tool.ImportTool: Import failed: Cannot convert to AVRO type java.math.BigDecimal_*


There is any way in which we can instruct sqoop to map the column from Oracle table to AVRO Decimal logical type (precision=20, scale=2) and import the "3.2" value from Oracle database as "3.20" type Decimal(20,2) in AVRO file?

Thanks,
Liviu





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)