Review Request 39271: SQOOP-2612: Allow rules-based column mapping

[John Todd <jt...@gmail.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39271/
-----------------------------------------------------------

Review request for Sqoop.


Repository: sqoop-trunk


Description
-------

Allow users to map source columns to Java and Avro types based on rules rather than a list.

About half the data types on my current project are NUMERIC, and many of those have a SCALE of 0. It would be helpful to be able to process those as Integers or Longs. The map-column-java feature is helpful, but it requires continuous surveillance of the source tables and maintenance of lists of columns and what type they should be mapped to.

It would be helpful to be able to create a rules-based interpretation of the type inference provided by Sqoop.


Diffs
-----

  .gitignore 68cbe28 
  src/java/com/cloudera/sqoop/hive/HiveTypes.java e67f6b8 
  src/java/com/cloudera/sqoop/manager/ConnManager.java a4b4457 
  src/java/org/apache/sqoop/SqoopOptions.java ef6e0ce 
  src/java/org/apache/sqoop/hive/HiveTypes.java ad00535 
  src/java/org/apache/sqoop/hive/TableDefWriter.java c9962e9 
  src/java/org/apache/sqoop/manager/ConnManager.java d9569c5 
  src/java/org/apache/sqoop/manager/CubridManager.java 5a1a0e8 
  src/java/org/apache/sqoop/manager/MainframeManager.java 101f3ce 
  src/java/org/apache/sqoop/manager/MySQLManager.java e1d5a36 
  src/java/org/apache/sqoop/manager/OracleManager.java 69b613f 
  src/java/org/apache/sqoop/manager/SQLServerManager.java 9a3d918 
  src/java/org/apache/sqoop/manager/SqlManager.java ead581d 
  src/java/org/apache/sqoop/manager/oracle/OraOopConnManager.java 55a2b4c 
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 93d438a 
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java 4070c24 
  src/java/org/apache/sqoop/orm/AvroSchemaGenerator.java 2576673 
  src/java/org/apache/sqoop/orm/ClassWriter.java 1c6f7f4 
  src/java/org/apache/sqoop/orm/ColumnMapper.java PRE-CREATION 
  src/java/org/apache/sqoop/orm/DefaultColumnMapper.java PRE-CREATION 
  src/java/org/apache/sqoop/orm/InferringColumnMapper.java PRE-CREATION 
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java c97bb58 
  src/java/org/apache/sqoop/tool/ImportTool.java c79e044 
  src/java/org/apache/sqoop/util/SqlTypeMap.java cd5d468 
  src/java/org/apache/sqoop/util/SqlTypeObject.java PRE-CREATION 
  src/test/com/cloudera/sqoop/TestAvroImport.java af83ce0 
  src/test/com/cloudera/sqoop/TestConnFactory.java 59c3455 
  src/test/com/cloudera/sqoop/hive/TestTableDefWriter.java 55e572e 
  src/test/com/cloudera/sqoop/manager/TestSqlManager.java 57855fa 
  src/test/com/cloudera/sqoop/testutil/HsqldbTestServer.java 8d0a30d 
  src/test/org/apache/sqoop/manager/TestMainframeManager.java 79cbcb1 
  src/test/org/apache/sqoop/manager/sqlserver/SQLServerManagerManualTest.java ee576c9 
  src/test/org/apache/sqoop/orm/TestAvroSchemaGenerator.java PRE-CREATION 
  src/test/org/apache/sqoop/util/TestSqlTypeObject.java PRE-CREATION 

Diff: https://reviews.apache.org/r/39271/diff/


Testing
-------

Created new test classes TestAvroSchemaGenerator, TestSqlTypeObject
Added tests to TestAvroImport
All existing unit tests pass


Thanks,

John Todd