You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Markus Kemper (JIRA)" <ji...@apache.org> on 2016/12/10 17:31:59 UTC
[jira] [Created] (SQOOP-3076) Modify default support with (import +
--hcatalog + --as-textfile) and (DATE,TIMESTAMP)
Markus Kemper created SQOOP-3076:
------------------------------------
Summary: Modify default support with (import + --hcatalog + --as-textfile) and (DATE,TIMESTAMP)
Key: SQOOP-3076
URL: https://issues.apache.org/jira/browse/SQOOP-3076
Project: Sqoop
Issue Type: Improvement
Reporter: Markus Kemper
Please consider modifying the default behavior when RDBMS types are supported in Hive as seen in the test case below for (DATE and TIMESTAMP)
Test Case
{noformat}
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "drop table t1_dates"
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "create table t1_dates (c1_int integer, c2_date date, c3_timestamp timestamp)"
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "select dbms_metadata.get_ddl('TABLE', 'T1_DATES', 'SQOOP') from dual"
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "insert into t1_dates values (1, current_date, current_timestamp)"
sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "select * from t1_dates"
Output:
------------------------
| DBMS_METADATA.GET_DDL('TABLE','T1_DATES','SQOOP') |
------------------------
|
CREATE TABLE "SQOOP"."T1_DATES"
( "C1_INT" NUMBER(*,0),
"C2_DATE" DATE,
"C3_TIMESTAMP" TIMESTAMP (6)
) SEGMENT CREATION DEFERRED
PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255
NOCOMPRESS LOGGING
TABLESPACE "SQOOP" |
------------------------
---
------------------------------------------------
| C1_INT | C2_DATE | C3_TIMESTAMP |
------------------------------------------------
| 1 | 2016-12-10 15:48:23.0 | 2016-12-10 15:48:23.707327 |
------------------------------------------------
#################
# STEP 02 - Demonstrate Current Default Behavior
#################
sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table T1_DATES --hcatalog-database default --hcatalog-table t1_dates_text --create-hcatalog-table --hcatalog-storage-stanza 'stored as textfile' --num-mappers 1
beeline -u jdbc:hive2:// -e "use default; describe t1_dates_text; select * from t1_dates_text;"
+---------------+----------------+----------+--+
| col_name | data_type | comment |
+---------------+----------------+----------+--+
| c1_int | decimal(38,0) | |
| c2_date | string | |
| c3_timestamp | string | |
+---------------+----------------+----------+--+
---
+-----------------------+------------------------+-----------------------------+--+
| t1_dates_text.c1_int | t1_dates_text.c2_date | t1_dates_text.c3_timestamp |
+-----------------------+------------------------+-----------------------------+--+
| 1 | 2016-12-10 15:48:23.0 | 2016-12-10 15:48:23.707327 |
+-----------------------+------------------------+-----------------------------+--+
#################
# STEP 03 - Demonstrate Suggested Default Behavior
#################
beeline -u jdbc:hive2:// -e "use default; drop table t1_dates_text;"
sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table T1_DATES --hcatalog-database default --hcatalog-table t1_dates_text --create-hcatalog-table --hcatalog-storage-stanza 'stored as textfile' --num-mappers 1 --map-column-hive c2_date=date,c3_timestamp=timestamp
beeline -u jdbc:hive2:// -e "use default; describe t1_dates_text; select * from t1_dates_text;"
+---------------+----------------+----------+--+
| col_name | data_type | comment |
+---------------+----------------+----------+--+
| c1_int | decimal(38,0) | |
| c2_date | date | |
| c3_timestamp | timestamp | |
+---------------+----------------+----------+--+
--
+-----------------------+------------------------+-----------------------------+--+
| t1_dates_text.c1_int | t1_dates_text.c2_date | t1_dates_text.c3_timestamp |
+-----------------------+------------------------+-----------------------------+--+
| 1 | 2016-12-10 | 2016-12-10 15:48:23.707327 |
+-----------------------+------------------------+-----------------------------+--+
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)