You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2019/04/05 14:16:01 UTC
[jira] [Commented] (SQOOP-3134) --class-name should override
default Avro schema name
[ https://issues.apache.org/jira/browse/SQOOP-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810872#comment-16810872 ]
ASF subversion and git services commented on SQOOP-3134:
--------------------------------------------------------
Commit 1a04d2007d7e4e111d9cde9efbe3485547075fc6 in sqoop's branch refs/heads/trunk from daniel voros
[ https://gitbox.apache.org/repos/asf?p=sqoop.git;h=1a04d20 ]
SQOOP-3134: --class-name should override default Avro schema name
> --class-name should override default Avro schema name
> -----------------------------------------------------
>
> Key: SQOOP-3134
> URL: https://issues.apache.org/jira/browse/SQOOP-3134
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Markus Kemper
> Assignee: Daniel Voros
> Priority: Major
> Attachments: SQOOP-3134.1.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Please consider adding an option to configure the Avro schema output file name that is created with Sqoop (import + --as-avrodatafile), example cases below.
> {noformat}
> #################
> # STEP 01 - Create Data
> #################
> export MYCONN=jdbc:mysql://mysql.cloudera.com:3306/db_coe
> export MYUSER=sqoop
> export MYPSWD=cloudera
> sqoop list-tables --connect $MYCONN --username $MYUSER --password $MYPSWD
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "drop table t1"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "create table t1 (c1 int, c2 date, c3 varchar(10))"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "insert into t1 values (1, current_date, 'some data')"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "select * from t1"
> -----------------------------------------
> | c1 | c2 | c3 |
> -----------------------------------------
> | 1 | 2017-02-13 | some data |
> -----------------------------------------
> #################
> # STEP 02 - Import + --table + --as-avrodatafile
> #################
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table t1 --target-dir /user/root/t1 --delete-target-dir --num-mappers 1 --as-avrodatafile
> ls -l ./*
> Output:
> 17/02/13 12:14:52 INFO mapreduce.ImportJobBase: Transferred 413 bytes in 20.6988 seconds (19.9529 bytes/sec)
> 17/02/13 12:14:52 INFO mapreduce.ImportJobBase: Retrieved 1 records.
> ~~~~
> -rw-r--r-- 1 root root 492 Feb 13 12:14 ./t1.avsc <==== want option to configure this file name
> -rw-r--r-- 1 root root 12462 Feb 13 12:14 ./t1.java
> #################
> # STEP 03 - Import + --query + --as-avrodatafile
> #################
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --query "select * from t1 where \$CONDITIONS" --split-by c1 --target-dir /user/root/t1 --delete-target-dir --num-mappers 1 --as-avrodatafile
> ls -l ./*
> Output:
> 17/02/13 12:16:58 INFO mapreduce.ImportJobBase: Transferred 448 bytes in 25.2757 seconds (17.7245 bytes/sec)
> 17/02/13 12:16:58 INFO mapreduce.ImportJobBase: Retrieved 1 records.
> ~~~~~
> -rw-r--r-- 1 root root 527 Feb 13 12:16 ./AutoGeneratedSchema.avsc <==== want option to configure this file name
> -rw-r--r-- 1 root root 12590 Feb 13 12:16 ./QueryResult.java
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)