You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Kevin Zhang <zh...@yahoo.com> on 2013/11/19 21:16:12 UTC
cannot see hive table after import
Just started to use Hadoop, Hive, Sqoop today. Sorry for any stupid questions.
I managed to run a sqoop-import command. The command output says "Hive import complete". But I cannot see the table within hive. Hive command "show tables" show nothing.
My Sqoop command is:
/bin/sqoop-import --connect jdbc:mysql://xxx.com/yyyy --table user_entity --username uuuuuu --password ppppp --hive-table h_user_entity --create-hive-table
>
My Hive command is:
show tables;
Sqoop command output is:
Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
>Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
>Please set $HCAT_HOME to the root of your HCatalog installation.
>Warning: $HADOOP_HOME is deprecated.
>
>
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
>13/11/19 15:02:16 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
>13/11/19 15:02:16 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: It seems that you've specified at least one of following:
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-home
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-overwrite
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --create-hive-table
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-table
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-key
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-value
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --map-column-hive
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: Without specifying parameter --hive-import. Please note that
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: those arguments will not be used in this session. Either
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: specify --hive-import to apply them correctly or remove them
>13/11/19 15:02:16 WARN tool.BaseSqoopTool: from command line to remove this warning.
>13/11/19 15:02:16 INFO tool.BaseSqoopTool: Please note that --hive-home, --hive-partition-key,
>13/11/19 15:02:16 INFO tool.BaseSqoopTool: hive-partition-value and --map-column-hive options are
>13/11/19 15:02:16 INFO tool.BaseSqoopTool: are also valid for HCatalog imports and exports
>13/11/19 15:02:16 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
>13/11/19 15:02:16 INFO tool.CodeGenTool: Beginning code generation
>13/11/19 15:02:16 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_entity` AS t LIMIT 1
>13/11/19 15:02:16 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_entity` AS t LIMIT 1
>13/11/19 15:02:16 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /data/hadoop/current
>Note: /tmp/sqoop-root/compile/39e8d314afcc1b81c1cc50a6c3d396b1/user_entity.java uses or overrides a deprecated API.
>Note: Recompile with -Xlint:deprecation for details.
>13/11/19 15:02:17 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/39e8d314afcc1b81c1cc50a6c3d396b1/user_entity.jar
>13/11/19 15:02:17 WARN manager.MySQLManager: It looks like you are importing from mysql.
>13/11/19 15:02:17 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
>13/11/19 15:02:17 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
>13/11/19 15:02:17 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
>13/11/19 15:02:17 INFO mapreduce.ImportJobBase: Beginning import of user_entity
>13/11/19 15:02:18 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`id`), MAX(`id`) FROM `user_entity`
>13/11/19 15:02:18 WARN db.TextSplitter: Generating splits for a textual index column.
>13/11/19 15:02:18 WARN db.TextSplitter: If your database sorts in a case-insensitive order, this may result in a partial import or duplicate records.
>13/11/19 15:02:18 WARN db.TextSplitter: You are strongly encouraged to choose an integral split column.
>13/11/19 15:02:19 INFO mapred.JobClient: Running job: job_201311181910_0007
>13/11/19 15:02:20 INFO mapred.JobClient: map 0% reduce 0%
>13/11/19 15:02:29 INFO mapred.JobClient: map 16% reduce 0%
>13/11/19 15:02:32 INFO mapred.JobClient: map 33% reduce 0%
>13/11/19 15:02:35 INFO mapred.JobClient: map 50% reduce 0%
>13/11/19 15:02:39 INFO mapred.JobClient: map 66% reduce 0%
>13/11/19 15:02:42 INFO mapred.JobClient: map 83% reduce 0%
>13/11/19 15:02:43 INFO mapred.JobClient: map 100% reduce 0%
>13/11/19 15:02:44 INFO mapred.JobClient: Job complete: job_201311181910_0007
>13/11/19 15:02:44 INFO mapred.JobClient: Counters: 18
>13/11/19 15:02:44 INFO mapred.JobClient: Job Counters
>13/11/19 15:02:44 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=38087
>13/11/19 15:02:44 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
>13/11/19 15:02:44 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
>13/11/19 15:02:44 INFO mapred.JobClient: Launched map tasks=6
>13/11/19 15:02:44 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
>13/11/19 15:02:44 INFO mapred.JobClient: File Output Format Counters
>13/11/19 15:02:44 INFO mapred.JobClient: Bytes Written=85103032
>13/11/19 15:02:44 INFO mapred.JobClient: FileSystemCounters
>13/11/19 15:02:44 INFO mapred.JobClient: HDFS_BYTES_READ=825
>13/11/19 15:02:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=389460
>13/11/19 15:02:44 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=85103032
>13/11/19 15:02:44 INFO mapred.JobClient: File Input Format Counters
>13/11/19 15:02:44 INFO mapred.JobClient: Bytes Read=0
>13/11/19 15:02:44 INFO mapred.JobClient: Map-Reduce Framework
>13/11/19 15:02:44 INFO mapred.JobClient: Map input records=387423
>13/11/19 15:02:44 INFO mapred.JobClient: Physical memory (bytes) snapshot=787578880
>13/11/19 15:02:44 INFO mapred.JobClient: Spilled Records=0
>13/11/19 15:02:44 INFO mapred.JobClient: CPU time spent (ms)=22420
>13/11/19 15:02:44 INFO mapred.JobClient: Total committed heap usage (bytes)=1205403648
>13/11/19 15:02:44 INFO mapred.JobClient: Virtual memory (bytes) snapshot=6980870144
>13/11/19 15:02:44 INFO mapred.JobClient: Map output records=387423
>13/11/19 15:02:44 INFO mapred.JobClient: SPLIT_RAW_BYTES=825
>13/11/19 15:02:44 INFO mapreduce.ImportJobBase: Transferred 81.1606 MB in 26.1394 seconds (3.1049 MB/sec)
>13/11/19 15:02:44 INFO mapreduce.ImportJobBase: Retrieved 387423 records.
>13/11/19 15:02:44 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_entity` AS t LIMIT 1
>13/11/19 15:02:44 INFO hive.HiveImport: Removing temporary files from import process: hdfs://localhost:9000/user/root/user_entity/_logs
>13/11/19 15:02:44 INFO hive.HiveImport: Loading uploaded data into Hive
>13/11/19 15:02:46 INFO hive.HiveImport:
>13/11/19 15:02:46 INFO hive.HiveImport: Logging initialized using configuration in jar:file:/data/hive/hive-0.12.0/lib/hive-common-0.12.0.jar!/hive-log4j.properties
>13/11/19 15:02:51 INFO hive.HiveImport: OK
>13/11/19 15:02:51 INFO hive.HiveImport: Time taken: 5.221 seconds
>13/11/19 15:02:51 INFO hive.HiveImport: Loading data to table default.h_user_entity
>13/11/19 15:02:52 INFO hive.HiveImport: Table default.h_user_entity stats: [num_partitions: 0, num_files: 7, num_rows: 0, total_size: 85103032, raw_data_size: 0]
>13/11/19 15:02:52 INFO hive.HiveImport: OK
>13/11/19 15:02:52 INFO hive.HiveImport: Time taken: 0.636 seconds
>13/11/19 15:02:52 INFO hive.HiveImport: Hive import complete.
>13/11/19 15:02:52 INFO hive.HiveImport: Export directory is empty, removing it.
Any help is highly appreciated.
-Kevin
Re: cannot see hive table after import
Posted by Kevin Zhang <zh...@yahoo.com>.
It turns out that hive uses the current directory as the metadata store location.
Running the sqoop command and hive command from the same location fixed my problem.
________________________________
From: "phil@insufficient-light.com" <ph...@insufficient-light.com>
To: user@sqoop.apache.org
Sent: Tuesday, November 19, 2013 2:44 PM
Subject: Re: cannot see hive table after import
It looks like the answer lies here:
13/11/19 15:02:16 WARN tool.BaseSqoopTool: It seems that you've specified
at least one of following:
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-home
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-overwrite
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --create-hive-table
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-table
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-key
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-value
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --map-column-hive
13/11/19 15:02:16 WARN tool.BaseSqoopTool: Without specifying parameter
--hive-import. Please note that
13/11/19 15:02:16 WARN tool.BaseSqoopTool: those arguments will not be
used in this session. Either
13/11/19 15:02:16 WARN tool.BaseSqoopTool: specify --hive-import to apply
them correctly or remove them
13/11/19 15:02:16 WARN tool.BaseSqoopTool: from command line to remove
this warning.
Perhaps you should try re-running the command you used, adding the
--hive-import parameter.
> Just started to use Hadoop, Hive, Sqoop today. Sorry for any stupid
> questions.
>
> I managed to run a sqoop-import command. The command output says "Hive
> import complete". But I cannot see the table within hive. Hive command
> "show tables" show nothing.
>
> My Sqoop command is:
> /bin/sqoop-import --connect jdbc:mysql://xxx.com/yyyy --table user_entity
> --username uuuuuu --password ppppp --hive-table
> h_user_entity --create-hive-table
>>
> My Hive command is:
> show tables;
> Sqoop command output is:
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
>>Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
>>Please set $HCAT_HOME to the root of your HCatalog installation.
>>Warning: $HADOOP_HOME is deprecated.
>>
>>
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
>> for output. You can override
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: delimiters with
>> --fields-terminated-by, etc.
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: It seems that you've specified
>> at least one of following:
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-home
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-overwrite
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --create-hive-table
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-table
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-key
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-value
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --map-column-hive
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: Without specifying parameter
>> --hive-import. Please note that
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: those arguments will not be
>> used in this session. Either
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: specify --hive-import to apply
>> them correctly or remove them
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: from command line to remove
>> this warning.
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: Please note that --hive-home,
>> --hive-partition-key,
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: hive-partition-value and
>> --map-column-hive options are
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: are also valid for HCatalog
>> imports and exports
>>13/11/19 15:02:16 INFO manager.MySQLManager: Preparing to use a MySQL
>> streaming resultset.
>>13/11/19 15:02:16 INFO tool.CodeGenTool: Beginning code generation
>>13/11/19 15:02:16 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM `user_entity` AS t LIMIT 1
>>13/11/19 15:02:16 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM `user_entity` AS t LIMIT 1
>>13/11/19 15:02:16 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>> /data/hadoop/current
>>Note:
>> /tmp/sqoop-root/compile/39e8d314afcc1b81c1cc50a6c3d396b1/user_entity.java
>> uses or overrides a deprecated API.
>>Note: Recompile with -Xlint:deprecation for details.
>>13/11/19 15:02:17 INFO orm.CompilationManager: Writing jar file:
>> /tmp/sqoop-root/compile/39e8d314afcc1b81c1cc50a6c3d396b1/user_entity.jar
>>13/11/19 15:02:17 WARN manager.MySQLManager: It looks like you are
>> importing from mysql.
>>13/11/19 15:02:17 WARN manager.MySQLManager: This transfer can be faster!
>> Use the --direct
>>13/11/19 15:02:17 WARN manager.MySQLManager: option to exercise a
>> MySQL-specific fast path.
>>13/11/19 15:02:17 INFO manager.MySQLManager: Setting zero DATETIME
>> behavior to convertToNull (mysql)
>>13/11/19 15:02:17 INFO mapreduce.ImportJobBase: Beginning import of
>> user_entity
>>13/11/19 15:02:18 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
>> SELECT MIN(`id`), MAX(`id`) FROM `user_entity`
>>13/11/19 15:02:18 WARN db.TextSplitter: Generating splits for a textual
>> index column.
>>13/11/19 15:02:18 WARN db.TextSplitter: If your database sorts in a
>> case-insensitive order, this may result in a partial import or duplicate
>> records.
>>13/11/19 15:02:18 WARN db.TextSplitter: You are strongly encouraged to
>> choose an integral split column.
>>13/11/19 15:02:19 INFO mapred.JobClient: Running job:
>> job_201311181910_0007
>>13/11/19 15:02:20 INFO mapred.JobClient: map 0% reduce 0%
>>13/11/19 15:02:29 INFO mapred.JobClient: map 16% reduce 0%
>>13/11/19 15:02:32 INFO mapred.JobClient: map 33% reduce 0%
>>13/11/19 15:02:35 INFO mapred.JobClient: map 50% reduce 0%
>>13/11/19 15:02:39 INFO mapred.JobClient: map 66% reduce 0%
>>13/11/19 15:02:42 INFO mapred.JobClient: map 83% reduce 0%
>>13/11/19 15:02:43 INFO mapred.JobClient: map 100% reduce 0%
>>13/11/19 15:02:44 INFO mapred.JobClient: Job complete:
>> job_201311181910_0007
>>13/11/19 15:02:44 INFO mapred.JobClient: Counters: 18
>>13/11/19 15:02:44 INFO mapred.JobClient: Job Counters
>>13/11/19 15:02:44 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=38087
>>13/11/19 15:02:44 INFO mapred.JobClient: Total time spent by all
>> reduces waiting after reserving slots (ms)=0
>>13/11/19 15:02:44 INFO mapred.JobClient: Total time spent by all maps
>> waiting after reserving slots (ms)=0
>>13/11/19 15:02:44 INFO mapred.JobClient: Launched map tasks=6
>>13/11/19 15:02:44 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
>>13/11/19 15:02:44 INFO mapred.JobClient: File Output Format Counters
>>13/11/19 15:02:44 INFO mapred.JobClient: Bytes Written=85103032
>>13/11/19 15:02:44 INFO mapred.JobClient: FileSystemCounters
>>13/11/19 15:02:44 INFO mapred.JobClient: HDFS_BYTES_READ=825
>>13/11/19 15:02:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=389460
>>13/11/19 15:02:44 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=85103032
>>13/11/19 15:02:44 INFO mapred.JobClient: File Input Format Counters
>>13/11/19 15:02:44 INFO mapred.JobClient: Bytes Read=0
>>13/11/19 15:02:44 INFO mapred.JobClient: Map-Reduce Framework
>>13/11/19 15:02:44 INFO mapred.JobClient: Map input records=387423
>>13/11/19 15:02:44 INFO mapred.JobClient: Physical memory (bytes)
>> snapshot=787578880
>>13/11/19 15:02:44 INFO mapred.JobClient: Spilled Records=0
>>13/11/19 15:02:44 INFO mapred.JobClient: CPU time spent (ms)=22420
>>13/11/19 15:02:44 INFO mapred.JobClient: Total committed heap usage
>> (bytes)=1205403648
>>13/11/19 15:02:44 INFO mapred.JobClient: Virtual memory (bytes)
>> snapshot=6980870144
>>13/11/19 15:02:44 INFO mapred.JobClient: Map output records=387423
>>13/11/19 15:02:44 INFO mapred.JobClient: SPLIT_RAW_BYTES=825
>>13/11/19 15:02:44 INFO mapreduce.ImportJobBase: Transferred 81.1606 MB in
>> 26.1394 seconds (3.1049 MB/sec)
>>13/11/19 15:02:44 INFO mapreduce.ImportJobBase: Retrieved 387423 records.
>>13/11/19 15:02:44 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM `user_entity` AS t LIMIT 1
>>13/11/19 15:02:44 INFO hive.HiveImport: Removing temporary files from
>> import process: hdfs://localhost:9000/user/root/user_entity/_logs
>>13/11/19 15:02:44 INFO hive.HiveImport: Loading uploaded data into Hive
>>13/11/19 15:02:46 INFO hive.HiveImport:
>>13/11/19 15:02:46 INFO hive.HiveImport: Logging initialized using
>> configuration in
>> jar:file:/data/hive/hive-0.12.0/lib/hive-common-0.12.0.jar!/hive-log4j.properties
>>13/11/19 15:02:51 INFO hive.HiveImport: OK
>>13/11/19 15:02:51 INFO hive.HiveImport: Time taken: 5.221 seconds
>>13/11/19 15:02:51 INFO hive.HiveImport: Loading data to table
>> default.h_user_entity
>>13/11/19 15:02:52 INFO hive.HiveImport: Table default.h_user_entity
>> stats: [num_partitions: 0, num_files: 7, num_rows: 0, total_size:
>> 85103032, raw_data_size: 0]
>>13/11/19 15:02:52 INFO hive.HiveImport: OK
>>13/11/19 15:02:52 INFO hive.HiveImport: Time taken: 0.636 seconds
>>13/11/19 15:02:52 INFO hive.HiveImport: Hive import complete.
>>13/11/19 15:02:52 INFO hive.HiveImport: Export directory is empty,
>> removing it.
>
> Any help is highly appreciated.
>
> -Kevin
Re: cannot see hive table after import
Posted by ph...@insufficient-light.com.
It looks like the answer lies here:
13/11/19 15:02:16 WARN tool.BaseSqoopTool: It seems that you've specified
at least one of following:
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-home
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-overwrite
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --create-hive-table
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-table
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-key
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-value
13/11/19 15:02:16 WARN tool.BaseSqoopTool: --map-column-hive
13/11/19 15:02:16 WARN tool.BaseSqoopTool: Without specifying parameter
--hive-import. Please note that
13/11/19 15:02:16 WARN tool.BaseSqoopTool: those arguments will not be
used in this session. Either
13/11/19 15:02:16 WARN tool.BaseSqoopTool: specify --hive-import to apply
them correctly or remove them
13/11/19 15:02:16 WARN tool.BaseSqoopTool: from command line to remove
this warning.
Perhaps you should try re-running the command you used, adding the
--hive-import parameter.
> Just started to use Hadoop, Hive, Sqoop today. Sorry for any stupid
> questions.
>
> I managed to run a sqoop-import command. The command output says "Hive
> import complete". But I cannot see the table within hive. Hive command
> "show tables" show nothing.
>
> My Sqoop command is:
> /bin/sqoop-import --connect jdbc:mysql://xxx.com/yyyy --table user_entity
> --username uuuuuu --password ppppp --hive-table
> h_user_entity --create-hive-table
>>
> My Hive command is:
> show tables;
> Sqoop command output is:
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
>>Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
>>Please set $HCAT_HOME to the root of your HCatalog installation.
>>Warning: $HADOOP_HOME is deprecated.
>>
>>
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: Using Hive-specific delimiters
>> for output. You can override
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: delimiters with
>> --fields-terminated-by, etc.
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: It seems that you've specified
>> at least one of following:
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-home
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-overwrite
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --create-hive-table
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-table
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-key
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --hive-partition-value
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: --map-column-hive
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: Without specifying parameter
>> --hive-import. Please note that
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: those arguments will not be
>> used in this session. Either
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: specify --hive-import to apply
>> them correctly or remove them
>>13/11/19 15:02:16 WARN tool.BaseSqoopTool: from command line to remove
>> this warning.
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: Please note that --hive-home,
>> --hive-partition-key,
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: hive-partition-value and
>> --map-column-hive options are
>>13/11/19 15:02:16 INFO tool.BaseSqoopTool: are also valid for HCatalog
>> imports and exports
>>13/11/19 15:02:16 INFO manager.MySQLManager: Preparing to use a MySQL
>> streaming resultset.
>>13/11/19 15:02:16 INFO tool.CodeGenTool: Beginning code generation
>>13/11/19 15:02:16 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM `user_entity` AS t LIMIT 1
>>13/11/19 15:02:16 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM `user_entity` AS t LIMIT 1
>>13/11/19 15:02:16 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>> /data/hadoop/current
>>Note:
>> /tmp/sqoop-root/compile/39e8d314afcc1b81c1cc50a6c3d396b1/user_entity.java
>> uses or overrides a deprecated API.
>>Note: Recompile with -Xlint:deprecation for details.
>>13/11/19 15:02:17 INFO orm.CompilationManager: Writing jar file:
>> /tmp/sqoop-root/compile/39e8d314afcc1b81c1cc50a6c3d396b1/user_entity.jar
>>13/11/19 15:02:17 WARN manager.MySQLManager: It looks like you are
>> importing from mysql.
>>13/11/19 15:02:17 WARN manager.MySQLManager: This transfer can be faster!
>> Use the --direct
>>13/11/19 15:02:17 WARN manager.MySQLManager: option to exercise a
>> MySQL-specific fast path.
>>13/11/19 15:02:17 INFO manager.MySQLManager: Setting zero DATETIME
>> behavior to convertToNull (mysql)
>>13/11/19 15:02:17 INFO mapreduce.ImportJobBase: Beginning import of
>> user_entity
>>13/11/19 15:02:18 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
>> SELECT MIN(`id`), MAX(`id`) FROM `user_entity`
>>13/11/19 15:02:18 WARN db.TextSplitter: Generating splits for a textual
>> index column.
>>13/11/19 15:02:18 WARN db.TextSplitter: If your database sorts in a
>> case-insensitive order, this may result in a partial import or duplicate
>> records.
>>13/11/19 15:02:18 WARN db.TextSplitter: You are strongly encouraged to
>> choose an integral split column.
>>13/11/19 15:02:19 INFO mapred.JobClient: Running job:
>> job_201311181910_0007
>>13/11/19 15:02:20 INFO mapred.JobClient: map 0% reduce 0%
>>13/11/19 15:02:29 INFO mapred.JobClient: map 16% reduce 0%
>>13/11/19 15:02:32 INFO mapred.JobClient: map 33% reduce 0%
>>13/11/19 15:02:35 INFO mapred.JobClient: map 50% reduce 0%
>>13/11/19 15:02:39 INFO mapred.JobClient: map 66% reduce 0%
>>13/11/19 15:02:42 INFO mapred.JobClient: map 83% reduce 0%
>>13/11/19 15:02:43 INFO mapred.JobClient: map 100% reduce 0%
>>13/11/19 15:02:44 INFO mapred.JobClient: Job complete:
>> job_201311181910_0007
>>13/11/19 15:02:44 INFO mapred.JobClient: Counters: 18
>>13/11/19 15:02:44 INFO mapred.JobClient: Job Counters
>>13/11/19 15:02:44 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=38087
>>13/11/19 15:02:44 INFO mapred.JobClient: Total time spent by all
>> reduces waiting after reserving slots (ms)=0
>>13/11/19 15:02:44 INFO mapred.JobClient: Total time spent by all maps
>> waiting after reserving slots (ms)=0
>>13/11/19 15:02:44 INFO mapred.JobClient: Launched map tasks=6
>>13/11/19 15:02:44 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
>>13/11/19 15:02:44 INFO mapred.JobClient: File Output Format Counters
>>13/11/19 15:02:44 INFO mapred.JobClient: Bytes Written=85103032
>>13/11/19 15:02:44 INFO mapred.JobClient: FileSystemCounters
>>13/11/19 15:02:44 INFO mapred.JobClient: HDFS_BYTES_READ=825
>>13/11/19 15:02:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=389460
>>13/11/19 15:02:44 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=85103032
>>13/11/19 15:02:44 INFO mapred.JobClient: File Input Format Counters
>>13/11/19 15:02:44 INFO mapred.JobClient: Bytes Read=0
>>13/11/19 15:02:44 INFO mapred.JobClient: Map-Reduce Framework
>>13/11/19 15:02:44 INFO mapred.JobClient: Map input records=387423
>>13/11/19 15:02:44 INFO mapred.JobClient: Physical memory (bytes)
>> snapshot=787578880
>>13/11/19 15:02:44 INFO mapred.JobClient: Spilled Records=0
>>13/11/19 15:02:44 INFO mapred.JobClient: CPU time spent (ms)=22420
>>13/11/19 15:02:44 INFO mapred.JobClient: Total committed heap usage
>> (bytes)=1205403648
>>13/11/19 15:02:44 INFO mapred.JobClient: Virtual memory (bytes)
>> snapshot=6980870144
>>13/11/19 15:02:44 INFO mapred.JobClient: Map output records=387423
>>13/11/19 15:02:44 INFO mapred.JobClient: SPLIT_RAW_BYTES=825
>>13/11/19 15:02:44 INFO mapreduce.ImportJobBase: Transferred 81.1606 MB in
>> 26.1394 seconds (3.1049 MB/sec)
>>13/11/19 15:02:44 INFO mapreduce.ImportJobBase: Retrieved 387423 records.
>>13/11/19 15:02:44 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM `user_entity` AS t LIMIT 1
>>13/11/19 15:02:44 INFO hive.HiveImport: Removing temporary files from
>> import process: hdfs://localhost:9000/user/root/user_entity/_logs
>>13/11/19 15:02:44 INFO hive.HiveImport: Loading uploaded data into Hive
>>13/11/19 15:02:46 INFO hive.HiveImport:
>>13/11/19 15:02:46 INFO hive.HiveImport: Logging initialized using
>> configuration in
>> jar:file:/data/hive/hive-0.12.0/lib/hive-common-0.12.0.jar!/hive-log4j.properties
>>13/11/19 15:02:51 INFO hive.HiveImport: OK
>>13/11/19 15:02:51 INFO hive.HiveImport: Time taken: 5.221 seconds
>>13/11/19 15:02:51 INFO hive.HiveImport: Loading data to table
>> default.h_user_entity
>>13/11/19 15:02:52 INFO hive.HiveImport: Table default.h_user_entity
>> stats: [num_partitions: 0, num_files: 7, num_rows: 0, total_size:
>> 85103032, raw_data_size: 0]
>>13/11/19 15:02:52 INFO hive.HiveImport: OK
>>13/11/19 15:02:52 INFO hive.HiveImport: Time taken: 0.636 seconds
>>13/11/19 15:02:52 INFO hive.HiveImport: Hive import complete.
>>13/11/19 15:02:52 INFO hive.HiveImport: Export directory is empty,
>> removing it.
>
> Any help is highly appreciated.
>
> -Kevin