RE: SQOOP v1.4.6 - sqoop export error

["Xu, Qian A" <qi...@intel.com>]

Good tip!

From: Timothy Garza [mailto:timothy.garza@collinsongroup.com]
Sent: Saturday, June 20, 2015 1:36 AM
To: user@sqoop.apache.org
Subject: RE: SQOOP v1.4.6 - sqoop export error

I noticed the text containing my resolution has gone missing.

This problem was caused by data-type mismatch between the HIVE table column (STRING) and the target RDBMS column (CHAR, VARCHAR or TEXT). Basically Hive STRING will not convert naturally into these ANSI data-types, I had to CAST the STRING into CHAR before the SQOOP Map-Reduce would let me move the data across.

I noticed other people encountering this issue but nobody has posted a solution.

________________________________
Hi there

I had trouble with SQOOP export out of the box and have researched (ie. googled) this one to death.

I wanted to publish the answer here for future generations as I myself couldn’t find a solution and worked literally for hours with a colleague to investigate these errors.

I configured SQOOP locally successfully, which I confirmed by calling ‘sqoop version’:

[hadoop@ip-123-45-67-890 ~]$ sqoop version
15/06/19 09:07:29 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
Sqoop 1.4.6
git commit id c0c5a81723759fa575844a0a1eae8f510fa32c25
Compiled by root on Mon Apr 27 14:38:36 CST 2015

Linux version:    Amazon Linux AMI release 2015.03

Syntax:
sqoop export --connect "jdbc:sqlserver://***URL GOES HERE***.us-west-2.rds.amazonaws.com:1433;databaseName=<databasename>" --table <tablename> --export-dir hdfs://<IPgoeshere>:<port>/user/hive/warehouse/<hive_file> --username <username> --password <password>

15/06/19 08:44:33 INFO mapreduce.Job: Task Id : attempt_1434676168391_0012_m_000001_0, Status : FAILED
Error: java.lang.ClassCastException: org.apache.hadoop.io.BytesWritable cannot be cast to org.apache.hadoop.io.LongWritable
        at org.apache.sqoop.mapreduce.CombineShimRecordReader.getCurrentKey(CombineShimRecordReader.java:95)
        at org.apache.sqoop.mapreduce.CombineShimRecordReader.getCurrentKey(CombineShimRecordReader.java:38)
        at org.apache.sqoop.mapreduce.CombineFileRecordReader.getCurrentKey(CombineFileRecordReader.java:79)
        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.getCurrentKey(MapTask.java:516)
        at org.apache.hadoop.mapreduce.task.MapContextImpl.getCurrentKey(MapContextImpl.java:70)
        at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.getCurrentKey(WrappedMapper.java:81)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:152)
        at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170)


Kind Regards

Timothy Garza


[Collinson Group]<http://www.collinsongroup.com/>




The Collinson Group Limited; Registered number: 2577557, Registered in England & Wales; Registered Office: Cutlers Exchange, 123 Houndsditch, London, EC3A 7BU.


This e-mail may contain privileged and confidential information and/or copyright material and is intended for the use of the addressee only. If you receive this e-mail by mistake please advise the sender immediately by using the reply facility in your e-mail software and delete this e-mail from your computer system. You may not deliver, copy or disclose its contents to anyone else. Any unauthorised use may be unlawful. Any views expressed in this e-mail are those of the individual sender and may not necessarily reflect the views of The Collinson Group Ltd and/or its subsidiaries or any other associated company (collectively “Collinson Group”).

As communications via the Internet are not secure Collinson Group cannot accept any liability if this e-mail is accessed by third parties during the course of transmission or is modified or amended in any way following despatch. Collinson Group cannot guarantee that any attachment to this email does not contain a virus, therefore it is strongly recommended that you carry out your own virus check before opening any attachment, as we cannot accept liability for any damage sustained as a result of software virus infection. Senders of messages shall be taken to consent to the monitoring and recording of e-mails addressed to members of the Company.