migrate data from SQLServer to AWS

[David Liu <da...@gmail.com>]

Hell all,

I am investigating possible solutions to data migration from SQLServer at
local data center to either
AWS S3, Redshift, or Hive instances on AWS. Either way, I may want to
migrate from data center to AWS.

Is there any such use cases using Sqoop or even w/o Sqoop ?

Alternatively, if I do not migrate all the data, I may want to use
Spark/Hive on AWS to do some analysis based on the data from Sql server in
real time, so is there such a 'middleware' that could talk to sql server
and load data to Hive on AWS with relatively low latency?

Cheers,

David

NoSuchMethodError: org.apache.hadoop.hbase.HTableDescriptor.addFamily

[行云流水 <xi...@163.com>]

Hi,all：
when I use the following command to import data from mysql to hbase, I meet the problem:
command: sqoop import --connect jdbc:mysql://test.host:3306/hive --table test --username hive --password hive --hbase-create-table --hbase-table version --column-family C -m 1 --verbose


I have made all the necessary installation and environment is sourced.From the stackoverflow,I found the answer: keep the compatible version, but my env is as  follow:


    Hadoop version 2.6.0 HBase 1.0.1.1 Running Sqoop version: 1.4.6


I tried to compile the sqoop-1.4.6 with hbase 1.0.1.1, but failed. Could you help me out ?  Thanks advance!




Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V
at org.apache.sqoop.mapreduce.HBaseImportJob.jobSetup(HBaseImportJob.java:222)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)