You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Han Sen Tey <ax...@yahoo.com> on 2013/05/21 14:56:16 UTC
SQOOP export from Hadoop Hive to MySQL
Greeting experts,
I have a table "CHRYSLER_Anametrix_Aggregated" in Hive and in MySQL.
Similarly I can find this table in hdfs --
/user/hive/warehouse/chrysler_anametrix_aggregated.
I can successfully execute the following SQOOP command from namenode CLI -
sqoop export --verbose --connect
jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
CHRYSLER_Anametrix_Aggregated --update-key
VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
--update-mode allowinsert --export-dir
/user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
--input-fields-terminated-by '|' --input-null-string '\\N'
--input-null-non-string '\\N'
However, when I want to run the above as part of Oozie workflow action ...
<action name="sqoop-node">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>sqoop.connection.factories</name>
<value>com.cloudera.sqoop.manager.DefaultManagerFactory</value>
</property>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
<property>
<name>mapred.compress.map.output</name>
<value>true</value>
</property>
<property>
<name>oozie.service.WorkflowAppService.system.libpath</name>
<value>${nameNode}/user/oozie/share/lib/sqoop</value>
</property>
</configuration>
<command>export --verbose --connect
jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
CHRYSLER_Anametrix_Aggregated --update-key
VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
--update-mode allowinsert --export-dir
/user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
--input-fields-terminated-by '|' --input-null-string '\\N'
--input-null-non-string '\\N'</command>
</sqoop>
<ok to="end"/>
<error to="fail"/>
</action>
I get the following error message. I have "mysql-connector-java.5.1.24-bin.jar" shared library - /user/oozie/share/lib/sqoop.
Hadoop version - 2.00 - CDH4.2.0
Sqoop version - 1.4.2. - CDH4.2.0
Can you experts please guide me ... a rookie to Hadoop and its projects.
2013-05-21 00:48:35,135 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:48:37,394 WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id 2013-05-21 00:48:37,408 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2013-05-21 00:48:38,042 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2013-05-21 00:48:38,058 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5dce1bea 2013-05-21 00:48:38,423 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/input/dummy.txt:0+5 2013-05-21 00:48:38,453 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES
is deprecated. Use FileInputFormatCounters as group name and BYTES_READ as counter name instead 2013-05-21 00:48:38,459 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0 2013-05-21 00:48:39,743 WARN org.apache.sqoop.tool.SqoopTool: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. 2013-05-21 00:48:40,032 DEBUG org.apache.sqoop.tool.BaseSqoopTool: Enabled debug logging. 2013-05-21 00:48:40,036 WARN org.apache.sqoop.SqoopOptions: Character argument '|' has multiple characters; only the first will be used. 2013-05-21 00:48:40,084 DEBUG org.apache.sqoop.ConnFactory: sqoop.connection.factories is set; ignoring managers.d 2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21 00:48:40,140
DEBUG org.apache.sqoop.manager.DefaultManagerFactory: Trying with scheme: jdbc:mysql: 2013-05-21 00:48:40,183 INFO org.apache.sqoop.manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2013-05-21 00:48:40,183 DEBUG org.apache.sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.MySQLManager@795e0c2b 2013-05-21 00:48:40,184 INFO org.apache.sqoop.tool.CodeGenTool: Beginning code generation 2013-05-21 00:48:40,233 DEBUG org.apache.sqoop.manager.SqlManager: No connection paramenters specified. Using regular API for making connection. 2013-05-21 00:48:40,911 DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21 00:48:40,911 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,930 DEBUG org.apache.sqoop.orm.ClassWriter: selected columns: 2013-05-21 00:48:40,931 DEBUG
org.apache.sqoop.orm.ClassWriter: VisitorID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorDay 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorMonth 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorYear 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorSession 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: Brand 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceType 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceBrowser 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: TrafficSource 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: SiteID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: BuyID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: AdvertisementID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
PlacementID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: CreativeID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: PaidKeyword 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model200 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model200Conv 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model300 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model300TownCountry 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: BMOLineup 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOModels 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOColors 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOOptions 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOSummary 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureLineup 2013-05-21
00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureForm 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureMailLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureDownloadLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: PreDL 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DealerLocate 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLHours 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLServices 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: DLDealerSite 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: PreSNI 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SearchNewInventory 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SNIVIN 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SNIDetails 2013-05-21 00:48:40,936 DEBUG
org.apache.sqoop.orm.ClassWriter: SNIDealer 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: TestDriveForm 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: TestDriveComplete 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: Inventives 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreCal 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PaymentCalculator 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreGAQ 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: GetAQuote 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: LandingPage 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Models 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PhotosGallery 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Interior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
Exterior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Performance 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Safety 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Specifications 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: ModelHighlights 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Dimensions 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Warranty 2013-05-21 00:48:40,965 DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21 00:48:40,965 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Writing source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,990 DEBUG
org.apache.sqoop.orm.ClassWriter: Table name: CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Columns: VisitorID:12, VisitorDay:12, VisitorMonth:12, VisitorYear:12, VisitorSession:4, Brand:12, DeviceType:12, DeviceBrowser:12, TrafficSource:12, SiteID:12, BuyID:12, AdvertisementID:12, PlacementID:12, CreativeID:12, PaidKeyword:12, Model200:4, Model200Conv:4, Model300:4, Model300TownCountry:4, BMOLineup:4, BMOModels:4, BMOColors:4, BMOOptions:4, BMOSummary:4, BrochureLineup:4, BrochureForm:4, BrochureMailLink:4, BrochureDownloadLink:4, PreDL:4, DealerLocate:4, DLHours:4, DLServices:4, DLDealerSite:4, PreSNI:4, SearchNewInventory:4, SNIVIN:4, SNIDetails:4, SNIDealer:4, TestDriveForm:4, TestDriveComplete:4, Inventives:4, PreCal:4, PaymentCalculator:4, PreGAQ:4, GetAQuote:4, LandingPage:4, Models:4, PhotosGallery:4, Interior:4, Exterior:4, Performance:4, Safety:4, Specifications:4, ModelHighlights:4,
Dimensions:4, Warranty:4, 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: sourceFilename is CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,991 DEBUG org.apache.sqoop.orm.CompilationManager: Found existing /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:40,992 INFO org.apache.sqoop.orm.CompilationManager: $HADOOP_MAPRED_HOME is not set 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Adding source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Invoking javac with args: 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: -sourcepath 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: -d
2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager: -classpath 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:
/run/cloudera-scm-agent/process/76-mapreduce-TASKTRACKER:/usr/lib/jvm/j2sdk1.6-oracle/lib/tools.jar:/usr/lib/hadoop-0.20-mapreduce:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-
0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/j
asper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jline-0.9.94.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-map
reduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar:/usr/share/cmf/lib/plugins/navigator-plugin-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/event-publish-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.5.1.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:
/usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/
usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/avro-1.7.3.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-c
onfiguration-1.6.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hado
op/lib/hue-plugins-2.2.0-cdh4.2.0.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common.jar:/mnt/mapred/local/taskTracker/distcache/-3359360379488511723_1277082233_1150217957/ip-10-147-191-48.ec2.internal/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/sqoop-launcher.jar:/mnt/mapred/local/taskTracker/distcache/-3926870669093555941_811939984_1868217431/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoo
p/avro-ipc-1.7.3-tests.jar:/mnt/mapred/local/taskTracker/distcache/4298483340499219542_2138859044_1868217453/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-ipc-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/-1975491928978283532_-1185784129_1868217475/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-mapred-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/8448357694523829347_-1491317642_1868217500/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-1.7.0.jar:/mnt/mapred/local/taskTracker/distcache/-334079317866666788_-232552041_1868217523/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-core-1.8.0.jar:/mnt/mapred/local/taskTracker/distcache/-644448586073609072_-372575484_1868217547/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-configuration-1.6.jar:/mnt/mapred/local/taskTracker/distcache/246054495199880993_357051411_1868217589/ip-10-147-191-48.ec2.internal/
user/oozie/share/lib/sqoop/commons-digester-1.8.jar:/mnt/mapred/local/taskTracker/distcache/-4144140590487253933_766503300_1868217614/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-io-2.1.jar:/mnt/mapred/local/taskTracker/distcache/5753546228468760323_-553874526_1868217665/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/guava-11.0.2.jar:/mnt/mapred/local/taskTracker/distcache/8662866019753074519_-1003108238_1868217924/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hbase-0.94.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4840377369495417802_845918870_1868217954/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/high-scale-lib-1.1.1.jar:/mnt/mapred/local/taskTracker/distcache/-142623586016294918_445946554_1868217996/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hsqldb-1.8.0.7.jar:/mnt/mapred/local/taskTracker/distcache/5142583573507929188_-105788043_1868218025/ip-10-147-191-48.ec2.internal/us
er/oozie/share/lib/sqoop/httpclient-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/-6413369533723682352_-180904991_1868218051/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/httpcore-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/3446550291828223401_633399508_1868218088/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/jsr305-1.3.9.jar:/mnt/mapred/local/taskTracker/distcache/-6492962697058022370_-616322509_1868218133/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/libthrift-0.9.0.jar:/mnt/mapred/local/taskTracker/distcache/7049151581973813525_2017354882_1868218156/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/metrics-core-2.1.2.jar:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar:/mnt/mapred/local/taskTracker/distcache/-7982675208196719520_-1207761111_1868218197/ip-10-147-191-48.ec2.interna
l/user/oozie/share/lib/sqoop/netty-3.4.0.Final.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-412267918101123238_-517736257_1868215515/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/oozie/json-simple-1.1.jar:/mnt/mapred/local/taskTracker/ubuntu/jobcache/job_201305131705_0181/attempt_201305131705_0181_m_000000_0/work:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,206 INFO org.apache.sqoop.orm.CompilationManager: Writing jar file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar 2013-05-21 00:48:48,210 DEBUG
org.apache.sqoop.orm.CompilationManager: Scanning for .class files in directory: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca 2013-05-21 00:48:48,221 DEBUG org.apache.sqoop.orm.CompilationManager: Got classfile: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.class -> CHRYSLER_Anametrix_Aggregated.class 2013-05-21 00:48:48,236 DEBUG org.apache.sqoop.orm.CompilationManager: Finished writing jar file /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: MySQL Connector upsert functionality is using INSERT ON 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: DUPLICATE KEY UPDATE clause that relies on table's unique key. 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: Insert/update distinction is therefore independent on column 2013-05-21 00:48:48,243 WARN
org.apache.sqoop.manager.MySQLManager: names specified in --update-key parameter. Please see MySQL 2013-05-21 00:48:48,245 WARN org.apache.sqoop.manager.MySQLManager: documentation for additional limitations. 2013-05-21 00:48:48,290 INFO org.apache.sqoop.mapreduce.ExportJobBase: Beginning export of CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:48,319 DEBUG org.apache.sqoop.mapreduce.JobBase: Using InputFormat: class org.apache.sqoop.mapreduce.ExportInputFormat 2013-05-21 00:48:48,575 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,581 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
file:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar 2013-05-21 00:48:48,585 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,588 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,588 WARN org.apache.sqoop.mapreduce.JobBase: SQOOP_HOME is unset. May not be able to find all job dependencies. 2013-05-21 00:48:48,739 WARN org.apache.hadoop.mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications
should implement Tool for the same. 2013-05-21 00:48:49,058 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Target numMapTasks=1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Total input bytes=8187428 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: maxSplitSize=8187428 2013-05-21 00:48:49,063 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Generated splits: 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Paths:/user/hive/warehouse/chrysler_anametrix_aggregated/part-AG-00000-1.txt:0+8187428 Locations:ip-10-147-191-48.ec2.internal:; 2013-05-21 00:48:49,403 INFO org.apache.hadoop.mapred.JobClient: Running job: job_201305131705_0182
2013-05-21 00:48:50,417 INFO org.apache.hadoop.mapred.JobClient: map 0% reduce 0% 2013-05-21 00:49:06,478 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_0, Status : FAILED 2013-05-21 00:49:16,703 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_1, Status : FAILED 2013-05-21 00:49:25,861 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_2, Status : FAILED 2013-05-21 00:49:42,012 INFO org.apache.hadoop.mapred.JobClient: Job complete: job_201305131705_0182 2013-05-21 00:49:42,015 INFO org.apache.hadoop.mapred.JobClient: Counters: 7 2013-05-21 00:49:42,020 INFO org.apache.hadoop.mapred.JobClient: Job Counters 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient: Failed map tasks=1 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient: Launched map tasks=4 2013-05-21 00:49:42,023 INFO
org.apache.hadoop.mapred.JobClient: Data-local map tasks=4 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all maps in occupied slots (ms)=50187 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all reduces in occupied slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 2013-05-21 00:49:42,032 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2013-05-21 00:49:42,033 INFO org.apache.sqoop.mapreduce.ExportJobBase: Transferred 0 bytes in 53.4336 seconds (0 bytes/sec) 2013-05-21 00:49:42,035 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated.
Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:49:42,035 INFO org.apache.sqoop.mapreduce.ExportJobBase: Exported 0 records. 2013-05-21 00:49:42,035 ERROR org.apache.sqoop.tool.ExportTool: Error during export: Export job failed! 2013-05-21 00:49:42,063 INFO org.apache.hadoop.mapred.Task: Task:attempt_201305131705_0181_m_000000_0 is done. And is in the process of commiting 2013-05-21 00:49:43,124 INFO org.apache.hadoop.mapred.Task: Task attempt_201305131705_0181_m_000000_0 is allowed to commit now 2013-05-21 00:49:43,292 INFO org.apache.hadoop.mapred.FileOutputCommitter: Saved output of task 'attempt_201305131705_0181_m_000000_0' to hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/output 2013-05-21 00:49:43,300 INFO org.apache.hadoop.mapred.Task: Task 'attempt_201305131705_0181_m_000000_0' done. 2013-05-21 00:49:43,309 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
Regards,
Andy Tey
Student !!
Re: SQOOP export from Hadoop Hive to MySQL
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Andy,
I'm glad to hear that Sqoop has started working for you!
There is great article on Cloudera blog [1] that describes what sort of logs is Hadoop generating and how to get them. The task logs are describe there as well.
Jarcec
Links:
1: http://blog.cloudera.com/blog/2009/09/apache-hadoop-log-files-where-to-find-them-in-cdh-and-what-info-they-contain/
On Thu, May 23, 2013 at 06:08:47AM -0700, Han Sen Tey wrote:
> Good morning Jarcec,
>
> Thank you very much for your insight. I would have never attempted to remove all single quote from my sqoop command. Upon removing all quotes from my sqoop command, my Oozie workflow that comprised of a pig-, hive- and sqoop- actions flow like a river.
>
> export --verbose --connect jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table CHRYSLER_Anametrix_Aggregated --update-key VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession --update-mode allowinsert --export-dir /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1 --input-fields-terminated-by | --input-null-string \\N --input-null-non-string \\N
>
>
> One question to help with my future debugging. Where or how can I find the map task log in CDH4 ?
>
> Thanks,
> Andy
>
>
>
>
>
> ________________________________
> From: Jarek Jarcec Cecho <ja...@apache.org>
> To: user@sqoop.apache.org; Han Sen Tey <ax...@yahoo.com>
> Sent: Thursday, May 23, 2013 4:45:55 AM
> Subject: Re: SQOOP export from Hadoop Hive to MySQL
>
>
> Hi Han,
> would you mind sharing entire Sqoop log generated with parameter --verbose as an attachment? It might be also beneficial to share the map task log.
>
> Oozie is not using bash nor any other shell to execute Sqoop, so you should remove any escaping introduced especially for shell. Looking at your example, I think that it should be something like:
>
> ... --innput-fields-terminated-by | --input-null-string \\N ...
>
> Jarcec
>
> On Tue, May 21, 2013 at 05:56:16AM -0700, Han Sen Tey wrote:
> > Greeting experts,
> >
> > I have a table "CHRYSLER_Anametrix_Aggregated" in Hive and in MySQL.
> > Similarly I can find this table in hdfs --
> > /user/hive/warehouse/chrysler_anametrix_aggregated.
> >
> >
> > I can successfully execute the following SQOOP command from namenode CLI -
> >
> > sqoop export --verbose --connect
> > jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
> > CHRYSLER_Anametrix_Aggregated --update-key
> > VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
> > --update-mode allowinsert --export-dir
> > /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
> > --input-fields-terminated-by '|' --input-null-string '\\N'
> > --input-null-non-string '\\N'
> >
> > However, when I want to run the above as part of Oozie workflow action ...
> >
> >
> > <action name="sqoop-node">
> > <sqoop xmlns="uri:oozie:sqoop-action:0.2">
> > <job-tracker>${jobTracker}</job-tracker>
> > <name-node>${nameNode}</name-node>
> >
> > <configuration>
> > <property>
> > <name>sqoop.connection.factories</name>
> > <value>com.cloudera.sqoop.manager.DefaultManagerFactory</value>
> > </property>
> > <property>
> >
> > <name>mapred.job.queue.name</name>
> > <value>${queueName}</value>
> > </property>
> > <property>
> > <name>mapred.compress.map.output</name>
> > <value>true</value>
> > </property>
> >
> > <property>
> > <name>oozie.service.WorkflowAppService.system.libpath</name>
> > <value>${nameNode}/user/oozie/share/lib/sqoop</value>
> > </property>
> > </configuration>
> >
> > <command>export --verbose --connect
> > jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
> > CHRYSLER_Anametrix_Aggregated --update-key
> > VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
> > --update-mode allowinsert --export-dir
> > /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
> > --input-fields-terminated-by '|' --input-null-string '\\N'
> > --input-null-non-string '\\N'</command>
> > </sqoop>
> > <ok to="end"/>
> > <error to="fail"/>
> > </action>
> >
> > I get the following error message. I have "mysql-connector-java.5.1.24-bin.jar" shared library - /user/oozie/share/lib/sqoop.
> >
> > Hadoop version - 2.00 - CDH4.2.0
> > Sqoop version - 1.4.2. - CDH4.2.0
> >
> >
> > Can you experts please guide me ... a rookie to Hadoop and its projects.
> >
> > 2013-05-21 00:48:35,135 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:48:37,394 WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id 2013-05-21 00:48:37,408 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2013-05-21 00:48:38,042 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2013-05-21 00:48:38,058 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5dce1bea 2013-05-21 00:48:38,423 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/input/dummy.txt:0+5 2013-05-21 00:48:38,453 WARN mapreduce.Counters: Counter name
> MAP_INPUT_BYTES
> > is deprecated. Use FileInputFormatCounters as group name and BYTES_READ as counter name instead 2013-05-21 00:48:38,459 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0 2013-05-21 00:48:39,743 WARN org.apache.sqoop.tool.SqoopTool: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. 2013-05-21 00:48:40,032 DEBUG org.apache.sqoop.tool.BaseSqoopTool: Enabled debug logging. 2013-05-21 00:48:40,036 WARN org.apache.sqoop.SqoopOptions: Character argument '|' has multiple characters; only the first will be used. 2013-05-21 00:48:40,084 DEBUG org.apache.sqoop.ConnFactory: sqoop.connection.factories is set; ignoring managers.d 2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21
> 00:48:40,140
> > DEBUG org.apache.sqoop.manager.DefaultManagerFactory: Trying with scheme: jdbc:mysql: 2013-05-21 00:48:40,183 INFO org.apache.sqoop.manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2013-05-21 00:48:40,183 DEBUG org.apache.sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.MySQLManager@795e0c2b 2013-05-21 00:48:40,184 INFO org.apache.sqoop.tool.CodeGenTool: Beginning code generation 2013-05-21 00:48:40,233 DEBUG org.apache.sqoop.manager.SqlManager: No connection paramenters specified. Using regular API for making connection. 2013-05-21 00:48:40,911 DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21 00:48:40,911 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,930 DEBUG org.apache.sqoop.orm.ClassWriter: selected columns: 2013-05-21 00:48:40,931 DEBUG
> > org.apache.sqoop.orm.ClassWriter: VisitorID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorDay 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorMonth 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorYear 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorSession 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: Brand 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceType 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceBrowser 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: TrafficSource 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: SiteID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: BuyID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: AdvertisementID 2013-05-21 00:48:40,931 DEBUG
> org.apache.sqoop.orm.ClassWriter:
> > PlacementID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: CreativeID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: PaidKeyword 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model200 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model200Conv 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model300 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model300TownCountry 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: BMOLineup 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOModels 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOColors 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOOptions 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOSummary 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureLineup
> 2013-05-21
> > 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureForm 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureMailLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureDownloadLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: PreDL 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DealerLocate 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLHours 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLServices 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: DLDealerSite 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: PreSNI 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SearchNewInventory 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SNIVIN 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SNIDetails 2013-05-21 00:48:40,936
> DEBUG
> > org.apache.sqoop.orm.ClassWriter: SNIDealer 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: TestDriveForm 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: TestDriveComplete 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: Inventives 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreCal 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PaymentCalculator 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreGAQ 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: GetAQuote 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: LandingPage 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Models 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PhotosGallery 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Interior 2013-05-21 00:48:40,937 DEBUG
> org.apache.sqoop.orm.ClassWriter:
> > Exterior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Performance 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Safety 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Specifications 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: ModelHighlights 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Dimensions 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Warranty 2013-05-21 00:48:40,965 DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21 00:48:40,965 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Writing source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,990 DEBUG
> > org.apache.sqoop.orm.ClassWriter: Table name: CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Columns: VisitorID:12, VisitorDay:12, VisitorMonth:12, VisitorYear:12, VisitorSession:4, Brand:12, DeviceType:12, DeviceBrowser:12, TrafficSource:12, SiteID:12, BuyID:12, AdvertisementID:12, PlacementID:12, CreativeID:12, PaidKeyword:12, Model200:4, Model200Conv:4, Model300:4, Model300TownCountry:4, BMOLineup:4, BMOModels:4, BMOColors:4, BMOOptions:4, BMOSummary:4, BrochureLineup:4, BrochureForm:4, BrochureMailLink:4, BrochureDownloadLink:4, PreDL:4, DealerLocate:4, DLHours:4, DLServices:4, DLDealerSite:4, PreSNI:4, SearchNewInventory:4, SNIVIN:4, SNIDetails:4, SNIDealer:4, TestDriveForm:4, TestDriveComplete:4, Inventives:4, PreCal:4, PaymentCalculator:4, PreGAQ:4, GetAQuote:4, LandingPage:4, Models:4, PhotosGallery:4, Interior:4, Exterior:4, Performance:4, Safety:4, Specifications:4, ModelHighlights:4,
> > Dimensions:4, Warranty:4, 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: sourceFilename is CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,991 DEBUG org.apache.sqoop.orm.CompilationManager: Found existing /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:40,992 INFO org.apache.sqoop.orm.CompilationManager: $HADOOP_MAPRED_HOME is not set 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Adding source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Invoking javac with args: 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: -sourcepath 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:
> -d
> > 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager: -classpath 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:
> >
> /run/cloudera-scm-agent/process/76-mapreduce-TASKTRACKER:/usr/lib/jvm/j2sdk1.6-oracle/lib/tools.jar:/usr/lib/hadoop-0.20-mapreduce:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-
> > 0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/j
> > asper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jline-0.9.94.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-map
> > reduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar:/usr/share/cmf/lib/plugins/navigator-plugin-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/event-publish-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.5.1.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:
> > /usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/
> > usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/avro-1.7.3.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-c
> > onfiguration-1.6.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hado
> > op/lib/hue-plugins-2.2.0-cdh4.2.0.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common.jar:/mnt/mapred/local/taskTracker/distcache/-3359360379488511723_1277082233_1150217957/ip-10-147-191-48.ec2.internal/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/sqoop-launcher.jar:/mnt/mapred/local/taskTracker/distcache/-3926870669093555941_811939984_1868217431/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoo
> > p/avro-ipc-1.7.3-tests.jar:/mnt/mapred/local/taskTracker/distcache/4298483340499219542_2138859044_1868217453/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-ipc-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/-1975491928978283532_-1185784129_1868217475/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-mapred-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/8448357694523829347_-1491317642_1868217500/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-1.7.0.jar:/mnt/mapred/local/taskTracker/distcache/-334079317866666788_-232552041_1868217523/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-core-1.8.0.jar:/mnt/mapred/local/taskTracker/distcache/-644448586073609072_-372575484_1868217547/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-configuration-1.6.jar:/mnt/mapred/local/taskTracker/distcache/246054495199880993_357051411_1868217589/ip-10-147-191-48.ec2.internal/
> > user/oozie/share/lib/sqoop/commons-digester-1.8.jar:/mnt/mapred/local/taskTracker/distcache/-4144140590487253933_766503300_1868217614/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-io-2.1.jar:/mnt/mapred/local/taskTracker/distcache/5753546228468760323_-553874526_1868217665/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/guava-11.0.2.jar:/mnt/mapred/local/taskTracker/distcache/8662866019753074519_-1003108238_1868217924/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hbase-0.94.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4840377369495417802_845918870_1868217954/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/high-scale-lib-1.1.1.jar:/mnt/mapred/local/taskTracker/distcache/-142623586016294918_445946554_1868217996/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hsqldb-1.8.0.7.jar:/mnt/mapred/local/taskTracker/distcache/5142583573507929188_-105788043_1868218025/ip-10-147-191-48.ec2.internal/us
> > er/oozie/share/lib/sqoop/httpclient-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/-6413369533723682352_-180904991_1868218051/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/httpcore-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/3446550291828223401_633399508_1868218088/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/jsr305-1.3.9.jar:/mnt/mapred/local/taskTracker/distcache/-6492962697058022370_-616322509_1868218133/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/libthrift-0.9.0.jar:/mnt/mapred/local/taskTracker/distcache/7049151581973813525_2017354882_1868218156/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/metrics-core-2.1.2.jar:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar:/mnt/mapred/local/taskTracker/distcache/-7982675208196719520_-1207761111_1868218197/ip-10-147-191-48.ec2.interna
> > l/user/oozie/share/lib/sqoop/netty-3.4.0.Final.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-412267918101123238_-517736257_1868215515/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/oozie/json-simple-1.1.jar:/mnt/mapred/local/taskTracker/ubuntu/jobcache/job_201305131705_0181/attempt_201305131705_0181_m_000000_0/work:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,206 INFO org.apache.sqoop.orm.CompilationManager: Writing jar file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar 2013-05-21 00:48:48,210 DEBUG
> > org.apache.sqoop.orm.CompilationManager: Scanning for .class files in directory: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca 2013-05-21 00:48:48,221 DEBUG org.apache.sqoop.orm.CompilationManager: Got classfile: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.class -> CHRYSLER_Anametrix_Aggregated.class 2013-05-21 00:48:48,236 DEBUG org.apache.sqoop.orm.CompilationManager: Finished writing jar file /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: MySQL Connector upsert functionality is using INSERT ON 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: DUPLICATE KEY UPDATE clause that relies on table's unique key. 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: Insert/update distinction is therefore independent on column 2013-05-21 00:48:48,243 WARN
> > org.apache.sqoop.manager.MySQLManager: names specified in --update-key parameter. Please see MySQL 2013-05-21 00:48:48,245 WARN org.apache.sqoop.manager.MySQLManager: documentation for additional limitations. 2013-05-21 00:48:48,290 INFO org.apache.sqoop.mapreduce.ExportJobBase: Beginning export of CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:48,319 DEBUG org.apache.sqoop.mapreduce.JobBase: Using InputFormat: class org.apache.sqoop.mapreduce.ExportInputFormat 2013-05-21 00:48:48,575 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,581 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
> > file:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar 2013-05-21 00:48:48,585 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,588 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,588 WARN org.apache.sqoop.mapreduce.JobBase: SQOOP_HOME is unset. May not be able to find all job dependencies. 2013-05-21 00:48:48,739 WARN org.apache.hadoop.mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications
> > should implement Tool for the same. 2013-05-21 00:48:49,058 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Target numMapTasks=1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Total input bytes=8187428 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: maxSplitSize=8187428 2013-05-21 00:48:49,063 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Generated splits: 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Paths:/user/hive/warehouse/chrysler_anametrix_aggregated/part-AG-00000-1.txt:0+8187428 Locations:ip-10-147-191-48.ec2.internal:; 2013-05-21 00:48:49,403 INFO org.apache.hadoop.mapred.JobClient: Running job:
> job_201305131705_0182
> > 2013-05-21 00:48:50,417 INFO org.apache.hadoop.mapred.JobClient: map 0% reduce 0% 2013-05-21 00:49:06,478 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_0, Status : FAILED 2013-05-21 00:49:16,703 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_1, Status : FAILED 2013-05-21 00:49:25,861 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_2, Status : FAILED 2013-05-21 00:49:42,012 INFO org.apache.hadoop.mapred.JobClient: Job complete: job_201305131705_0182 2013-05-21 00:49:42,015 INFO org.apache.hadoop.mapred.JobClient: Counters: 7 2013-05-21 00:49:42,020 INFO org.apache.hadoop.mapred.JobClient: Job Counters 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient: Failed map tasks=1 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient: Launched map tasks=4 2013-05-21 00:49:42,023 INFO
> > org.apache.hadoop.mapred.JobClient: Data-local map tasks=4 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all maps in occupied slots (ms)=50187 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all reduces in occupied slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 2013-05-21 00:49:42,032 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2013-05-21 00:49:42,033 INFO org.apache.sqoop.mapreduce.ExportJobBase: Transferred 0 bytes in 53.4336 seconds (0 bytes/sec) 2013-05-21 00:49:42,035 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
> is deprecated.
> > Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:49:42,035 INFO org.apache.sqoop.mapreduce.ExportJobBase: Exported 0 records. 2013-05-21 00:49:42,035 ERROR org.apache.sqoop.tool.ExportTool: Error during export: Export job failed! 2013-05-21 00:49:42,063 INFO org.apache.hadoop.mapred.Task: Task:attempt_201305131705_0181_m_000000_0 is done. And is in the process of commiting 2013-05-21 00:49:43,124 INFO org.apache.hadoop.mapred.Task: Task attempt_201305131705_0181_m_000000_0 is allowed to commit now 2013-05-21 00:49:43,292 INFO org.apache.hadoop.mapred.FileOutputCommitter: Saved output of task 'attempt_201305131705_0181_m_000000_0' to hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/output 2013-05-21 00:49:43,300 INFO org.apache.hadoop.mapred.Task: Task 'attempt_201305131705_0181_m_000000_0' done. 2013-05-21 00:49:43,309 INFO
> org.apache.hadoop.mapred.TaskLogsTruncater:
> > Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
> >
> > Regards,
> > Andy Tey
> > Student !!
Re: SQOOP export from Hadoop Hive to MySQL
Posted by Han Sen Tey <ax...@yahoo.com>.
Good morning Jarcec,
Thank you very much for your insight. I would have never attempted to remove all single quote from my sqoop command. Upon removing all quotes from my sqoop command, my Oozie workflow that comprised of a pig-, hive- and sqoop- actions flow like a river.
export --verbose --connect jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table CHRYSLER_Anametrix_Aggregated --update-key VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession --update-mode allowinsert --export-dir /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1 --input-fields-terminated-by | --input-null-string \\N --input-null-non-string \\N
One question to help with my future debugging. Where or how can I find the map task log in CDH4 ?
Thanks,
Andy
________________________________
From: Jarek Jarcec Cecho <ja...@apache.org>
To: user@sqoop.apache.org; Han Sen Tey <ax...@yahoo.com>
Sent: Thursday, May 23, 2013 4:45:55 AM
Subject: Re: SQOOP export from Hadoop Hive to MySQL
Hi Han,
would you mind sharing entire Sqoop log generated with parameter --verbose as an attachment? It might be also beneficial to share the map task log.
Oozie is not using bash nor any other shell to execute Sqoop, so you should remove any escaping introduced especially for shell. Looking at your example, I think that it should be something like:
... --innput-fields-terminated-by | --input-null-string \\N ...
Jarcec
On Tue, May 21, 2013 at 05:56:16AM -0700, Han Sen Tey wrote:
> Greeting experts,
>
> I have a table "CHRYSLER_Anametrix_Aggregated" in Hive and in MySQL.
> Similarly I can find this table in hdfs --
> /user/hive/warehouse/chrysler_anametrix_aggregated.
>
>
> I can successfully execute the following SQOOP command from namenode CLI -
>
> sqoop export --verbose --connect
> jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
> CHRYSLER_Anametrix_Aggregated --update-key
> VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
> --update-mode allowinsert --export-dir
> /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
> --input-fields-terminated-by '|' --input-null-string '\\N'
> --input-null-non-string '\\N'
>
> However, when I want to run the above as part of Oozie workflow action ...
>
>
> <action name="sqoop-node">
> <sqoop xmlns="uri:oozie:sqoop-action:0.2">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
>
> <configuration>
> <property>
> <name>sqoop.connection.factories</name>
> <value>com.cloudera.sqoop.manager.DefaultManagerFactory</value>
> </property>
> <property>
>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> <property>
> <name>mapred.compress.map.output</name>
> <value>true</value>
> </property>
>
> <property>
> <name>oozie.service.WorkflowAppService.system.libpath</name>
> <value>${nameNode}/user/oozie/share/lib/sqoop</value>
> </property>
> </configuration>
>
> <command>export --verbose --connect
> jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
> CHRYSLER_Anametrix_Aggregated --update-key
> VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
> --update-mode allowinsert --export-dir
> /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
> --input-fields-terminated-by '|' --input-null-string '\\N'
> --input-null-non-string '\\N'</command>
> </sqoop>
> <ok to="end"/>
> <error to="fail"/>
> </action>
>
> I get the following error message. I have "mysql-connector-java.5.1.24-bin.jar" shared library - /user/oozie/share/lib/sqoop.
>
> Hadoop version - 2.00 - CDH4.2.0
> Sqoop version - 1.4.2. - CDH4.2.0
>
>
> Can you experts please guide me ... a rookie to Hadoop and its projects.
>
> 2013-05-21 00:48:35,135 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:48:37,394 WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id 2013-05-21 00:48:37,408 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2013-05-21 00:48:38,042 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2013-05-21 00:48:38,058 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5dce1bea 2013-05-21 00:48:38,423 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/input/dummy.txt:0+5 2013-05-21 00:48:38,453 WARN mapreduce.Counters: Counter name
MAP_INPUT_BYTES
> is deprecated. Use FileInputFormatCounters as group name and BYTES_READ as counter name instead 2013-05-21 00:48:38,459 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0 2013-05-21 00:48:39,743 WARN org.apache.sqoop.tool.SqoopTool: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. 2013-05-21 00:48:40,032 DEBUG org.apache.sqoop.tool.BaseSqoopTool: Enabled debug logging. 2013-05-21 00:48:40,036 WARN org.apache.sqoop.SqoopOptions: Character argument '|' has multiple characters; only the first will be used. 2013-05-21 00:48:40,084 DEBUG org.apache.sqoop.ConnFactory: sqoop.connection.factories is set; ignoring managers.d 2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21
00:48:40,140
> DEBUG org.apache.sqoop.manager.DefaultManagerFactory: Trying with scheme: jdbc:mysql: 2013-05-21 00:48:40,183 INFO org.apache.sqoop.manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2013-05-21 00:48:40,183 DEBUG org.apache.sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.MySQLManager@795e0c2b 2013-05-21 00:48:40,184 INFO org.apache.sqoop.tool.CodeGenTool: Beginning code generation 2013-05-21 00:48:40,233 DEBUG org.apache.sqoop.manager.SqlManager: No connection paramenters specified. Using regular API for making connection. 2013-05-21 00:48:40,911 DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21 00:48:40,911 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,930 DEBUG org.apache.sqoop.orm.ClassWriter: selected columns: 2013-05-21 00:48:40,931 DEBUG
> org.apache.sqoop.orm.ClassWriter: VisitorID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorDay 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorMonth 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorYear 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorSession 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: Brand 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceType 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceBrowser 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: TrafficSource 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: SiteID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: BuyID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: AdvertisementID 2013-05-21 00:48:40,931 DEBUG
org.apache.sqoop.orm.ClassWriter:
> PlacementID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: CreativeID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: PaidKeyword 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model200 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model200Conv 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model300 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model300TownCountry 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: BMOLineup 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOModels 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOColors 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOOptions 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOSummary 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureLineup
2013-05-21
> 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureForm 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureMailLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureDownloadLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: PreDL 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DealerLocate 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLHours 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLServices 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: DLDealerSite 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: PreSNI 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SearchNewInventory 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SNIVIN 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SNIDetails 2013-05-21 00:48:40,936
DEBUG
> org.apache.sqoop.orm.ClassWriter: SNIDealer 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: TestDriveForm 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: TestDriveComplete 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: Inventives 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreCal 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PaymentCalculator 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreGAQ 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: GetAQuote 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: LandingPage 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Models 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PhotosGallery 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Interior 2013-05-21 00:48:40,937 DEBUG
org.apache.sqoop.orm.ClassWriter:
> Exterior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Performance 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Safety 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Specifications 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: ModelHighlights 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Dimensions 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Warranty 2013-05-21 00:48:40,965 DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21 00:48:40,965 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Writing source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,990 DEBUG
> org.apache.sqoop.orm.ClassWriter: Table name: CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Columns: VisitorID:12, VisitorDay:12, VisitorMonth:12, VisitorYear:12, VisitorSession:4, Brand:12, DeviceType:12, DeviceBrowser:12, TrafficSource:12, SiteID:12, BuyID:12, AdvertisementID:12, PlacementID:12, CreativeID:12, PaidKeyword:12, Model200:4, Model200Conv:4, Model300:4, Model300TownCountry:4, BMOLineup:4, BMOModels:4, BMOColors:4, BMOOptions:4, BMOSummary:4, BrochureLineup:4, BrochureForm:4, BrochureMailLink:4, BrochureDownloadLink:4, PreDL:4, DealerLocate:4, DLHours:4, DLServices:4, DLDealerSite:4, PreSNI:4, SearchNewInventory:4, SNIVIN:4, SNIDetails:4, SNIDealer:4, TestDriveForm:4, TestDriveComplete:4, Inventives:4, PreCal:4, PaymentCalculator:4, PreGAQ:4, GetAQuote:4, LandingPage:4, Models:4, PhotosGallery:4, Interior:4, Exterior:4, Performance:4, Safety:4, Specifications:4, ModelHighlights:4,
> Dimensions:4, Warranty:4, 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: sourceFilename is CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,991 DEBUG org.apache.sqoop.orm.CompilationManager: Found existing /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:40,992 INFO org.apache.sqoop.orm.CompilationManager: $HADOOP_MAPRED_HOME is not set 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Adding source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Invoking javac with args: 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: -sourcepath 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager:
-d
> 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager: -classpath 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:
>
/run/cloudera-scm-agent/process/76-mapreduce-TASKTRACKER:/usr/lib/jvm/j2sdk1.6-oracle/lib/tools.jar:/usr/lib/hadoop-0.20-mapreduce:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-
> 0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/j
> asper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jline-0.9.94.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-map
> reduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar:/usr/share/cmf/lib/plugins/navigator-plugin-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/event-publish-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.5.1.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:
> /usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/
> usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/avro-1.7.3.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-c
> onfiguration-1.6.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hado
> op/lib/hue-plugins-2.2.0-cdh4.2.0.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common.jar:/mnt/mapred/local/taskTracker/distcache/-3359360379488511723_1277082233_1150217957/ip-10-147-191-48.ec2.internal/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/sqoop-launcher.jar:/mnt/mapred/local/taskTracker/distcache/-3926870669093555941_811939984_1868217431/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoo
> p/avro-ipc-1.7.3-tests.jar:/mnt/mapred/local/taskTracker/distcache/4298483340499219542_2138859044_1868217453/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-ipc-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/-1975491928978283532_-1185784129_1868217475/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-mapred-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/8448357694523829347_-1491317642_1868217500/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-1.7.0.jar:/mnt/mapred/local/taskTracker/distcache/-334079317866666788_-232552041_1868217523/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-core-1.8.0.jar:/mnt/mapred/local/taskTracker/distcache/-644448586073609072_-372575484_1868217547/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-configuration-1.6.jar:/mnt/mapred/local/taskTracker/distcache/246054495199880993_357051411_1868217589/ip-10-147-191-48.ec2.internal/
> user/oozie/share/lib/sqoop/commons-digester-1.8.jar:/mnt/mapred/local/taskTracker/distcache/-4144140590487253933_766503300_1868217614/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-io-2.1.jar:/mnt/mapred/local/taskTracker/distcache/5753546228468760323_-553874526_1868217665/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/guava-11.0.2.jar:/mnt/mapred/local/taskTracker/distcache/8662866019753074519_-1003108238_1868217924/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hbase-0.94.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4840377369495417802_845918870_1868217954/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/high-scale-lib-1.1.1.jar:/mnt/mapred/local/taskTracker/distcache/-142623586016294918_445946554_1868217996/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hsqldb-1.8.0.7.jar:/mnt/mapred/local/taskTracker/distcache/5142583573507929188_-105788043_1868218025/ip-10-147-191-48.ec2.internal/us
> er/oozie/share/lib/sqoop/httpclient-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/-6413369533723682352_-180904991_1868218051/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/httpcore-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/3446550291828223401_633399508_1868218088/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/jsr305-1.3.9.jar:/mnt/mapred/local/taskTracker/distcache/-6492962697058022370_-616322509_1868218133/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/libthrift-0.9.0.jar:/mnt/mapred/local/taskTracker/distcache/7049151581973813525_2017354882_1868218156/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/metrics-core-2.1.2.jar:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar:/mnt/mapred/local/taskTracker/distcache/-7982675208196719520_-1207761111_1868218197/ip-10-147-191-48.ec2.interna
> l/user/oozie/share/lib/sqoop/netty-3.4.0.Final.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-412267918101123238_-517736257_1868215515/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/oozie/json-simple-1.1.jar:/mnt/mapred/local/taskTracker/ubuntu/jobcache/job_201305131705_0181/attempt_201305131705_0181_m_000000_0/work:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,206 INFO org.apache.sqoop.orm.CompilationManager: Writing jar file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar 2013-05-21 00:48:48,210 DEBUG
> org.apache.sqoop.orm.CompilationManager: Scanning for .class files in directory: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca 2013-05-21 00:48:48,221 DEBUG org.apache.sqoop.orm.CompilationManager: Got classfile: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.class -> CHRYSLER_Anametrix_Aggregated.class 2013-05-21 00:48:48,236 DEBUG org.apache.sqoop.orm.CompilationManager: Finished writing jar file /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: MySQL Connector upsert functionality is using INSERT ON 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: DUPLICATE KEY UPDATE clause that relies on table's unique key. 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: Insert/update distinction is therefore independent on column 2013-05-21 00:48:48,243 WARN
> org.apache.sqoop.manager.MySQLManager: names specified in --update-key parameter. Please see MySQL 2013-05-21 00:48:48,245 WARN org.apache.sqoop.manager.MySQLManager: documentation for additional limitations. 2013-05-21 00:48:48,290 INFO org.apache.sqoop.mapreduce.ExportJobBase: Beginning export of CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:48,319 DEBUG org.apache.sqoop.mapreduce.JobBase: Using InputFormat: class org.apache.sqoop.mapreduce.ExportInputFormat 2013-05-21 00:48:48,575 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,581 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
> file:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar 2013-05-21 00:48:48,585 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,588 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,588 WARN org.apache.sqoop.mapreduce.JobBase: SQOOP_HOME is unset. May not be able to find all job dependencies. 2013-05-21 00:48:48,739 WARN org.apache.hadoop.mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications
> should implement Tool for the same. 2013-05-21 00:48:49,058 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Target numMapTasks=1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Total input bytes=8187428 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: maxSplitSize=8187428 2013-05-21 00:48:49,063 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Generated splits: 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Paths:/user/hive/warehouse/chrysler_anametrix_aggregated/part-AG-00000-1.txt:0+8187428 Locations:ip-10-147-191-48.ec2.internal:; 2013-05-21 00:48:49,403 INFO org.apache.hadoop.mapred.JobClient: Running job:
job_201305131705_0182
> 2013-05-21 00:48:50,417 INFO org.apache.hadoop.mapred.JobClient: map 0% reduce 0% 2013-05-21 00:49:06,478 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_0, Status : FAILED 2013-05-21 00:49:16,703 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_1, Status : FAILED 2013-05-21 00:49:25,861 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_2, Status : FAILED 2013-05-21 00:49:42,012 INFO org.apache.hadoop.mapred.JobClient: Job complete: job_201305131705_0182 2013-05-21 00:49:42,015 INFO org.apache.hadoop.mapred.JobClient: Counters: 7 2013-05-21 00:49:42,020 INFO org.apache.hadoop.mapred.JobClient: Job Counters 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient: Failed map tasks=1 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient: Launched map tasks=4 2013-05-21 00:49:42,023 INFO
> org.apache.hadoop.mapred.JobClient: Data-local map tasks=4 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all maps in occupied slots (ms)=50187 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all reduces in occupied slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 2013-05-21 00:49:42,032 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2013-05-21 00:49:42,033 INFO org.apache.sqoop.mapreduce.ExportJobBase: Transferred 0 bytes in 53.4336 seconds (0 bytes/sec) 2013-05-21 00:49:42,035 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter
is deprecated.
> Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:49:42,035 INFO org.apache.sqoop.mapreduce.ExportJobBase: Exported 0 records. 2013-05-21 00:49:42,035 ERROR org.apache.sqoop.tool.ExportTool: Error during export: Export job failed! 2013-05-21 00:49:42,063 INFO org.apache.hadoop.mapred.Task: Task:attempt_201305131705_0181_m_000000_0 is done. And is in the process of commiting 2013-05-21 00:49:43,124 INFO org.apache.hadoop.mapred.Task: Task attempt_201305131705_0181_m_000000_0 is allowed to commit now 2013-05-21 00:49:43,292 INFO org.apache.hadoop.mapred.FileOutputCommitter: Saved output of task 'attempt_201305131705_0181_m_000000_0' to hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/output 2013-05-21 00:49:43,300 INFO org.apache.hadoop.mapred.Task: Task 'attempt_201305131705_0181_m_000000_0' done. 2013-05-21 00:49:43,309 INFO
org.apache.hadoop.mapred.TaskLogsTruncater:
> Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
>
> Regards,
> Andy Tey
> Student !!
Re: SQOOP export from Hadoop Hive to MySQL
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Han,
would you mind sharing entire Sqoop log generated with parameter --verbose as an attachment? It might be also beneficial to share the map task log.
Oozie is not using bash nor any other shell to execute Sqoop, so you should remove any escaping introduced especially for shell. Looking at your example, I think that it should be something like:
... --innput-fields-terminated-by | --input-null-string \\N ...
Jarcec
On Tue, May 21, 2013 at 05:56:16AM -0700, Han Sen Tey wrote:
> Greeting experts,
>
> I have a table "CHRYSLER_Anametrix_Aggregated" in Hive and in MySQL.
> Similarly I can find this table in hdfs --
> /user/hive/warehouse/chrysler_anametrix_aggregated.
>
>
> I can successfully execute the following SQOOP command from namenode CLI -
>
> sqoop export --verbose --connect
> jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
> CHRYSLER_Anametrix_Aggregated --update-key
> VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
> --update-mode allowinsert --export-dir
> /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
> --input-fields-terminated-by '|' --input-null-string '\\N'
> --input-null-non-string '\\N'
>
> However, when I want to run the above as part of Oozie workflow action ...
>
>
> <action name="sqoop-node">
> <sqoop xmlns="uri:oozie:sqoop-action:0.2">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
>
> <configuration>
> <property>
> <name>sqoop.connection.factories</name>
> <value>com.cloudera.sqoop.manager.DefaultManagerFactory</value>
> </property>
> <property>
>
> <name>mapred.job.queue.name</name>
> <value>${queueName}</value>
> </property>
> <property>
> <name>mapred.compress.map.output</name>
> <value>true</value>
> </property>
>
> <property>
> <name>oozie.service.WorkflowAppService.system.libpath</name>
> <value>${nameNode}/user/oozie/share/lib/sqoop</value>
> </property>
> </configuration>
>
> <command>export --verbose --connect
> jdbc:mysql://ec2-23-22-223-47.compute-1.amazonaws.com:3306/test --table
> CHRYSLER_Anametrix_Aggregated --update-key
> VisitorID,VisitorDay,VisitorMonth,VisitorYear,VisitorSession
> --update-mode allowinsert --export-dir
> /user/hive/warehouse/chrysler_anametrix_aggregated --username root -m 1
> --input-fields-terminated-by '|' --input-null-string '\\N'
> --input-null-non-string '\\N'</command>
> </sqoop>
> <ok to="end"/>
> <error to="fail"/>
> </action>
>
> I get the following error message. I have "mysql-connector-java.5.1.24-bin.jar" shared library - /user/oozie/share/lib/sqoop.
>
> Hadoop version - 2.00 - CDH4.2.0
> Sqoop version - 1.4.2. - CDH4.2.0
>
>
> Can you experts please guide me ... a rookie to Hadoop and its projects.
>
> 2013-05-21 00:48:35,135 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:48:37,394 WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id 2013-05-21 00:48:37,408 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2013-05-21 00:48:38,042 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2013-05-21 00:48:38,058 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5dce1bea 2013-05-21 00:48:38,423 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/input/dummy.txt:0+5 2013-05-21 00:48:38,453 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES
> is deprecated. Use FileInputFormatCounters as group name and BYTES_READ as counter name instead 2013-05-21 00:48:38,459 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0 2013-05-21 00:48:39,743 WARN org.apache.sqoop.tool.SqoopTool: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. 2013-05-21 00:48:40,032 DEBUG org.apache.sqoop.tool.BaseSqoopTool: Enabled debug logging. 2013-05-21 00:48:40,036 WARN org.apache.sqoop.SqoopOptions: Character argument '|' has multiple characters; only the first will be used. 2013-05-21 00:48:40,084 DEBUG org.apache.sqoop.ConnFactory: sqoop.connection.factories is set; ignoring managers.d 2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21 00:48:40,139 DEBUG org.apache.sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory 2013-05-21 00:48:40,140
> DEBUG org.apache.sqoop.manager.DefaultManagerFactory: Trying with scheme: jdbc:mysql: 2013-05-21 00:48:40,183 INFO org.apache.sqoop.manager.MySQLManager: Preparing to use a MySQL streaming resultset. 2013-05-21 00:48:40,183 DEBUG org.apache.sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.MySQLManager@795e0c2b 2013-05-21 00:48:40,184 INFO org.apache.sqoop.tool.CodeGenTool: Beginning code generation 2013-05-21 00:48:40,233 DEBUG org.apache.sqoop.manager.SqlManager: No connection paramenters specified. Using regular API for making connection. 2013-05-21 00:48:40,911 DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21 00:48:40,911 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,930 DEBUG org.apache.sqoop.orm.ClassWriter: selected columns: 2013-05-21 00:48:40,931 DEBUG
> org.apache.sqoop.orm.ClassWriter: VisitorID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorDay 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorMonth 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorYear 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: VisitorSession 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: Brand 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceType 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: DeviceBrowser 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: TrafficSource 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: SiteID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: BuyID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter: AdvertisementID 2013-05-21 00:48:40,931 DEBUG org.apache.sqoop.orm.ClassWriter:
> PlacementID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: CreativeID 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: PaidKeyword 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model200 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model200Conv 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model300 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: Model300TownCountry 2013-05-21 00:48:40,934 DEBUG org.apache.sqoop.orm.ClassWriter: BMOLineup 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOModels 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOColors 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOOptions 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BMOSummary 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureLineup 2013-05-21
> 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureForm 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureMailLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: BrochureDownloadLink 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: PreDL 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DealerLocate 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLHours 2013-05-21 00:48:40,935 DEBUG org.apache.sqoop.orm.ClassWriter: DLServices 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: DLDealerSite 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: PreSNI 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SearchNewInventory 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SNIVIN 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: SNIDetails 2013-05-21 00:48:40,936 DEBUG
> org.apache.sqoop.orm.ClassWriter: SNIDealer 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: TestDriveForm 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: TestDriveComplete 2013-05-21 00:48:40,936 DEBUG org.apache.sqoop.orm.ClassWriter: Inventives 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreCal 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PaymentCalculator 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PreGAQ 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: GetAQuote 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: LandingPage 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Models 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: PhotosGallery 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Interior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter:
> Exterior 2013-05-21 00:48:40,937 DEBUG org.apache.sqoop.orm.ClassWriter: Performance 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Safety 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Specifications 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: ModelHighlights 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Dimensions 2013-05-21 00:48:40,938 DEBUG org.apache.sqoop.orm.ClassWriter: Warranty 2013-05-21 00:48:40,965 DEBUG org.apache.sqoop.manager.SqlManager: Using fetchSize for next query: -2147483648 2013-05-21 00:48:40,965 INFO org.apache.sqoop.manager.SqlManager: Executing SQL statement: SELECT t.* FROM `CHRYSLER_Anametrix_Aggregated` AS t LIMIT 1 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Writing source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,990 DEBUG
> org.apache.sqoop.orm.ClassWriter: Table name: CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: Columns: VisitorID:12, VisitorDay:12, VisitorMonth:12, VisitorYear:12, VisitorSession:4, Brand:12, DeviceType:12, DeviceBrowser:12, TrafficSource:12, SiteID:12, BuyID:12, AdvertisementID:12, PlacementID:12, CreativeID:12, PaidKeyword:12, Model200:4, Model200Conv:4, Model300:4, Model300TownCountry:4, BMOLineup:4, BMOModels:4, BMOColors:4, BMOOptions:4, BMOSummary:4, BrochureLineup:4, BrochureForm:4, BrochureMailLink:4, BrochureDownloadLink:4, PreDL:4, DealerLocate:4, DLHours:4, DLServices:4, DLDealerSite:4, PreSNI:4, SearchNewInventory:4, SNIVIN:4, SNIDetails:4, SNIDealer:4, TestDriveForm:4, TestDriveComplete:4, Inventives:4, PreCal:4, PaymentCalculator:4, PreGAQ:4, GetAQuote:4, LandingPage:4, Models:4, PhotosGallery:4, Interior:4, Exterior:4, Performance:4, Safety:4, Specifications:4, ModelHighlights:4,
> Dimensions:4, Warranty:4, 2013-05-21 00:48:40,990 DEBUG org.apache.sqoop.orm.ClassWriter: sourceFilename is CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:40,991 DEBUG org.apache.sqoop.orm.CompilationManager: Found existing /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:40,992 INFO org.apache.sqoop.orm.CompilationManager: $HADOOP_MAPRED_HOME is not set 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Adding source file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.java 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: Invoking javac with args: 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: -sourcepath 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:41,098 DEBUG org.apache.sqoop.orm.CompilationManager: -d
> 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/ 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager: -classpath 2013-05-21 00:48:41,099 DEBUG org.apache.sqoop.orm.CompilationManager:
> /run/cloudera-scm-agent/process/76-mapreduce-TASKTRACKER:/usr/lib/jvm/j2sdk1.6-oracle/lib/tools.jar:/usr/lib/hadoop-0.20-mapreduce:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/activation-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/aspectjtools-1.6.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/avro-compiler-1.7.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-codec-1.4.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-configuration-1.6.jar:/usr/lib/hadoop-
> 0.20-mapreduce/lib/commons-digester-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-el-1.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-lang-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/commons-net-3.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr1-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/j
> asper-runtime-5.5.23.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-json-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jettison-1.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jline-0.9.94.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-0.20-mapreduce/lib/junit-4.8.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/usr/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-0.20-map
> reduce/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-0.20-mapreduce/lib/servlet-api-2.5.jar:/usr/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/stax-api-1.0.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/usr/lib/hadoop-0.20-mapreduce/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/usr/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.jar:/usr/share/cmf/lib/plugins/navigator-plugin-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/event-publish-4.5.1-shaded.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.5.1.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:
> /usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs.jar:/
> usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/avro-1.7.3.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-c
> onfiguration-1.6.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/zookeeper-3.4.5-cdh4.2.0.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hado
> op/lib/hue-plugins-2.2.0-cdh4.2.0.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/hadoop-auth.jar:/usr/lib/hadoop/hadoop-auth-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-annotations.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0-tests.jar:/usr/lib/hadoop/hadoop-annotations-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common-2.0.0-cdh4.2.0.jar:/usr/lib/hadoop/hadoop-common.jar:/mnt/mapred/local/taskTracker/distcache/-3359360379488511723_1277082233_1150217957/ip-10-147-191-48.ec2.internal/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/sqoop-launcher.jar:/mnt/mapred/local/taskTracker/distcache/-3926870669093555941_811939984_1868217431/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoo
> p/avro-ipc-1.7.3-tests.jar:/mnt/mapred/local/taskTracker/distcache/4298483340499219542_2138859044_1868217453/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-ipc-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/-1975491928978283532_-1185784129_1868217475/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/avro-mapred-1.7.3.jar:/mnt/mapred/local/taskTracker/distcache/8448357694523829347_-1491317642_1868217500/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-1.7.0.jar:/mnt/mapred/local/taskTracker/distcache/-334079317866666788_-232552041_1868217523/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-beanutils-core-1.8.0.jar:/mnt/mapred/local/taskTracker/distcache/-644448586073609072_-372575484_1868217547/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-configuration-1.6.jar:/mnt/mapred/local/taskTracker/distcache/246054495199880993_357051411_1868217589/ip-10-147-191-48.ec2.internal/
> user/oozie/share/lib/sqoop/commons-digester-1.8.jar:/mnt/mapred/local/taskTracker/distcache/-4144140590487253933_766503300_1868217614/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/commons-io-2.1.jar:/mnt/mapred/local/taskTracker/distcache/5753546228468760323_-553874526_1868217665/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/guava-11.0.2.jar:/mnt/mapred/local/taskTracker/distcache/8662866019753074519_-1003108238_1868217924/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hbase-0.94.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4840377369495417802_845918870_1868217954/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/high-scale-lib-1.1.1.jar:/mnt/mapred/local/taskTracker/distcache/-142623586016294918_445946554_1868217996/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/hsqldb-1.8.0.7.jar:/mnt/mapred/local/taskTracker/distcache/5142583573507929188_-105788043_1868218025/ip-10-147-191-48.ec2.internal/us
> er/oozie/share/lib/sqoop/httpclient-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/-6413369533723682352_-180904991_1868218051/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/httpcore-4.1.3.jar:/mnt/mapred/local/taskTracker/distcache/3446550291828223401_633399508_1868218088/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/jsr305-1.3.9.jar:/mnt/mapred/local/taskTracker/distcache/-6492962697058022370_-616322509_1868218133/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/libthrift-0.9.0.jar:/mnt/mapred/local/taskTracker/distcache/7049151581973813525_2017354882_1868218156/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/metrics-core-2.1.2.jar:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar:/mnt/mapred/local/taskTracker/distcache/-7982675208196719520_-1207761111_1868218197/ip-10-147-191-48.ec2.interna
> l/user/oozie/share/lib/sqoop/netty-3.4.0.Final.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-412267918101123238_-517736257_1868215515/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/oozie/json-simple-1.1.jar:/mnt/mapred/local/taskTracker/ubuntu/jobcache/job_201305131705_0181/attempt_201305131705_0181_m_000000_0/work:/usr/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.2.0.jar:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,206 INFO org.apache.sqoop.orm.CompilationManager: Writing jar file: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar 2013-05-21 00:48:48,210 DEBUG
> org.apache.sqoop.orm.CompilationManager: Scanning for .class files in directory: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca 2013-05-21 00:48:48,221 DEBUG org.apache.sqoop.orm.CompilationManager: Got classfile: /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.class -> CHRYSLER_Anametrix_Aggregated.class 2013-05-21 00:48:48,236 DEBUG org.apache.sqoop.orm.CompilationManager: Finished writing jar file /tmp/sqoop-mapred/compile/f9c22cb042b8b020c4190df2f8e30bca/CHRYSLER_Anametrix_Aggregated.jar 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: MySQL Connector upsert functionality is using INSERT ON 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: DUPLICATE KEY UPDATE clause that relies on table's unique key. 2013-05-21 00:48:48,243 WARN org.apache.sqoop.manager.MySQLManager: Insert/update distinction is therefore independent on column 2013-05-21 00:48:48,243 WARN
> org.apache.sqoop.manager.MySQLManager: names specified in --update-key parameter. Please see MySQL 2013-05-21 00:48:48,245 WARN org.apache.sqoop.manager.MySQLManager: documentation for additional limitations. 2013-05-21 00:48:48,290 INFO org.apache.sqoop.mapreduce.ExportJobBase: Beginning export of CHRYSLER_Anametrix_Aggregated 2013-05-21 00:48:48,319 DEBUG org.apache.sqoop.mapreduce.JobBase: Using InputFormat: class org.apache.sqoop.mapreduce.ExportInputFormat 2013-05-21 00:48:48,575 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,581 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath:
> file:/mnt/mapred/local/taskTracker/distcache/623583253106481693_846556431_1142287025/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/mysql-connector-java-5.1.24-bin.jar 2013-05-21 00:48:48,585 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,588 DEBUG org.apache.sqoop.mapreduce.JobBase: Adding to job classpath: file:/mnt/mapred/local/taskTracker/distcache/-4577237135972373202_-1630140361_1868218251/ip-10-147-191-48.ec2.internal/user/oozie/share/lib/sqoop/sqoop-1.4.2-cdh4.2.0.jar 2013-05-21 00:48:48,588 WARN org.apache.sqoop.mapreduce.JobBase: SQOOP_HOME is unset. May not be able to find all job dependencies. 2013-05-21 00:48:48,739 WARN org.apache.hadoop.mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications
> should implement Tool for the same. 2013-05-21 00:48:49,058 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Target numMapTasks=1 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Total input bytes=8187428 2013-05-21 00:48:49,059 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: maxSplitSize=8187428 2013-05-21 00:48:49,063 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat: Total input paths to process : 1 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Generated splits: 2013-05-21 00:48:49,073 DEBUG org.apache.sqoop.mapreduce.ExportInputFormat: Paths:/user/hive/warehouse/chrysler_anametrix_aggregated/part-AG-00000-1.txt:0+8187428 Locations:ip-10-147-191-48.ec2.internal:; 2013-05-21 00:48:49,403 INFO org.apache.hadoop.mapred.JobClient: Running job: job_201305131705_0182
> 2013-05-21 00:48:50,417 INFO org.apache.hadoop.mapred.JobClient: map 0% reduce 0% 2013-05-21 00:49:06,478 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_0, Status : FAILED 2013-05-21 00:49:16,703 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_1, Status : FAILED 2013-05-21 00:49:25,861 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201305131705_0182_m_000000_2, Status : FAILED 2013-05-21 00:49:42,012 INFO org.apache.hadoop.mapred.JobClient: Job complete: job_201305131705_0182 2013-05-21 00:49:42,015 INFO org.apache.hadoop.mapred.JobClient: Counters: 7 2013-05-21 00:49:42,020 INFO org.apache.hadoop.mapred.JobClient: Job Counters 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient: Failed map tasks=1 2013-05-21 00:49:42,023 INFO org.apache.hadoop.mapred.JobClient: Launched map tasks=4 2013-05-21 00:49:42,023 INFO
> org.apache.hadoop.mapred.JobClient: Data-local map tasks=4 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all maps in occupied slots (ms)=50187 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all reduces in occupied slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 2013-05-21 00:49:42,024 INFO org.apache.hadoop.mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 2013-05-21 00:49:42,032 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 2013-05-21 00:49:42,033 INFO org.apache.sqoop.mapreduce.ExportJobBase: Transferred 0 bytes in 53.4336 seconds (0 bytes/sec) 2013-05-21 00:49:42,035 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated.
> Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-05-21 00:49:42,035 INFO org.apache.sqoop.mapreduce.ExportJobBase: Exported 0 records. 2013-05-21 00:49:42,035 ERROR org.apache.sqoop.tool.ExportTool: Error during export: Export job failed! 2013-05-21 00:49:42,063 INFO org.apache.hadoop.mapred.Task: Task:attempt_201305131705_0181_m_000000_0 is done. And is in the process of commiting 2013-05-21 00:49:43,124 INFO org.apache.hadoop.mapred.Task: Task attempt_201305131705_0181_m_000000_0 is allowed to commit now 2013-05-21 00:49:43,292 INFO org.apache.hadoop.mapred.FileOutputCommitter: Saved output of task 'attempt_201305131705_0181_m_000000_0' to hdfs://ip-10-147-191-48.ec2.internal:8020/user/ubuntu/oozie-oozi/0000042-130520131050057-oozie-oozi-W/sqoop-node--sqoop/output 2013-05-21 00:49:43,300 INFO org.apache.hadoop.mapred.Task: Task 'attempt_201305131705_0181_m_000000_0' done. 2013-05-21 00:49:43,309 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
> Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
>
> Regards,
> Andy Tey
> Student !!