You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Muthu Pandi <mu...@gmail.com> on 2014/08/25 06:55:28 UTC
Exporting to non standard databases
Hi every one,
Is it possible to export data from HDFS to vetica.I
successfully imported data from vertica to HDFS using sqoop.When i Export i
get the following error.Please give me solution as soon as possible.this is
the error which am getting,
*WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver
however appropriate connection manager is not being set (via
--connection-manager). Sqoop is going to fall back to
org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
which connection manager should be used next time.*
will above will have impact.How do i create connection manager for vertica.
My sqoop query is
sqoop export --driver com.vertica.jdbc.Driver --connect
"jdbc:vertica://X.X.X.X:5433/dbname" --username username -P --table
tablename -m 1 --export-dir /ver.txt --input-fields-terminated-by ','
--lines-terminated-by '\n'
and am using hadoop2.4.1 and sqoop1.4.4 version
14/08/25 10:12:57 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
14/08/25 10:12:57 WARN sqoop.ConnFactory: Parameter --driver is set to an
explicit driver however appropriate connection manager is not being set
(via --connection-manager). Sqoop is going to fall back to
org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
which connection manager should be used next time.
14/08/25 10:12:57 INFO manager.SqlManager: Using default fetchSize of 1000
14/08/25 10:12:57 INFO tool.CodeGenTool: Beginning code generation
14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM AGE AS t WHERE 1=0
14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM AGE AS t WHERE 1=0
14/08/25 10:12:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
/opt/hadoop-2.4.1
Note: /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
14/08/25 10:13:02 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.jar
14/08/25 10:13:03 INFO mapreduce.ExportJobBase: Beginning export of AGE
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/08/25 10:13:03 INFO Configuration.deprecation: mapred.jar is deprecated.
Instead, use mapreduce.job.jar
14/08/25 10:13:06 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM AGE AS t WHERE 1=0
14/08/25 10:13:06 INFO Configuration.deprecation:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
14/08/25 10:13:06 INFO Configuration.deprecation:
mapred.map.tasks.speculative.execution is deprecated. Instead, use
mapreduce.map.speculative
14/08/25 10:13:06 INFO Configuration.deprecation: mapred.map.tasks is
deprecated. Instead, use mapreduce.job.maps
14/08/25 10:13:06 INFO client.RMProxy: Connecting to ResourceManager at
nn01/10.10.200.74:8032
14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to process
: 1
14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to process
: 1
14/08/25 10:13:27 INFO mapreduce.JobSubmitter: number of splits:1
14/08/25 10:13:28 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1408941947869_0001
14/08/25 10:13:30 INFO impl.YarnClientImpl: Submitted application
application_1408941947869_0001
14/08/25 10:13:30 INFO mapreduce.Job: The url to track the job:
http://nn01:8088/proxy/application_1408941947869_0001/
14/08/25 10:13:30 INFO mapreduce.Job: Running job: job_1408941947869_0001
14/08/25 10:13:44 INFO mapreduce.Job: Job job_1408941947869_0001 running in
uber mode : false
14/08/25 10:13:44 INFO mapreduce.Job: map 0% reduce 0%
14/08/25 10:14:11 INFO mapreduce.Job: map 100% reduce 0%
14/08/25 10:14:11 INFO mapreduce.Job: Task Id :
attempt_1408941947869_0001_m_000000_0, Status : FAILED
Error: java.io.IOException: java.sql.SQLException: [Vertica][VJDBC](4856)
ERROR: Syntax error at or near ","
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
error at or near ","
at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown Source)
at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown
Source)
at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
at
org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
... 8 more
Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856) ERROR:
Syntax error at or near ","
... 20 more
14/08/25 10:14:12 INFO mapreduce.Job: map 0% reduce 0%
14/08/25 10:14:25 INFO mapreduce.Job: Task Id :
attempt_1408941947869_0001_m_000000_1, Status : FAILED
Error: java.io.IOException: java.sql.SQLException: [Vertica][VJDBC](4856)
ERROR: Syntax error at or near ","
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
error at or near ","
at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown Source)
at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown
Source)
at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
at
org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
... 8 more
Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856) ERROR:
Syntax error at or near ","
... 20 more
14/08/25 10:14:34 INFO mapreduce.Job: Task Id :
attempt_1408941947869_0001_m_000000_2, Status : FAILED
Error: java.io.IOException: java.sql.SQLSyntaxErrorException:
[Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](4856) ERROR:
Syntax error at or near ","
at com.vertica.util.ServerErrorData.buildException(Unknown Source)
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown Source)
at com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown
Source)
at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown Source)
at
org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
... 8 more
Caused by: com.vertica.support.exceptions.SyntaxErrorException:
[Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
... 21 more
14/08/25 10:14:43 INFO mapreduce.Job: map 100% reduce 0%
14/08/25 10:14:44 INFO mapreduce.Job: Job job_1408941947869_0001 failed
with state FAILED due to: Task failed task_1408941947869_0001_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0
14/08/25 10:14:45 INFO mapreduce.Job: Counters: 9
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=3
Rack-local map tasks=1
Total time spent by all maps in occupied slots (ms)=53177
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=53177
Total vcore-seconds taken by all map tasks=53177
Total megabyte-seconds taken by all map tasks=54453248
14/08/25 10:14:45 WARN mapreduce.Counters: Group FileSystemCounters is
deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Transferred 0 bytes in
99.0548 seconds (0 bytes/sec)
14/08/25 10:14:45 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Exported 0 records.
14/08/25 10:14:45 ERROR tool.ExportTool: Error during export: Export job
failed!
*Regards Muthupandi.K*
[image: Picture (Device Independent Bitmap)]
Re: Exporting to non standard databases
Posted by Muthu Pandi <mu...@gmail.com>.
Pls Ignore the previous mail Thanks guys for the help and for info found
that "--batch" option is working which support batch export without "
-Dsqoop.export.records.per.statement=1".
*Regards Muthupandi.K*
[image: Picture (Device Independent Bitmap)]
On Mon, Aug 25, 2014 at 11:18 AM, Muthu Pandi <mu...@gmail.com> wrote:
> Thanks guys for the help and for info found that "--direct" option is
> working which support batch export without " -Dsqoop.export.records.per.
> statement=1".
>
>
>
> *Regards Muthupandi.K*
>
> [image: Picture (Device Independent Bitmap)]
>
>
>
> On Mon, Aug 25, 2014 at 11:04 AM, Gwen Shapira <gs...@cloudera.com>
> wrote:
>
>> It won't be super fast since its row-by-row insert, but 2GB is not huge,
>> so I'll expect this will take only few minutes.
>>
>>
>> On Sun, Aug 24, 2014 at 10:23 PM, Muthu Pandi <mu...@gmail.com>
>> wrote:
>>
>>> Spot on!!! Jarcec
>>>
>>> Thanks for the quick reply, you have saved my lot of time.
>>>
>>> Can this be used for exporting larger data since we are exporting
>>> in bulk. My data will be of size 2GB Per day and will export at end of the
>>> day.
>>>
>>>
>>>
>>>
>>> *Regards Muthupandi.K*
>>>
>>> [image: Picture (Device Independent Bitmap)]
>>>
>>>
>>>
>>> On Mon, Aug 25, 2014 at 10:40 AM, Jarek Jarcec Cecho <ja...@apache.org>
>>> wrote:
>>>
>>>> Hi Muthu,
>>>> the given message is a “Warning” and not an “Error”. Sqoop is warning
>>>> you that you are using —driver option that will force Sqoop to use Generic
>>>> JDBC connector. That is fine in your case as Sqoop don’t have specialized
>>>> connector for Vertica. You can always use --connect-manager parameter to
>>>> get rid of the message as is the warning suggesting:
>>>>
>>>> sqoop export --connect-manager
>>>> org.apache.sqoop.manager.GenericJdbcManager ...
>>>>
>>>> You are however hitting a different problem, my best guess that Vertica
>>>> is not supporting multi value insert statements that Sqoop is generating by
>>>> default. Try to disable this feature using property
>>>> sqoop.export.records.per.statement:
>>>>
>>>> sqoop export -Dsqoop.export.records.per.statement=1 —connect ...
>>>>
>>>> Jarcec
>>>>
>>>> On Aug 25, 2014, at 6:55 AM, Muthu Pandi <mu...@gmail.com> wrote:
>>>>
>>>> > Hi every one,
>>>> > Is it possible to export data from HDFS to vetica.I
>>>> successfully imported data from vertica to HDFS using sqoop.When i Export i
>>>> get the following error.Please give me solution as soon as possible.this is
>>>> the error which am getting,
>>>> >
>>>> > WARN sqoop.ConnFactory: Parameter --driver is set to an explicit
>>>> driver however appropriate connection manager is not being set (via
>>>> --connection-manager). Sqoop is going to fall back to
>>>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>>>> which connection manager should be used next time.
>>>> >
>>>> > will above will have impact.How do i create connection manager for
>>>> vertica.
>>>> >
>>>> > My sqoop query is
>>>> >
>>>> > sqoop export --driver com.vertica.jdbc.Driver --connect
>>>> "jdbc:vertica://X.X.X.X:5433/dbname" --username username -P --table
>>>> tablename -m 1 --export-dir /ver.txt --input-fields-terminated-by ','
>>>> --lines-terminated-by '\n'
>>>> >
>>>> >
>>>> > and am using hadoop2.4.1 and sqoop1.4.4 version
>>>> >
>>>> >
>>>> >
>>>> > 14/08/25 10:12:57 WARN tool.BaseSqoopTool: Setting your password on
>>>> the command-line is insecure. Consider using -P instead.
>>>> > 14/08/25 10:12:57 WARN sqoop.ConnFactory: Parameter --driver is set
>>>> to an explicit driver however appropriate connection manager is not being
>>>> set (via --connection-manager). Sqoop is going to fall back to
>>>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>>>> which connection manager should be used next time.
>>>> > 14/08/25 10:12:57 INFO manager.SqlManager: Using default fetchSize of
>>>> 1000
>>>> > 14/08/25 10:12:57 INFO tool.CodeGenTool: Beginning code generation
>>>> > 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement:
>>>> SELECT t.* FROM AGE AS t WHERE 1=0
>>>> > 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement:
>>>> SELECT t.* FROM AGE AS t WHERE 1=0
>>>> > 14/08/25 10:12:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>>>> /opt/hadoop-2.4.1
>>>> > Note:
>>>> /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.java uses
>>>> or overrides a deprecated API.
>>>> > Note: Recompile with -Xlint:deprecation for details.
>>>> > 14/08/25 10:13:02 INFO orm.CompilationManager: Writing jar file:
>>>> /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.jar
>>>> > 14/08/25 10:13:03 INFO mapreduce.ExportJobBase: Beginning export of
>>>> AGE
>>>> > SLF4J: Class path contains multiple SLF4J bindings.
>>>> > SLF4J: Found binding in
>>>> [jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>>> > SLF4J: Found binding in
>>>> [jar:file:/opt/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>>> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>>> explanation.
>>>> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>>> > 14/08/25 10:13:03 INFO Configuration.deprecation: mapred.jar is
>>>> deprecated. Instead, use mapreduce.job.jar
>>>> > 14/08/25 10:13:06 INFO manager.SqlManager: Executing SQL statement:
>>>> SELECT t.* FROM AGE AS t WHERE 1=0
>>>> > 14/08/25 10:13:06 INFO Configuration.deprecation:
>>>> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
>>>> mapreduce.reduce.speculative
>>>> > 14/08/25 10:13:06 INFO Configuration.deprecation:
>>>> mapred.map.tasks.speculative.execution is deprecated. Instead, use
>>>> mapreduce.map.speculative
>>>> > 14/08/25 10:13:06 INFO Configuration.deprecation: mapred.map.tasks is
>>>> deprecated. Instead, use mapreduce.job.maps
>>>> > 14/08/25 10:13:06 INFO client.RMProxy: Connecting to ResourceManager
>>>> at nn01/10.10.200.74:8032
>>>> > 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to
>>>> process : 1
>>>> > 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to
>>>> process : 1
>>>> > 14/08/25 10:13:27 INFO mapreduce.JobSubmitter: number of splits:1
>>>> > 14/08/25 10:13:28 INFO mapreduce.JobSubmitter: Submitting tokens for
>>>> job: job_1408941947869_0001
>>>> > 14/08/25 10:13:30 INFO impl.YarnClientImpl: Submitted application
>>>> application_1408941947869_0001
>>>> > 14/08/25 10:13:30 INFO mapreduce.Job: The url to track the job:
>>>> http://nn01:8088/proxy/application_1408941947869_0001/
>>>> > 14/08/25 10:13:30 INFO mapreduce.Job: Running job:
>>>> job_1408941947869_0001
>>>> > 14/08/25 10:13:44 INFO mapreduce.Job: Job job_1408941947869_0001
>>>> running in uber mode : false
>>>> > 14/08/25 10:13:44 INFO mapreduce.Job: map 0% reduce 0%
>>>> > 14/08/25 10:14:11 INFO mapreduce.Job: map 100% reduce 0%
>>>> > 14/08/25 10:14:11 INFO mapreduce.Job: Task Id :
>>>> attempt_1408941947869_0001_m_000000_0, Status : FAILED
>>>> > Error: java.io.IOException: java.sql.SQLException:
>>>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>>>> > at
>>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>>> > at
>>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
>>>> > at
>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>>>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>>>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>>>> > at java.security.AccessController.doPrivileged(Native Method)
>>>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>>>> > at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>>>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>>>> > Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR:
>>>> Syntax error at or near ","
>>>> > at com.vertica.util.ServerErrorData.buildException(Unknown
>>>> Source)
>>>> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
>>>> Source)
>>>> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown
>>>> Source)
>>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>>> > at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
>>>> > at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown
>>>> Source)
>>>> > at
>>>> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
>>>> > at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
>>>> > at
>>>> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
>>>> > at
>>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>>> > at
>>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>>> > ... 8 more
>>>> > Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856)
>>>> ERROR: Syntax error at or near ","
>>>> > ... 20 more
>>>> >
>>>> > 14/08/25 10:14:12 INFO mapreduce.Job: map 0% reduce 0%
>>>> > 14/08/25 10:14:25 INFO mapreduce.Job: Task Id :
>>>> attempt_1408941947869_0001_m_000000_1, Status : FAILED
>>>> > Error: java.io.IOException: java.sql.SQLException:
>>>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>>>> > at
>>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>>> > at
>>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
>>>> > at
>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>>>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>>>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>>>> > at java.security.AccessController.doPrivileged(Native Method)
>>>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>>>> > at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>>>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>>>> > Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR:
>>>> Syntax error at or near ","
>>>> > at com.vertica.util.ServerErrorData.buildException(Unknown
>>>> Source)
>>>> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
>>>> Source)
>>>> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown
>>>> Source)
>>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>>> > at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
>>>> > at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown
>>>> Source)
>>>> > at
>>>> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
>>>> > at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
>>>> > at
>>>> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
>>>> > at
>>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>>> > at
>>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>>> > ... 8 more
>>>> > Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856)
>>>> ERROR: Syntax error at or near ","
>>>> > ... 20 more
>>>> >
>>>> > 14/08/25 10:14:34 INFO mapreduce.Job: Task Id :
>>>> attempt_1408941947869_0001_m_000000_2, Status : FAILED
>>>> > Error: java.io.IOException: java.sql.SQLSyntaxErrorException:
>>>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>>>> > at
>>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>>> > at
>>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
>>>> > at
>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>>>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>>>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>>>> > at java.security.AccessController.doPrivileged(Native Method)
>>>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>>>> > at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>>>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>>>> > Caused by: java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](4856)
>>>> ERROR: Syntax error at or near ","
>>>> > at com.vertica.util.ServerErrorData.buildException(Unknown
>>>> Source)
>>>> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
>>>> Source)
>>>> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown
>>>> Source)
>>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>>> > at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown
>>>> Source)
>>>> > at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown
>>>> Source)
>>>> > at
>>>> com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
>>>> > at
>>>> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
>>>> > at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown
>>>> Source)
>>>> > at
>>>> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
>>>> > at
>>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>>> > at
>>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>>> > ... 8 more
>>>> > Caused by: com.vertica.support.exceptions.SyntaxErrorException:
>>>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>>>> > ... 21 more
>>>> >
>>>> > 14/08/25 10:14:43 INFO mapreduce.Job: map 100% reduce 0%
>>>> > 14/08/25 10:14:44 INFO mapreduce.Job: Job job_1408941947869_0001
>>>> failed with state FAILED due to: Task failed
>>>> task_1408941947869_0001_m_000000
>>>> > Job failed as tasks failed. failedMaps:1 failedReduces:0
>>>> >
>>>> > 14/08/25 10:14:45 INFO mapreduce.Job: Counters: 9
>>>> > Job Counters
>>>> > Failed map tasks=4
>>>> > Launched map tasks=4
>>>> > Other local map tasks=3
>>>> > Rack-local map tasks=1
>>>> > Total time spent by all maps in occupied slots
>>>> (ms)=53177
>>>> > Total time spent by all reduces in occupied slots (ms)=0
>>>> > Total time spent by all map tasks (ms)=53177
>>>> > Total vcore-seconds taken by all map tasks=53177
>>>> > Total megabyte-seconds taken by all map tasks=54453248
>>>> > 14/08/25 10:14:45 WARN mapreduce.Counters: Group FileSystemCounters
>>>> is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
>>>> > 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Transferred 0 bytes
>>>> in 99.0548 seconds (0 bytes/sec)
>>>> > 14/08/25 10:14:45 WARN mapreduce.Counters: Group
>>>> org.apache.hadoop.mapred.Task$Counter is deprecated. Use
>>>> org.apache.hadoop.mapreduce.TaskCounter instead
>>>> > 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Exported 0 records.
>>>> > 14/08/25 10:14:45 ERROR tool.ExportTool: Error during export: Export
>>>> job failed!
>>>> > Regards
>>>> > Muthupandi.K
>>>> >
>>>> >
>>>> >
>>>> > <image002.png>
>>>> >
>>>>
>>>>
>>>
>>
>
Re: Exporting to non standard databases
Posted by Muthu Pandi <mu...@gmail.com>.
Thanks guys for the help and for info found that "--direct" option is
working which support batch export without " -Dsqoop.export.records.per.
statement=1".
*Regards Muthupandi.K*
[image: Picture (Device Independent Bitmap)]
On Mon, Aug 25, 2014 at 11:04 AM, Gwen Shapira <gs...@cloudera.com>
wrote:
> It won't be super fast since its row-by-row insert, but 2GB is not huge,
> so I'll expect this will take only few minutes.
>
>
> On Sun, Aug 24, 2014 at 10:23 PM, Muthu Pandi <mu...@gmail.com> wrote:
>
>> Spot on!!! Jarcec
>>
>> Thanks for the quick reply, you have saved my lot of time.
>>
>> Can this be used for exporting larger data since we are exporting
>> in bulk. My data will be of size 2GB Per day and will export at end of the
>> day.
>>
>>
>>
>>
>> *Regards Muthupandi.K*
>>
>> [image: Picture (Device Independent Bitmap)]
>>
>>
>>
>> On Mon, Aug 25, 2014 at 10:40 AM, Jarek Jarcec Cecho <ja...@apache.org>
>> wrote:
>>
>>> Hi Muthu,
>>> the given message is a “Warning” and not an “Error”. Sqoop is warning
>>> you that you are using —driver option that will force Sqoop to use Generic
>>> JDBC connector. That is fine in your case as Sqoop don’t have specialized
>>> connector for Vertica. You can always use --connect-manager parameter to
>>> get rid of the message as is the warning suggesting:
>>>
>>> sqoop export --connect-manager
>>> org.apache.sqoop.manager.GenericJdbcManager ...
>>>
>>> You are however hitting a different problem, my best guess that Vertica
>>> is not supporting multi value insert statements that Sqoop is generating by
>>> default. Try to disable this feature using property
>>> sqoop.export.records.per.statement:
>>>
>>> sqoop export -Dsqoop.export.records.per.statement=1 —connect ...
>>>
>>> Jarcec
>>>
>>> On Aug 25, 2014, at 6:55 AM, Muthu Pandi <mu...@gmail.com> wrote:
>>>
>>> > Hi every one,
>>> > Is it possible to export data from HDFS to vetica.I
>>> successfully imported data from vertica to HDFS using sqoop.When i Export i
>>> get the following error.Please give me solution as soon as possible.this is
>>> the error which am getting,
>>> >
>>> > WARN sqoop.ConnFactory: Parameter --driver is set to an explicit
>>> driver however appropriate connection manager is not being set (via
>>> --connection-manager). Sqoop is going to fall back to
>>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>>> which connection manager should be used next time.
>>> >
>>> > will above will have impact.How do i create connection manager for
>>> vertica.
>>> >
>>> > My sqoop query is
>>> >
>>> > sqoop export --driver com.vertica.jdbc.Driver --connect
>>> "jdbc:vertica://X.X.X.X:5433/dbname" --username username -P --table
>>> tablename -m 1 --export-dir /ver.txt --input-fields-terminated-by ','
>>> --lines-terminated-by '\n'
>>> >
>>> >
>>> > and am using hadoop2.4.1 and sqoop1.4.4 version
>>> >
>>> >
>>> >
>>> > 14/08/25 10:12:57 WARN tool.BaseSqoopTool: Setting your password on
>>> the command-line is insecure. Consider using -P instead.
>>> > 14/08/25 10:12:57 WARN sqoop.ConnFactory: Parameter --driver is set to
>>> an explicit driver however appropriate connection manager is not being set
>>> (via --connection-manager). Sqoop is going to fall back to
>>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>>> which connection manager should be used next time.
>>> > 14/08/25 10:12:57 INFO manager.SqlManager: Using default fetchSize of
>>> 1000
>>> > 14/08/25 10:12:57 INFO tool.CodeGenTool: Beginning code generation
>>> > 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement:
>>> SELECT t.* FROM AGE AS t WHERE 1=0
>>> > 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement:
>>> SELECT t.* FROM AGE AS t WHERE 1=0
>>> > 14/08/25 10:12:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>>> /opt/hadoop-2.4.1
>>> > Note:
>>> /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.java uses
>>> or overrides a deprecated API.
>>> > Note: Recompile with -Xlint:deprecation for details.
>>> > 14/08/25 10:13:02 INFO orm.CompilationManager: Writing jar file:
>>> /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.jar
>>> > 14/08/25 10:13:03 INFO mapreduce.ExportJobBase: Beginning export of AGE
>>> > SLF4J: Class path contains multiple SLF4J bindings.
>>> > SLF4J: Found binding in
>>> [jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> > SLF4J: Found binding in
>>> [jar:file:/opt/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>> explanation.
>>> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>> > 14/08/25 10:13:03 INFO Configuration.deprecation: mapred.jar is
>>> deprecated. Instead, use mapreduce.job.jar
>>> > 14/08/25 10:13:06 INFO manager.SqlManager: Executing SQL statement:
>>> SELECT t.* FROM AGE AS t WHERE 1=0
>>> > 14/08/25 10:13:06 INFO Configuration.deprecation:
>>> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
>>> mapreduce.reduce.speculative
>>> > 14/08/25 10:13:06 INFO Configuration.deprecation:
>>> mapred.map.tasks.speculative.execution is deprecated. Instead, use
>>> mapreduce.map.speculative
>>> > 14/08/25 10:13:06 INFO Configuration.deprecation: mapred.map.tasks is
>>> deprecated. Instead, use mapreduce.job.maps
>>> > 14/08/25 10:13:06 INFO client.RMProxy: Connecting to ResourceManager
>>> at nn01/10.10.200.74:8032
>>> > 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to
>>> process : 1
>>> > 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to
>>> process : 1
>>> > 14/08/25 10:13:27 INFO mapreduce.JobSubmitter: number of splits:1
>>> > 14/08/25 10:13:28 INFO mapreduce.JobSubmitter: Submitting tokens for
>>> job: job_1408941947869_0001
>>> > 14/08/25 10:13:30 INFO impl.YarnClientImpl: Submitted application
>>> application_1408941947869_0001
>>> > 14/08/25 10:13:30 INFO mapreduce.Job: The url to track the job:
>>> http://nn01:8088/proxy/application_1408941947869_0001/
>>> > 14/08/25 10:13:30 INFO mapreduce.Job: Running job:
>>> job_1408941947869_0001
>>> > 14/08/25 10:13:44 INFO mapreduce.Job: Job job_1408941947869_0001
>>> running in uber mode : false
>>> > 14/08/25 10:13:44 INFO mapreduce.Job: map 0% reduce 0%
>>> > 14/08/25 10:14:11 INFO mapreduce.Job: map 100% reduce 0%
>>> > 14/08/25 10:14:11 INFO mapreduce.Job: Task Id :
>>> attempt_1408941947869_0001_m_000000_0, Status : FAILED
>>> > Error: java.io.IOException: java.sql.SQLException:
>>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>>> > at
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>> > at
>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
>>> > at
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>>> > at java.security.AccessController.doPrivileged(Native Method)
>>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>>> > at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>>> > Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
>>> error at or near ","
>>> > at com.vertica.util.ServerErrorData.buildException(Unknown
>>> Source)
>>> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
>>> Source)
>>> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>> > at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
>>> > at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown
>>> Source)
>>> > at
>>> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
>>> > at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
>>> > at
>>> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
>>> > at
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>> > at
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>> > ... 8 more
>>> > Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856)
>>> ERROR: Syntax error at or near ","
>>> > ... 20 more
>>> >
>>> > 14/08/25 10:14:12 INFO mapreduce.Job: map 0% reduce 0%
>>> > 14/08/25 10:14:25 INFO mapreduce.Job: Task Id :
>>> attempt_1408941947869_0001_m_000000_1, Status : FAILED
>>> > Error: java.io.IOException: java.sql.SQLException:
>>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>>> > at
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>> > at
>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
>>> > at
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>>> > at java.security.AccessController.doPrivileged(Native Method)
>>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>>> > at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>>> > Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
>>> error at or near ","
>>> > at com.vertica.util.ServerErrorData.buildException(Unknown
>>> Source)
>>> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
>>> Source)
>>> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>> > at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
>>> > at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown
>>> Source)
>>> > at
>>> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
>>> > at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
>>> > at
>>> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
>>> > at
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>> > at
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>> > ... 8 more
>>> > Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856)
>>> ERROR: Syntax error at or near ","
>>> > ... 20 more
>>> >
>>> > 14/08/25 10:14:34 INFO mapreduce.Job: Task Id :
>>> attempt_1408941947869_0001_m_000000_2, Status : FAILED
>>> > Error: java.io.IOException: java.sql.SQLSyntaxErrorException:
>>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>>> > at
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>> > at
>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
>>> > at
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>>> > at java.security.AccessController.doPrivileged(Native Method)
>>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>>> > at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>>> > Caused by: java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](4856)
>>> ERROR: Syntax error at or near ","
>>> > at com.vertica.util.ServerErrorData.buildException(Unknown
>>> Source)
>>> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
>>> Source)
>>> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>>> > at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown
>>> Source)
>>> > at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown
>>> Source)
>>> > at
>>> com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
>>> > at
>>> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
>>> > at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown
>>> Source)
>>> > at
>>> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
>>> > at
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>> > at
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>> > ... 8 more
>>> > Caused by: com.vertica.support.exceptions.SyntaxErrorException:
>>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>>> > ... 21 more
>>> >
>>> > 14/08/25 10:14:43 INFO mapreduce.Job: map 100% reduce 0%
>>> > 14/08/25 10:14:44 INFO mapreduce.Job: Job job_1408941947869_0001
>>> failed with state FAILED due to: Task failed
>>> task_1408941947869_0001_m_000000
>>> > Job failed as tasks failed. failedMaps:1 failedReduces:0
>>> >
>>> > 14/08/25 10:14:45 INFO mapreduce.Job: Counters: 9
>>> > Job Counters
>>> > Failed map tasks=4
>>> > Launched map tasks=4
>>> > Other local map tasks=3
>>> > Rack-local map tasks=1
>>> > Total time spent by all maps in occupied slots (ms)=53177
>>> > Total time spent by all reduces in occupied slots (ms)=0
>>> > Total time spent by all map tasks (ms)=53177
>>> > Total vcore-seconds taken by all map tasks=53177
>>> > Total megabyte-seconds taken by all map tasks=54453248
>>> > 14/08/25 10:14:45 WARN mapreduce.Counters: Group FileSystemCounters is
>>> deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
>>> > 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Transferred 0 bytes in
>>> 99.0548 seconds (0 bytes/sec)
>>> > 14/08/25 10:14:45 WARN mapreduce.Counters: Group
>>> org.apache.hadoop.mapred.Task$Counter is deprecated. Use
>>> org.apache.hadoop.mapreduce.TaskCounter instead
>>> > 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Exported 0 records.
>>> > 14/08/25 10:14:45 ERROR tool.ExportTool: Error during export: Export
>>> job failed!
>>> > Regards
>>> > Muthupandi.K
>>> >
>>> >
>>> >
>>> > <image002.png>
>>> >
>>>
>>>
>>
>
Re: Exporting to non standard databases
Posted by Gwen Shapira <gs...@cloudera.com>.
It won't be super fast since its row-by-row insert, but 2GB is not huge, so
I'll expect this will take only few minutes.
On Sun, Aug 24, 2014 at 10:23 PM, Muthu Pandi <mu...@gmail.com> wrote:
> Spot on!!! Jarcec
>
> Thanks for the quick reply, you have saved my lot of time.
>
> Can this be used for exporting larger data since we are exporting
> in bulk. My data will be of size 2GB Per day and will export at end of the
> day.
>
>
>
>
> *Regards Muthupandi.K*
>
> [image: Picture (Device Independent Bitmap)]
>
>
>
> On Mon, Aug 25, 2014 at 10:40 AM, Jarek Jarcec Cecho <ja...@apache.org>
> wrote:
>
>> Hi Muthu,
>> the given message is a “Warning” and not an “Error”. Sqoop is warning you
>> that you are using —driver option that will force Sqoop to use Generic JDBC
>> connector. That is fine in your case as Sqoop don’t have specialized
>> connector for Vertica. You can always use --connect-manager parameter to
>> get rid of the message as is the warning suggesting:
>>
>> sqoop export --connect-manager
>> org.apache.sqoop.manager.GenericJdbcManager ...
>>
>> You are however hitting a different problem, my best guess that Vertica
>> is not supporting multi value insert statements that Sqoop is generating by
>> default. Try to disable this feature using property
>> sqoop.export.records.per.statement:
>>
>> sqoop export -Dsqoop.export.records.per.statement=1 —connect ...
>>
>> Jarcec
>>
>> On Aug 25, 2014, at 6:55 AM, Muthu Pandi <mu...@gmail.com> wrote:
>>
>> > Hi every one,
>> > Is it possible to export data from HDFS to vetica.I
>> successfully imported data from vertica to HDFS using sqoop.When i Export i
>> get the following error.Please give me solution as soon as possible.this is
>> the error which am getting,
>> >
>> > WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver
>> however appropriate connection manager is not being set (via
>> --connection-manager). Sqoop is going to fall back to
>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>> which connection manager should be used next time.
>> >
>> > will above will have impact.How do i create connection manager for
>> vertica.
>> >
>> > My sqoop query is
>> >
>> > sqoop export --driver com.vertica.jdbc.Driver --connect
>> "jdbc:vertica://X.X.X.X:5433/dbname" --username username -P --table
>> tablename -m 1 --export-dir /ver.txt --input-fields-terminated-by ','
>> --lines-terminated-by '\n'
>> >
>> >
>> > and am using hadoop2.4.1 and sqoop1.4.4 version
>> >
>> >
>> >
>> > 14/08/25 10:12:57 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>> > 14/08/25 10:12:57 WARN sqoop.ConnFactory: Parameter --driver is set to
>> an explicit driver however appropriate connection manager is not being set
>> (via --connection-manager). Sqoop is going to fall back to
>> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
>> which connection manager should be used next time.
>> > 14/08/25 10:12:57 INFO manager.SqlManager: Using default fetchSize of
>> 1000
>> > 14/08/25 10:12:57 INFO tool.CodeGenTool: Beginning code generation
>> > 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM AGE AS t WHERE 1=0
>> > 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM AGE AS t WHERE 1=0
>> > 14/08/25 10:12:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
>> /opt/hadoop-2.4.1
>> > Note:
>> /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.java uses
>> or overrides a deprecated API.
>> > Note: Recompile with -Xlint:deprecation for details.
>> > 14/08/25 10:13:02 INFO orm.CompilationManager: Writing jar file:
>> /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.jar
>> > 14/08/25 10:13:03 INFO mapreduce.ExportJobBase: Beginning export of AGE
>> > SLF4J: Class path contains multiple SLF4J bindings.
>> > SLF4J: Found binding in
>> [jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> > SLF4J: Found binding in
>> [jar:file:/opt/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> > 14/08/25 10:13:03 INFO Configuration.deprecation: mapred.jar is
>> deprecated. Instead, use mapreduce.job.jar
>> > 14/08/25 10:13:06 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM AGE AS t WHERE 1=0
>> > 14/08/25 10:13:06 INFO Configuration.deprecation:
>> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
>> mapreduce.reduce.speculative
>> > 14/08/25 10:13:06 INFO Configuration.deprecation:
>> mapred.map.tasks.speculative.execution is deprecated. Instead, use
>> mapreduce.map.speculative
>> > 14/08/25 10:13:06 INFO Configuration.deprecation: mapred.map.tasks is
>> deprecated. Instead, use mapreduce.job.maps
>> > 14/08/25 10:13:06 INFO client.RMProxy: Connecting to ResourceManager at
>> nn01/10.10.200.74:8032
>> > 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to
>> process : 1
>> > 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to
>> process : 1
>> > 14/08/25 10:13:27 INFO mapreduce.JobSubmitter: number of splits:1
>> > 14/08/25 10:13:28 INFO mapreduce.JobSubmitter: Submitting tokens for
>> job: job_1408941947869_0001
>> > 14/08/25 10:13:30 INFO impl.YarnClientImpl: Submitted application
>> application_1408941947869_0001
>> > 14/08/25 10:13:30 INFO mapreduce.Job: The url to track the job:
>> http://nn01:8088/proxy/application_1408941947869_0001/
>> > 14/08/25 10:13:30 INFO mapreduce.Job: Running job:
>> job_1408941947869_0001
>> > 14/08/25 10:13:44 INFO mapreduce.Job: Job job_1408941947869_0001
>> running in uber mode : false
>> > 14/08/25 10:13:44 INFO mapreduce.Job: map 0% reduce 0%
>> > 14/08/25 10:14:11 INFO mapreduce.Job: map 100% reduce 0%
>> > 14/08/25 10:14:11 INFO mapreduce.Job: Task Id :
>> attempt_1408941947869_0001_m_000000_0, Status : FAILED
>> > Error: java.io.IOException: java.sql.SQLException:
>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>> > at
>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>> > at
>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
>> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>> > at java.security.AccessController.doPrivileged(Native Method)
>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>> > at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>> > Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
>> error at or near ","
>> > at com.vertica.util.ServerErrorData.buildException(Unknown Source)
>> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
>> Source)
>> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>> > at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
>> > at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown
>> Source)
>> > at
>> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
>> > at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
>> > at
>> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
>> > at
>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>> > at
>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>> > ... 8 more
>> > Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856)
>> ERROR: Syntax error at or near ","
>> > ... 20 more
>> >
>> > 14/08/25 10:14:12 INFO mapreduce.Job: map 0% reduce 0%
>> > 14/08/25 10:14:25 INFO mapreduce.Job: Task Id :
>> attempt_1408941947869_0001_m_000000_1, Status : FAILED
>> > Error: java.io.IOException: java.sql.SQLException:
>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>> > at
>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>> > at
>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
>> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>> > at java.security.AccessController.doPrivileged(Native Method)
>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>> > at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>> > Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
>> error at or near ","
>> > at com.vertica.util.ServerErrorData.buildException(Unknown Source)
>> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
>> Source)
>> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>> > at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
>> > at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown
>> Source)
>> > at
>> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
>> > at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
>> > at
>> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
>> > at
>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>> > at
>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>> > ... 8 more
>> > Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856)
>> ERROR: Syntax error at or near ","
>> > ... 20 more
>> >
>> > 14/08/25 10:14:34 INFO mapreduce.Job: Task Id :
>> attempt_1408941947869_0001_m_000000_2, Status : FAILED
>> > Error: java.io.IOException: java.sql.SQLSyntaxErrorException:
>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>> > at
>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>> > at
>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
>> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>> > at java.security.AccessController.doPrivileged(Native Method)
>> > at javax.security.auth.Subject.doAs(Subject.java:415)
>> > at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>> > Caused by: java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](4856)
>> ERROR: Syntax error at or near ","
>> > at com.vertica.util.ServerErrorData.buildException(Unknown Source)
>> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
>> Source)
>> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
>> > at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown
>> Source)
>> > at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown
>> Source)
>> > at
>> com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
>> > at
>> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
>> > at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown
>> Source)
>> > at
>> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
>> > at
>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>> > at
>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>> > ... 8 more
>> > Caused by: com.vertica.support.exceptions.SyntaxErrorException:
>> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
>> > ... 21 more
>> >
>> > 14/08/25 10:14:43 INFO mapreduce.Job: map 100% reduce 0%
>> > 14/08/25 10:14:44 INFO mapreduce.Job: Job job_1408941947869_0001 failed
>> with state FAILED due to: Task failed task_1408941947869_0001_m_000000
>> > Job failed as tasks failed. failedMaps:1 failedReduces:0
>> >
>> > 14/08/25 10:14:45 INFO mapreduce.Job: Counters: 9
>> > Job Counters
>> > Failed map tasks=4
>> > Launched map tasks=4
>> > Other local map tasks=3
>> > Rack-local map tasks=1
>> > Total time spent by all maps in occupied slots (ms)=53177
>> > Total time spent by all reduces in occupied slots (ms)=0
>> > Total time spent by all map tasks (ms)=53177
>> > Total vcore-seconds taken by all map tasks=53177
>> > Total megabyte-seconds taken by all map tasks=54453248
>> > 14/08/25 10:14:45 WARN mapreduce.Counters: Group FileSystemCounters is
>> deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
>> > 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Transferred 0 bytes in
>> 99.0548 seconds (0 bytes/sec)
>> > 14/08/25 10:14:45 WARN mapreduce.Counters: Group
>> org.apache.hadoop.mapred.Task$Counter is deprecated. Use
>> org.apache.hadoop.mapreduce.TaskCounter instead
>> > 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Exported 0 records.
>> > 14/08/25 10:14:45 ERROR tool.ExportTool: Error during export: Export
>> job failed!
>> > Regards
>> > Muthupandi.K
>> >
>> >
>> >
>> > <image002.png>
>> >
>>
>>
>
Re: Exporting to non standard databases
Posted by Muthu Pandi <mu...@gmail.com>.
Spot on!!! Jarcec
Thanks for the quick reply, you have saved my lot of time.
Can this be used for exporting larger data since we are exporting in
bulk. My data will be of size 2GB Per day and will export at end of the day.
*Regards Muthupandi.K*
[image: Picture (Device Independent Bitmap)]
On Mon, Aug 25, 2014 at 10:40 AM, Jarek Jarcec Cecho <ja...@apache.org>
wrote:
> Hi Muthu,
> the given message is a “Warning” and not an “Error”. Sqoop is warning you
> that you are using —driver option that will force Sqoop to use Generic JDBC
> connector. That is fine in your case as Sqoop don’t have specialized
> connector for Vertica. You can always use --connect-manager parameter to
> get rid of the message as is the warning suggesting:
>
> sqoop export --connect-manager
> org.apache.sqoop.manager.GenericJdbcManager ...
>
> You are however hitting a different problem, my best guess that Vertica is
> not supporting multi value insert statements that Sqoop is generating by
> default. Try to disable this feature using property
> sqoop.export.records.per.statement:
>
> sqoop export -Dsqoop.export.records.per.statement=1 —connect ...
>
> Jarcec
>
> On Aug 25, 2014, at 6:55 AM, Muthu Pandi <mu...@gmail.com> wrote:
>
> > Hi every one,
> > Is it possible to export data from HDFS to vetica.I
> successfully imported data from vertica to HDFS using sqoop.When i Export i
> get the following error.Please give me solution as soon as possible.this is
> the error which am getting,
> >
> > WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver
> however appropriate connection manager is not being set (via
> --connection-manager). Sqoop is going to fall back to
> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
> which connection manager should be used next time.
> >
> > will above will have impact.How do i create connection manager for
> vertica.
> >
> > My sqoop query is
> >
> > sqoop export --driver com.vertica.jdbc.Driver --connect
> "jdbc:vertica://X.X.X.X:5433/dbname" --username username -P --table
> tablename -m 1 --export-dir /ver.txt --input-fields-terminated-by ','
> --lines-terminated-by '\n'
> >
> >
> > and am using hadoop2.4.1 and sqoop1.4.4 version
> >
> >
> >
> > 14/08/25 10:12:57 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> > 14/08/25 10:12:57 WARN sqoop.ConnFactory: Parameter --driver is set to
> an explicit driver however appropriate connection manager is not being set
> (via --connection-manager). Sqoop is going to fall back to
> org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly
> which connection manager should be used next time.
> > 14/08/25 10:12:57 INFO manager.SqlManager: Using default fetchSize of
> 1000
> > 14/08/25 10:12:57 INFO tool.CodeGenTool: Beginning code generation
> > 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement:
> SELECT t.* FROM AGE AS t WHERE 1=0
> > 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement:
> SELECT t.* FROM AGE AS t WHERE 1=0
> > 14/08/25 10:12:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /opt/hadoop-2.4.1
> > Note:
> /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.java uses
> or overrides a deprecated API.
> > Note: Recompile with -Xlint:deprecation for details.
> > 14/08/25 10:13:02 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.jar
> > 14/08/25 10:13:03 INFO mapreduce.ExportJobBase: Beginning export of AGE
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> [jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> [jar:file:/opt/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> > 14/08/25 10:13:03 INFO Configuration.deprecation: mapred.jar is
> deprecated. Instead, use mapreduce.job.jar
> > 14/08/25 10:13:06 INFO manager.SqlManager: Executing SQL statement:
> SELECT t.* FROM AGE AS t WHERE 1=0
> > 14/08/25 10:13:06 INFO Configuration.deprecation:
> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.reduce.speculative
> > 14/08/25 10:13:06 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> > 14/08/25 10:13:06 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> > 14/08/25 10:13:06 INFO client.RMProxy: Connecting to ResourceManager at
> nn01/10.10.200.74:8032
> > 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to
> process : 1
> > 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to
> process : 1
> > 14/08/25 10:13:27 INFO mapreduce.JobSubmitter: number of splits:1
> > 14/08/25 10:13:28 INFO mapreduce.JobSubmitter: Submitting tokens for
> job: job_1408941947869_0001
> > 14/08/25 10:13:30 INFO impl.YarnClientImpl: Submitted application
> application_1408941947869_0001
> > 14/08/25 10:13:30 INFO mapreduce.Job: The url to track the job:
> http://nn01:8088/proxy/application_1408941947869_0001/
> > 14/08/25 10:13:30 INFO mapreduce.Job: Running job: job_1408941947869_0001
> > 14/08/25 10:13:44 INFO mapreduce.Job: Job job_1408941947869_0001 running
> in uber mode : false
> > 14/08/25 10:13:44 INFO mapreduce.Job: map 0% reduce 0%
> > 14/08/25 10:14:11 INFO mapreduce.Job: map 100% reduce 0%
> > 14/08/25 10:14:11 INFO mapreduce.Job: Task Id :
> attempt_1408941947869_0001_m_000000_0, Status : FAILED
> > Error: java.io.IOException: java.sql.SQLException:
> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> > at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
> > at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> > at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> > Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
> error at or near ","
> > at com.vertica.util.ServerErrorData.buildException(Unknown Source)
> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
> Source)
> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> > at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
> > at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown
> Source)
> > at
> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
> > at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
> > at
> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
> > at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
> > at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
> > ... 8 more
> > Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856)
> ERROR: Syntax error at or near ","
> > ... 20 more
> >
> > 14/08/25 10:14:12 INFO mapreduce.Job: map 0% reduce 0%
> > 14/08/25 10:14:25 INFO mapreduce.Job: Task Id :
> attempt_1408941947869_0001_m_000000_1, Status : FAILED
> > Error: java.io.IOException: java.sql.SQLException:
> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> > at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
> > at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> > at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> > Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax
> error at or near ","
> > at com.vertica.util.ServerErrorData.buildException(Unknown Source)
> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
> Source)
> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> > at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
> > at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown
> Source)
> > at
> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
> > at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
> > at
> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
> > at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
> > at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
> > ... 8 more
> > Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856)
> ERROR: Syntax error at or near ","
> > ... 20 more
> >
> > 14/08/25 10:14:34 INFO mapreduce.Job: Task Id :
> attempt_1408941947869_0001_m_000000_2, Status : FAILED
> > Error: java.io.IOException: java.sql.SQLSyntaxErrorException:
> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> > at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
> > at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:415)
> > at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> > Caused by: java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](4856)
> ERROR: Syntax error at or near ","
> > at com.vertica.util.ServerErrorData.buildException(Unknown Source)
> > at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown
> Source)
> > at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> > at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> > at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown
> Source)
> > at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown
> Source)
> > at
> com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
> > at
> com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
> > at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown
> Source)
> > at
> org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
> > at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
> > at
> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
> > ... 8 more
> > Caused by: com.vertica.support.exceptions.SyntaxErrorException:
> [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> > ... 21 more
> >
> > 14/08/25 10:14:43 INFO mapreduce.Job: map 100% reduce 0%
> > 14/08/25 10:14:44 INFO mapreduce.Job: Job job_1408941947869_0001 failed
> with state FAILED due to: Task failed task_1408941947869_0001_m_000000
> > Job failed as tasks failed. failedMaps:1 failedReduces:0
> >
> > 14/08/25 10:14:45 INFO mapreduce.Job: Counters: 9
> > Job Counters
> > Failed map tasks=4
> > Launched map tasks=4
> > Other local map tasks=3
> > Rack-local map tasks=1
> > Total time spent by all maps in occupied slots (ms)=53177
> > Total time spent by all reduces in occupied slots (ms)=0
> > Total time spent by all map tasks (ms)=53177
> > Total vcore-seconds taken by all map tasks=53177
> > Total megabyte-seconds taken by all map tasks=54453248
> > 14/08/25 10:14:45 WARN mapreduce.Counters: Group FileSystemCounters is
> deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
> > 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Transferred 0 bytes in
> 99.0548 seconds (0 bytes/sec)
> > 14/08/25 10:14:45 WARN mapreduce.Counters: Group
> org.apache.hadoop.mapred.Task$Counter is deprecated. Use
> org.apache.hadoop.mapreduce.TaskCounter instead
> > 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Exported 0 records.
> > 14/08/25 10:14:45 ERROR tool.ExportTool: Error during export: Export job
> failed!
> > Regards
> > Muthupandi.K
> >
> >
> >
> > <image002.png>
> >
>
>
Re: Exporting to non standard databases
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Muthu,
the given message is a “Warning” and not an “Error”. Sqoop is warning you that you are using —driver option that will force Sqoop to use Generic JDBC connector. That is fine in your case as Sqoop don’t have specialized connector for Vertica. You can always use --connect-manager parameter to get rid of the message as is the warning suggesting:
sqoop export --connect-manager org.apache.sqoop.manager.GenericJdbcManager ...
You are however hitting a different problem, my best guess that Vertica is not supporting multi value insert statements that Sqoop is generating by default. Try to disable this feature using property sqoop.export.records.per.statement:
sqoop export -Dsqoop.export.records.per.statement=1 —connect ...
Jarcec
On Aug 25, 2014, at 6:55 AM, Muthu Pandi <mu...@gmail.com> wrote:
> Hi every one,
> Is it possible to export data from HDFS to vetica.I successfully imported data from vertica to HDFS using sqoop.When i Export i get the following error.Please give me solution as soon as possible.this is the error which am getting,
>
> WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
>
> will above will have impact.How do i create connection manager for vertica.
>
> My sqoop query is
>
> sqoop export --driver com.vertica.jdbc.Driver --connect "jdbc:vertica://X.X.X.X:5433/dbname" --username username -P --table tablename -m 1 --export-dir /ver.txt --input-fields-terminated-by ',' --lines-terminated-by '\n'
>
>
> and am using hadoop2.4.1 and sqoop1.4.4 version
>
>
>
> 14/08/25 10:12:57 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
> 14/08/25 10:12:57 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
> 14/08/25 10:12:57 INFO manager.SqlManager: Using default fetchSize of 1000
> 14/08/25 10:12:57 INFO tool.CodeGenTool: Beginning code generation
> 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM AGE AS t WHERE 1=0
> 14/08/25 10:12:58 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM AGE AS t WHERE 1=0
> 14/08/25 10:12:58 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/hadoop-2.4.1
> Note: /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.java uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 14/08/25 10:13:02 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop2/compile/f0a438b0e6007fea6da95808e5f9468e/AGE.jar
> 14/08/25 10:13:03 INFO mapreduce.ExportJobBase: Beginning export of AGE
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/opt/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 14/08/25 10:13:03 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 14/08/25 10:13:06 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM AGE AS t WHERE 1=0
> 14/08/25 10:13:06 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
> 14/08/25 10:13:06 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 14/08/25 10:13:06 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 14/08/25 10:13:06 INFO client.RMProxy: Connecting to ResourceManager at nn01/10.10.200.74:8032
> 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to process : 1
> 14/08/25 10:13:27 INFO input.FileInputFormat: Total input paths to process : 1
> 14/08/25 10:13:27 INFO mapreduce.JobSubmitter: number of splits:1
> 14/08/25 10:13:28 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1408941947869_0001
> 14/08/25 10:13:30 INFO impl.YarnClientImpl: Submitted application application_1408941947869_0001
> 14/08/25 10:13:30 INFO mapreduce.Job: The url to track the job: http://nn01:8088/proxy/application_1408941947869_0001/
> 14/08/25 10:13:30 INFO mapreduce.Job: Running job: job_1408941947869_0001
> 14/08/25 10:13:44 INFO mapreduce.Job: Job job_1408941947869_0001 running in uber mode : false
> 14/08/25 10:13:44 INFO mapreduce.Job: map 0% reduce 0%
> 14/08/25 10:14:11 INFO mapreduce.Job: map 100% reduce 0%
> 14/08/25 10:14:11 INFO mapreduce.Job: Task Id : attempt_1408941947869_0001_m_000000_0, Status : FAILED
> Error: java.io.IOException: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
> at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> at com.vertica.util.ServerErrorData.buildException(Unknown Source)
> at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
> at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
> at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
> at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown Source)
> at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
> at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
> at org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
> at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
> at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
> ... 8 more
> Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> ... 20 more
>
> 14/08/25 10:14:12 INFO mapreduce.Job: map 0% reduce 0%
> 14/08/25 10:14:25 INFO mapreduce.Job: Task Id : attempt_1408941947869_0001_m_000000_1, Status : FAILED
> Error: java.io.IOException: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
> at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> Caused by: java.sql.SQLException: [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> at com.vertica.util.ServerErrorData.buildException(Unknown Source)
> at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
> at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
> at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> at com.vertica.jdbc.SPreparedStatement.<init>(Unknown Source)
> at com.vertica.jdbc.VerticaPreparedStatementImpl.<init>(Unknown Source)
> at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
> at com.vertica.jdbc.SConnection.prepareStatement(Unknown Source)
> at org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
> at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
> at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
> ... 8 more
> Caused by: com.vertica.util.ServerException: [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> ... 20 more
>
> 14/08/25 10:14:34 INFO mapreduce.Job: Task Id : attempt_1408941947869_0001_m_000000_2, Status : FAILED
> Error: java.io.IOException: java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
> at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:647)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
> Caused by: java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> at com.vertica.util.ServerErrorData.buildException(Unknown Source)
> at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source)
> at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source)
> at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> at com.vertica.dataengine.VDataEngine.prepare(Unknown Source)
> at com.vertica.jdbc.common.SPreparedStatement.<init>(Unknown Source)
> at com.vertica.jdbc.jdbc4.S4PreparedStatement.<init>(Unknown Source)
> at com.vertica.jdbc.VerticaJdbc4PreparedStatementImpl.<init>(Unknown Source)
> at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source)
> at com.vertica.jdbc.common.SConnection.prepareStatement(Unknown Source)
> at org.apache.sqoop.mapreduce.ExportOutputFormat$ExportRecordWriter.getPreparedStatement(ExportOutputFormat.java:141)
> at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
> at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
> ... 8 more
> Caused by: com.vertica.support.exceptions.SyntaxErrorException: [Vertica][VJDBC](4856) ERROR: Syntax error at or near ","
> ... 21 more
>
> 14/08/25 10:14:43 INFO mapreduce.Job: map 100% reduce 0%
> 14/08/25 10:14:44 INFO mapreduce.Job: Job job_1408941947869_0001 failed with state FAILED due to: Task failed task_1408941947869_0001_m_000000
> Job failed as tasks failed. failedMaps:1 failedReduces:0
>
> 14/08/25 10:14:45 INFO mapreduce.Job: Counters: 9
> Job Counters
> Failed map tasks=4
> Launched map tasks=4
> Other local map tasks=3
> Rack-local map tasks=1
> Total time spent by all maps in occupied slots (ms)=53177
> Total time spent by all reduces in occupied slots (ms)=0
> Total time spent by all map tasks (ms)=53177
> Total vcore-seconds taken by all map tasks=53177
> Total megabyte-seconds taken by all map tasks=54453248
> 14/08/25 10:14:45 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
> 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 99.0548 seconds (0 bytes/sec)
> 14/08/25 10:14:45 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
> 14/08/25 10:14:45 INFO mapreduce.ExportJobBase: Exported 0 records.
> 14/08/25 10:14:45 ERROR tool.ExportTool: Error during export: Export job failed!
> Regards
> Muthupandi.K
>
>
>
> <image002.png>
>