You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Saikrishna Teja Bobba <sb...@progress.com> on 2017/07/06 21:04:12 UTC
Issue with Sqoop Import and Kerberos
Hi,
We are trying to import data from a SQL Server with Kerberos authentication using Sqoop and we are seeing the following error.
Caused by: java.sql.SQLException: [DataDirect][SQLServer JDBC Driver]A username was not specified and the driver could not establish a connection using Kerberos (type 4) integrated security: No valid credentials provided
at com.ddtek.jdbc.sqlserverbase.ddcw.b(Unknown Source)
at com.ddtek.jdbc.sqlserverbase.ddcw.a(Unknown Source)
at com.ddtek.jdbc.sqlserverbase.ddcv.b(Unknown Source)
at com.ddtek.jdbc.sqlserverbase.ddcv.a(Unknown Source)
at com.ddtek.jdbc.sqlserver.tds.ddc.<init>(Unknown Source)
at com.ddtek.jdbc.sqlserver.SQLServerImplConnection.f(Unknown Source)
at com.ddtek.jdbc.sqlserverbase.BaseConnection.b(Unknown Source)
at com.ddtek.jdbc.sqlserverbase.BaseConnection.k(Unknown Source)
at com.ddtek.jdbc.sqlserverbase.BaseConnection.b(Unknown Source)
at com.ddtek.jdbc.sqlserverbase.BaseConnection.a(Unknown Source)
at com.ddtek.jdbc.sqlserverbase.BaseDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:300)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
... 10 more
However, the similar with the list-tables works. Which means the Kerberos part works to some extent. We are looking to your team to assist or provide guidance. I can arrange a working session if that is more appropriate.
* This works with Authentication
sqoop list-tables --driver com.ddtek.jdbc.sqlserver.SQLServerDriver --connect "jdbc:Datadirect:Sqlserver://<server1>:1433;databaseName=srt_test;authenticationMethod=kerberos;encryptionMethod=SSL;validateServerCertificate=true;TrustStore=/etc/pki/tls/trust.jks;TruststorePassword=<pass>"
* This does not
sqoop import --driver com.ddtek.jdbc.sqlserver.SQLServerDriver --connect "jdbc:Datadirect:Sqlserver://<server1>:1433;databaseName=srt_test;authenticationMethod=kerberos;encryptionMethod=SSL;validateServerCertificate=true;TrustStore=/etc/pki/tls/trust.jks;TruststorePassword=<pass>" --table "dbo.QA_SQOOP_TEST_EXPORT" --target-dir "/tmp/dbo-export" -m 1
Can anyone please let me know what's going wrong here?
Thanks,
Saikrishna
Re: Issue with Sqoop Import and Kerberos
Posted by Szabolcs Vasas <va...@cloudera.com>.
Hi Saikrishna,
I have experimented with MSSQL Kerberos support recently and had a very
similar error. Can you confirm that the exception is thrown by the mapper?
The list-tables Sqoop tool creates the database connection on the same
machine the command is started on and I guess you have a keytab or a TGT on
that machine so the connection is built successfully.
However the import tool starts mappers on potentially different nodes of
the cluster and you probably do not have the keytab/TGT available on those
(or they are not accessible by the mapper).
A solution could be that Sqoop makes sure that the keytab/TGT will be
accessible by the mapper but this feature is not implemented at this point.
Regards,
Szabolcs
On Thu, Jul 6, 2017 at 11:04 PM, Saikrishna Teja Bobba <sb...@progress.com>
wrote:
> Hi,
>
>
>
> We are trying to import data from a SQL Server with Kerberos
> authentication using Sqoop and we are seeing the following error.
>
>
>
> *Caused by: java.sql.SQLException: [DataDirect][SQLServer JDBC Driver]A
> username was not specified and the driver could not establish a connection
> using Kerberos (type 4) integrated security: No valid credentials provided*
>
> * at com.ddtek.jdbc.sqlserverbase.ddcw.b(Unknown Source)*
>
> * at com.ddtek.jdbc.sqlserverbase.ddcw.a(Unknown Source)*
>
> * at com.ddtek.jdbc.sqlserverbase.ddcv.b(Unknown Source)*
>
> * at com.ddtek.jdbc.sqlserverbase.ddcv.a(Unknown Source)*
>
> * at com.ddtek.jdbc.sqlserver.tds.ddc.<init>(Unknown
> Source)*
>
> * at
> com.ddtek.jdbc.sqlserver.SQLServerImplConnection.f(Unknown Source)*
>
> * at com.ddtek.jdbc.sqlserverbase.BaseConnection.b(Unknown
> Source)*
>
> * at com.ddtek.jdbc.sqlserverbase.BaseConnection.k(Unknown
> Source)*
>
> * at com.ddtek.jdbc.sqlserverbase.BaseConnection.b(Unknown
> Source)*
>
> * at com.ddtek.jdbc.sqlserverbase.BaseConnection.a(Unknown
> Source)*
>
> * at
> com.ddtek.jdbc.sqlserverbase.BaseDriver.connect(Unknown Source)*
>
> * at
> java.sql.DriverManager.getConnection(DriverManager.java:664)*
>
> * at
> java.sql.DriverManager.getConnection(DriverManager.java:270)*
>
> * at
> org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:300)*
>
> * at
> org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)*
>
> * ... 10 more*
>
>
>
> *However, the similar with the list-tables works. Which means the Kerberos
> part works to some extent*. *We are looking to your team to assist or
> provide guidance. I can arrange a working session if that is more
> appropriate.*
>
>
>
>
>
> - *This works with Authentication*
>
> sqoop list-tables --driver com.ddtek.jdbc.sqlserver.SQLServerDriver
> --connect "jdbc:Datadirect:Sqlserver://<server1>:1433;databaseName=
> srt_test;authenticationMethod=kerberos;encryptionMethod=SSL;
> validateServerCertificate=true;TrustStore=/etc/pki/tls/
> trust.jks;TruststorePassword=<pass>"
>
>
>
>
>
> - *This does not *
>
> *sqoop import --driver com.ddtek.jdbc.sqlserver.SQLServerDriver --connect
> "jdbc:Datadirect:Sqlserver://<server1>:1433;databaseName=srt_test;authenticationMethod=kerberos;encryptionMethod=SSL;validateServerCertificate=true;TrustStore=/etc/pki/tls/trust.jks;TruststorePassword=<pass>"
> --table "dbo.QA_SQOOP_TEST_EXPORT" --target-dir "/tmp/dbo-export" -m 1*
>
>
>
> Can anyone please let me know what’s going wrong here?
>
>
>
>
>
> Thanks,
>
> Saikrishna
>
>
>
--
Szabolcs Vasas
Software Engineer
<http://www.cloudera.com>