[jira] [Created] (SPARK-36765) Support MS Sql JDBC connectorJDBC with Keytab

["Dilip Thallam Sridhar (Jira)" <ji...@apache.org>]

Dilip Thallam Sridhar created SPARK-36765:
---------------------------------------------

             Summary: Support MS Sql JDBC connectorJDBC with Keytab
                 Key: SPARK-36765
                 URL: https://issues.apache.org/jira/browse/SPARK-36765
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.1.2
         Environment: Unix Redhat Environment
            Reporter: Dilip Thallam Sridhar
             Fix For: 3.1.2


Hi Team,

 

We are using the Spark-3.0.2 to connect to MS SqlServer with the following instruction  

Also tried with the Spark-3.1.2 Version,

 
1) download mssql-jdbc-9.4.0.jre8.jar
 2) Generated Keytab using kinit
 3) Validate Keytab using klist
 4) Run the spark job with principal and keytabs passed
 5) connection_url = "jdbc:sqlserver://{}:{};databaseName={};integratedSecurity=true;authenticationSchema=JavaKerberos"\
 .format(jdbc_host_name, jdbc_port, jdbc_database_name)

 Note: without integratedSecurity=true;authenticationSchema=JavaKerberos it looks for the usual username/password option to connect

 6) passing the following options during spark read.
 .option("principal", database_principal) \
 .option("files", database_keytab) \
 .option("keytab", database_keytab) \
 
tried with files and keytab, just files, and with all above 3 parameters
 
We are unable to connect to SqlServer from Spark and getting the following error shown below. 
 
A) Wanted to know if anybody was successful Spark to SqlServer? (as I see the previous Jira has been closed)
https://issues.apache.org/jira/browse/SPARK-12312
https://issues.apache.org/jira/browse/SPARK-31337
 
B) If yes, could you let us know if there are any additional configs needed for Spark to connect to SqlServer please?
Appreciate if we can get inputs to resolve this error.
 
 
Full Stack Trace

Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: This driver is not configured for integrated authentication.         at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1352)         at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:2329)         at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:1905)         at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:41)         at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:1893)         at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4575)         at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1400)         at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1045)         at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:817)         at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:700)         at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:842)         at org.apache.spark.sql.execution.datasources.jdbc.connection.BasicConnectionProvider.getConnection(BasicConnectionProvider.scala:49)         at org.apache.spark.sql.execution.datasources.jdbc.connection.SecureConnectionProvider.getConnection(SecureConnectionProvider.scala:44)         at org.apache.spark.sql.execution.datasources.jdbc.connection.MSSQLConnectionProvider.org$apache$spark$sql$execution$datasources$jdbc$connection$MSSQLConnectionProvider$$super$getConnection(MSSQLConnectionProvider.scala:69)         at org.apache.spark.sql.execution.datasources.jdbc.connection.MSSQLConnectionProvider$$anon$1.run(MSSQLConnectionProvider.scala:69)         at org.apache.spark.sql.execution.datasources.jdbc.connection.MSSQLConnectionProvider$$anon$1.run(MSSQLConnectionProvider.scala:67)         at java.base/java.security.AccessController.doPrivileged(Native Method)         at java.base/javax.security.auth.Subject.doAs(Subject.java:423)         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)         ... 23 more Caused by: java.lang.UnsatisfiedLinkError: no sqljdbc_auth in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]         at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2660)         at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:827)         at java.base/java.lang.System.loadLibrary(System.java:1871)         at com.microsoft.sqlserver.jdbc.AuthenticationJNI.<clinit>(AuthenticationJNI.java:32)         at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:1902)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org