Classpath issue: Custom authentication with sparkSQL/Spark 1.2

["arin.g" <ar...@yahoo.com>]

Hi,
I am trying to launch a spark 1.2 cluster with SparkSQL and custom
authentication.  After launching the cluster using the ec2 scripts, I copied
the following hive-site.xml file into spark/conf  dir:

/<configuration>
<property>
  <name>hive.server2.authentication</name>
  <value>CUSTOM</value>
</property>
<property>
  <name>HIVE_SERVER2_CUSTOM_AUTHENTICATION_CLASS</name>
  <value>mypackage.MySharkAuthenticator</value>
</property>
<property>
  <name>hive.server2.enable.doAs</name>
  <value>false</value>
</property>
</configuration>/

And I also copied my custom authentication jar file into spark/lib dir.  I
am getting the following error from sparkSql thrift server:

/*java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
mypackage.SharkAuthenticator not found*
        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1587)
        at
org.apache.hive.service.auth.CustomAuthenticationProviderImpl.<init>(CustomAuthenticationProviderImpl.java:34)
        at
org.apache.hive.service.auth.AuthenticationProviderFactory.getAuthenticationProvider(AuthenticationProviderFactory.java:57)
        at
org.apache.hive.service.auth.PlainSaslHelper$PlainServerCallbackHandler.handle(PlainSaslHelper.java:61)
        at
org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(PlainSaslServer.java:127)
        at
org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:509)
        at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:264)
        at
org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
        at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
        at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
*Caused by: java.lang.ClassNotFoundException: Class
mypackage.SharkAuthenticator not found*
        at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1493)
        at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1585)
        ... 12 more/

In Another attempt, I also copied the custom authentication jar file to
spark/conf directory (instead of spark/lib), but this is not working too. I
didn’t have this issue with Spark 1.1. Any thoughts? Are there any changes
in Spark 1.2  regarding the Classpath?

Thanks,
-Arin




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Classpath-issue-Custom-authentication-with-sparkSQL-Spark-1-2-tp19806.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org