You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/09/03 04:32:30 UTC

[GitHub] [hudi] melin opened a new issue #3591: Failed to create a hudi table kerberos authentication

melin opened a new issue #3591:
URL: https://github.com/apache/hudi/issues/3591


   Failed to create a hudi table kerberos authentication, No problem creating the orc table
   
   ```
   21/09/03 12:28:34 INFO SparkExecuteStatementOperation: Submitting query 'create table bigdata.test_demo_test (
     name string comment '',
     age int comment ''
   )
   STORED AS orc
   TBLPROPERTIES ('orc.compress'='ZSTD', 'fileFormat'='orc')
   ' with 0a331fa3-cb28-4c60-8ba8-aace33f0e301
   21/09/03 12:28:34 INFO SparkExecuteStatementOperation: Running query with 0a331fa3-cb28-4c60-8ba8-aace33f0e301
   21/09/03 12:28:35 INFO DAGScheduler: Asked to cancel job group 0a331fa3-cb28-4c60-8ba8-aace33f0e301
   21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Close statement with 0a331fa3-cb28-4c60-8ba8-aace33f0e301
   21/09/03 12:28:35 INFO ThriftCLIService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
   21/09/03 12:28:35 INFO HiveMetaStore: 1: Shutting down the object store...
   21/09/03 12:28:35 INFO audit: ugi=hive/hadoop-test-nn2-9-12@DZTECH.COM  ip=unknown-ip-addr      cmd=Shutting down the object store...   
   21/09/03 12:28:35 INFO HiveMetaStore: 1: Metastore shutdown complete.
   21/09/03 12:28:35 INFO audit: ugi=hive/hadoop-test-nn2-9-12@DZTECH.COM  ip=unknown-ip-addr      cmd=Metastore shutdown complete.        
   21/09/03 12:28:35 INFO SessionState: Created local directory: /tmp/a4461f0d-84a0-4aa5-a967-24443027b3e9_resources
   21/09/03 12:28:35 INFO SessionState: Created HDFS directory: /tmp/hive/admin/a4461f0d-84a0-4aa5-a967-24443027b3e9
   21/09/03 12:28:35 INFO SessionState: Created local directory: /tmp/admin/a4461f0d-84a0-4aa5-a967-24443027b3e9
   21/09/03 12:28:35 INFO SessionState: Created HDFS directory: /tmp/hive/admin/a4461f0d-84a0-4aa5-a967-24443027b3e9/_tmp_space.db
   21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Submitting query 'show table extended like 'test_demo_test'' with c71ac5f9-e58e-4474-87f5-262452ba18bf
   21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Running query with c71ac5f9-e58e-4474-87f5-262452ba18bf
   21/09/03 12:28:35 INFO CodeGenerator: Code generated in 228.510447 ms
   21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Result Schema: StructType(StructField(database,StringType,false), StructField(tableName,StringType,false), StructField(isTemporary,BooleanType,false), StructField(information,StringType,false))
   21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Received getNextRowSet request order=FETCH_NEXT and maxRowsL=50 with c71ac5f9-e58e-4474-87f5-262452ba18bf
   21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Returning result set with 1 rows from offsets [0, 1) with c71ac5f9-e58e-4474-87f5-262452ba18bf
   21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Received getNextRowSet request order=FETCH_NEXT and maxRowsL=50 with c71ac5f9-e58e-4474-87f5-262452ba18bf
   21/09/03 12:28:35 INFO DAGScheduler: Asked to cancel job group c71ac5f9-e58e-4474-87f5-262452ba18bf
   21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Close statement with c71ac5f9-e58e-4474-87f5-262452ba18bf
   
   
   
   
   
   
   21/09/03 12:28:48 INFO ThriftCLIService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
   21/09/03 12:28:48 INFO HiveMetaStore: 1: Shutting down the object store...
   21/09/03 12:28:48 INFO audit: ugi=hive/hadoop-test-nn2-9-12@DZTECH.COM  ip=unknown-ip-addr      cmd=Shutting down the object store...   
   21/09/03 12:28:48 INFO HiveMetaStore: 1: Metastore shutdown complete.
   21/09/03 12:28:48 INFO audit: ugi=hive/hadoop-test-nn2-9-12@DZTECH.COM  ip=unknown-ip-addr      cmd=Metastore shutdown complete.        
   21/09/03 12:28:49 INFO SessionState: Created local directory: /tmp/a58180d1-7865-493a-8be4-503266f0d773_resources
   21/09/03 12:28:49 INFO SessionState: Created HDFS directory: /tmp/hive/admin/a58180d1-7865-493a-8be4-503266f0d773
   21/09/03 12:28:49 INFO SessionState: Created local directory: /tmp/admin/a58180d1-7865-493a-8be4-503266f0d773
   21/09/03 12:28:49 INFO SessionState: Created HDFS directory: /tmp/hive/admin/a58180d1-7865-493a-8be4-503266f0d773/_tmp_space.db
   21/09/03 12:28:49 INFO SparkExecuteStatementOperation: Submitting query 'create table bigdata.test_hudi_dt (
     id int comment '',
     name string comment '',
     price double comment '',
     ds string comment ''
   ) USING hudi
   OPTIONS (primaryKey = 'id', type = 'cow', 'compression'='ZSTD')
   PARTITIONED BY (ds)
   ' with b10c84b2-d141-48f5-994d-4ed3b930e074
   21/09/03 12:28:49 INFO SparkExecuteStatementOperation: Running query with b10c84b2-d141-48f5-994d-4ed3b930e074
   21/09/03 12:28:49 INFO HiveUtils: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
   21/09/03 12:28:49 INFO deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
   21/09/03 12:28:49 INFO metastore: Trying to connect to metastore with URI thrift://hadoop-test-nn1-9-11:9083
   21/09/03 12:28:49 ERROR TSaslTransport: SASL negotiation failure
   javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
           at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
           at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:95)
           at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
           at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:38)
           at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
           at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
           at java.security.AccessController.doPrivileged(Native Method)
           at javax.security.auth.Subject.doAs(Subject.java:422)
           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
           at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
           at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)
           at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
           at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
           at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
           at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
           at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
           at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
           at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
           at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
           at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
           at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
           at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
           at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
           at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
           at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
           at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
           at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
           at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:176)
           at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:129)
           at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
           at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
           at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
           at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
           at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:301)
           at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:431)
           at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:324)
           at org.apache.spark.sql.hive.HiveClientUtils$.newClientForMetadata(HiveClientUtils.scala:27)
           at org.apache.spark.sql.hudi.command.CreateHoodieTableCommand.createHiveDataSourceTable(CreateHoodieTableCommand.scala:212)
           at org.apache.spark.sql.hudi.command.CreateHoodieTableCommand.createTableInCatalog(CreateHoodieTableCommand.scala:177)
           at org.apache.spark.sql.hudi.command.CreateHoodieTableCommand.run(CreateHoodieTableCommand.scala:73)
           at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
           at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
           at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
           at org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:229)
           at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3630)
           at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:100)
           at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:160)
           at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:87)
           at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:767)
           at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
           at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3628)
           at org.apache.spark.sql.Dataset.<init>(Dataset.scala:229)
           at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:100)
           at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:767)
           at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
           at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:610)
           at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:767)
           at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:605)
           at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:650)
           at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:280)
           at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.$anonfun$run$1(SparkExecuteStatementOperation.scala:216)
           at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
           at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78)
           at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62)
           at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:46)
           at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:216)
           at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:211)
           at java.security.AccessController.doPrivileged(Native Method)
           at javax.security.auth.Subject.doAs(Subject.java:422)
           at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
           at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:227)
           at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
           at java.util.concurrent.FutureTask.run(FutureTask.java:266)
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
           at java.lang.Thread.run(Thread.java:748)
   Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
           at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
           at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
           at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
           at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
           at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
           at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
           at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
           ... 75 more
   21/09/03 12:28:49 WARN metastore: Failed to connect to the MetaStore Server...
   21/09/03 12:28:49 INFO metastore: Waiting 1 seconds before next connection attempt.
   21/09/03 12:28:50 INFO metastore: Trying to connect to metastore with URI thrift://hadoop-test-nn1-9-11:9083
   21/09/03 12:28:50 ERROR TSaslTransport: SASL negotiation failure
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] nsivabalan commented on issue #3591: Failed to create a hudi table kerberos authentication

Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #3591:
URL: https://github.com/apache/hudi/issues/3591#issuecomment-919723464


   Closing this as we have a tracking jira and a PR. thanks for the contribution :) 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [hudi] nsivabalan closed issue #3591: Failed to create a hudi table kerberos authentication

Posted by GitBox <gi...@apache.org>.
nsivabalan closed issue #3591:
URL: https://github.com/apache/hudi/issues/3591


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org