You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/09/03 04:32:30 UTC
[GitHub] [hudi] melin opened a new issue #3591: Failed to create a hudi table kerberos authentication
melin opened a new issue #3591:
URL: https://github.com/apache/hudi/issues/3591
Failed to create a hudi table kerberos authentication, No problem creating the orc table
```
21/09/03 12:28:34 INFO SparkExecuteStatementOperation: Submitting query 'create table bigdata.test_demo_test (
name string comment '',
age int comment ''
)
STORED AS orc
TBLPROPERTIES ('orc.compress'='ZSTD', 'fileFormat'='orc')
' with 0a331fa3-cb28-4c60-8ba8-aace33f0e301
21/09/03 12:28:34 INFO SparkExecuteStatementOperation: Running query with 0a331fa3-cb28-4c60-8ba8-aace33f0e301
21/09/03 12:28:35 INFO DAGScheduler: Asked to cancel job group 0a331fa3-cb28-4c60-8ba8-aace33f0e301
21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Close statement with 0a331fa3-cb28-4c60-8ba8-aace33f0e301
21/09/03 12:28:35 INFO ThriftCLIService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
21/09/03 12:28:35 INFO HiveMetaStore: 1: Shutting down the object store...
21/09/03 12:28:35 INFO audit: ugi=hive/hadoop-test-nn2-9-12@DZTECH.COM ip=unknown-ip-addr cmd=Shutting down the object store...
21/09/03 12:28:35 INFO HiveMetaStore: 1: Metastore shutdown complete.
21/09/03 12:28:35 INFO audit: ugi=hive/hadoop-test-nn2-9-12@DZTECH.COM ip=unknown-ip-addr cmd=Metastore shutdown complete.
21/09/03 12:28:35 INFO SessionState: Created local directory: /tmp/a4461f0d-84a0-4aa5-a967-24443027b3e9_resources
21/09/03 12:28:35 INFO SessionState: Created HDFS directory: /tmp/hive/admin/a4461f0d-84a0-4aa5-a967-24443027b3e9
21/09/03 12:28:35 INFO SessionState: Created local directory: /tmp/admin/a4461f0d-84a0-4aa5-a967-24443027b3e9
21/09/03 12:28:35 INFO SessionState: Created HDFS directory: /tmp/hive/admin/a4461f0d-84a0-4aa5-a967-24443027b3e9/_tmp_space.db
21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Submitting query 'show table extended like 'test_demo_test'' with c71ac5f9-e58e-4474-87f5-262452ba18bf
21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Running query with c71ac5f9-e58e-4474-87f5-262452ba18bf
21/09/03 12:28:35 INFO CodeGenerator: Code generated in 228.510447 ms
21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Result Schema: StructType(StructField(database,StringType,false), StructField(tableName,StringType,false), StructField(isTemporary,BooleanType,false), StructField(information,StringType,false))
21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Received getNextRowSet request order=FETCH_NEXT and maxRowsL=50 with c71ac5f9-e58e-4474-87f5-262452ba18bf
21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Returning result set with 1 rows from offsets [0, 1) with c71ac5f9-e58e-4474-87f5-262452ba18bf
21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Received getNextRowSet request order=FETCH_NEXT and maxRowsL=50 with c71ac5f9-e58e-4474-87f5-262452ba18bf
21/09/03 12:28:35 INFO DAGScheduler: Asked to cancel job group c71ac5f9-e58e-4474-87f5-262452ba18bf
21/09/03 12:28:35 INFO SparkExecuteStatementOperation: Close statement with c71ac5f9-e58e-4474-87f5-262452ba18bf
21/09/03 12:28:48 INFO ThriftCLIService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
21/09/03 12:28:48 INFO HiveMetaStore: 1: Shutting down the object store...
21/09/03 12:28:48 INFO audit: ugi=hive/hadoop-test-nn2-9-12@DZTECH.COM ip=unknown-ip-addr cmd=Shutting down the object store...
21/09/03 12:28:48 INFO HiveMetaStore: 1: Metastore shutdown complete.
21/09/03 12:28:48 INFO audit: ugi=hive/hadoop-test-nn2-9-12@DZTECH.COM ip=unknown-ip-addr cmd=Metastore shutdown complete.
21/09/03 12:28:49 INFO SessionState: Created local directory: /tmp/a58180d1-7865-493a-8be4-503266f0d773_resources
21/09/03 12:28:49 INFO SessionState: Created HDFS directory: /tmp/hive/admin/a58180d1-7865-493a-8be4-503266f0d773
21/09/03 12:28:49 INFO SessionState: Created local directory: /tmp/admin/a58180d1-7865-493a-8be4-503266f0d773
21/09/03 12:28:49 INFO SessionState: Created HDFS directory: /tmp/hive/admin/a58180d1-7865-493a-8be4-503266f0d773/_tmp_space.db
21/09/03 12:28:49 INFO SparkExecuteStatementOperation: Submitting query 'create table bigdata.test_hudi_dt (
id int comment '',
name string comment '',
price double comment '',
ds string comment ''
) USING hudi
OPTIONS (primaryKey = 'id', type = 'cow', 'compression'='ZSTD')
PARTITIONED BY (ds)
' with b10c84b2-d141-48f5-994d-4ed3b930e074
21/09/03 12:28:49 INFO SparkExecuteStatementOperation: Running query with b10c84b2-d141-48f5-994d-4ed3b930e074
21/09/03 12:28:49 INFO HiveUtils: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
21/09/03 12:28:49 INFO deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
21/09/03 12:28:49 INFO metastore: Trying to connect to metastore with URI thrift://hadoop-test-nn1-9-11:9083
21/09/03 12:28:49 ERROR TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:95)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:38)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
at org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:176)
at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:129)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:301)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:431)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:324)
at org.apache.spark.sql.hive.HiveClientUtils$.newClientForMetadata(HiveClientUtils.scala:27)
at org.apache.spark.sql.hudi.command.CreateHoodieTableCommand.createHiveDataSourceTable(CreateHoodieTableCommand.scala:212)
at org.apache.spark.sql.hudi.command.CreateHoodieTableCommand.createTableInCatalog(CreateHoodieTableCommand.scala:177)
at org.apache.spark.sql.hudi.command.CreateHoodieTableCommand.run(CreateHoodieTableCommand.scala:73)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
at org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:229)
at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3630)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:100)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:160)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:87)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:767)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3628)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:229)
at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:100)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:767)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:97)
at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:610)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:767)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:605)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:650)
at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:280)
at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.$anonfun$run$1(SparkExecuteStatementOperation.scala:216)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78)
at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62)
at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:46)
at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:216)
at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1$$anon$2.run(SparkExecuteStatementOperation.scala:211)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1.run(SparkExecuteStatementOperation.scala:227)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
... 75 more
21/09/03 12:28:49 WARN metastore: Failed to connect to the MetaStore Server...
21/09/03 12:28:49 INFO metastore: Waiting 1 seconds before next connection attempt.
21/09/03 12:28:50 INFO metastore: Trying to connect to metastore with URI thrift://hadoop-test-nn1-9-11:9083
21/09/03 12:28:50 ERROR TSaslTransport: SASL negotiation failure
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #3591: Failed to create a hudi table kerberos authentication
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #3591:
URL: https://github.com/apache/hudi/issues/3591#issuecomment-919723464
Closing this as we have a tracking jira and a PR. thanks for the contribution :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan closed issue #3591: Failed to create a hudi table kerberos authentication
Posted by GitBox <gi...@apache.org>.
nsivabalan closed issue #3591:
URL: https://github.com/apache/hudi/issues/3591
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org