You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "ighack (Jira)" <ji...@apache.org> on 2021/07/16 08:54:00 UTC
[jira] [Comment Edited] (ATLAS-4361) Hive Hook can't send
notification
[ https://issues.apache.org/jira/browse/ATLAS-4361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17381906#comment-17381906 ]
ighack edited comment on ATLAS-4361 at 7/16/21, 8:53 AM:
---------------------------------------------------------
and I modify [KafkaNotification.java|https://sourcegraph.com/github.com/apache/atlas/-/blob/notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java]
private KafkaProducer getOrCreateProducer(NotificationType notificationType) {
LOG.debug("==> KafkaNotification.getOrCreateProducer()");
KafkaProducer ret = producers.get(notificationType);
if (ret == null) {
synchronized (this) {
ret = producers.get(notificationType);
if (ret == null)
{ LOG.info("sasl.jaas.config is:",properties.getProperty("sasl.jaas.config")); LOG.info("security.protocol is:",properties.getProperty("security.protocol")); LOG.info("sasl.mechanism is:",properties.getProperty("sasl.mechanism")); LOG.info("bootstrap.servers is:",properties.getProperty("bootstrap.servers")); LOG.info("properties is:",properties.toString());
ret = new KafkaProducer(properties); producers.put(notificationType, ret); }
}
}
LOG.debug("<== KafkaNotification.getOrCreateProducer()");
return ret;
}
but in log I get null
!20210716165232274.png!
was (Author: ighack):
and I modify [KafkaNotification.java|https://sourcegraph.com/github.com/apache/atlas/-/blob/notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java]
private KafkaProducer getOrCreateProducer(NotificationType notificationType) {
LOG.debug("==> KafkaNotification.getOrCreateProducer()");
KafkaProducer ret = producers.get(notificationType);
if (ret == null) {
synchronized (this) {
ret = producers.get(notificationType);
if (ret == null) {
LOG.info("sasl.jaas.config is:",properties.getProperty("sasl.jaas.config"));
LOG.info("security.protocol is:",properties.getProperty("security.protocol"));
LOG.info("sasl.mechanism is:",properties.getProperty("sasl.mechanism"));
LOG.info("bootstrap.servers is:",properties.getProperty("bootstrap.servers"));
LOG.info("properties is:",properties.toString());
ret = new KafkaProducer(properties);
producers.put(notificationType, ret);
}
}
}
LOG.debug("<== KafkaNotification.getOrCreateProducer()");
return ret;
}
but in log I get null
!20210716165232274.png!
> Hive Hook can't send notification
> ---------------------------------
>
> Key: ATLAS-4361
> URL: https://issues.apache.org/jira/browse/ATLAS-4361
> Project: Atlas
> Issue Type: Bug
> Components: hive-integration
> Affects Versions: 2.1.0
> Reporter: ighack
> Priority: Blocker
> Attachments: 20210716165232274.png
>
>
> when I create a view
> I get a error
>
> Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user
>
> but I make sure keytab and principal is correct
> #Enable for Kerberized Kafka clusters
> atlas.notification.kafka.service.principal=jztwk/hadoop@JOIN.COM
> atlas.notification.kafka.keytab.location=/hadoop/app/jztwk.keytab
> atlas.kafka.security.protocol=SASL_PLAINTEXT
> atlas.kafka.sasl.kerberos.service.name=kafka
> ##############JAAS Configuration ########
> atlas.jaas.KafkaClient.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
> atlas.jaas.KafkaClient.loginModuleControlFlag=required
> atlas.jaas.KafkaClient.option.useKeyTab=true
> atlas.jaas.KafkaClient.option.storeKey=true
> atlas.jaas.KafkaClient.option.serviceName=kafka
> atlas.jaas.KafkaClient.option.keyTab=/hadoop/app/jztwk.keytab
> [atlas.jaas.KafkaClient.option.principal=jztwk/hadoop@JOIN.COM|mailto:atlas.jaas.KafkaClient.option.principal=jztwk/hadoop@JOIN.COM]
>
> and I modify [KafkaNotification.java|https://sourcegraph.com/github.com/apache/atlas/-/blob/notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java] add a debug
> LOG.debug("sasl.jaas.config is " + newJaasProperty);
>
> I see
> sasl.jaas.config is *com.sun.security.auth.module.Krb5LoginModule required keyTab="/hadoop/app/jztwk.keytab" storeKey=true useKeyTab=true principal="jztwk/hadoop@JOIN.COM" ;*
> in log
> I use this config to my application.
> that is OK.
>
> Properties props = new Properties();
> props.put("bootstrap.servers", "bigdser1:9092,bigdser2:9092,bigdser3:9092,bigdser4:9092,bigdser5:9092");
> props.put("sasl.jaas.config","com.sun.security.auth.module.Krb5LoginModule required keyTab=\"/hadoop/jztwk.keytab\" storeKey=true useKeyTab=true principal=\"jztwk/hadoop@JOIN.COM\" ;");
> props.put("security.protocol", "SASL_PLAINTEXT");
> props.put("sasl.kerberos.service.name", "kafka");
> props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
> props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
> KafkaProducer<String,String> producer = new KafkaProducer<>(props);
>
> and I can run import-hive.sh
> so, I don't understand why
>
> all node I have run *kinit -kt /hadoop/app/jztwk.keytab [jztwk/hadoop@JOIN.COM|mailto:jztwk/hadoop@JOIN.COM]*
> -rwxr-xr-x 1 jzyc jzyc 146 Jul 14 08:46 /hadoop/app/jztwk.keytab
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)