You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/09/14 19:51:00 UTC
[jira] [Commented] (METRON-1187) Indexing/Profiler Kafka ACL Groups
Not Setup Correctly
[ https://issues.apache.org/jira/browse/METRON-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166880#comment-16166880 ]
ASF GitHub Bot commented on METRON-1187:
----------------------------------------
GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/759
METRON-1187 Indexing/Profiler Kafka ACL Groups Not Setup Correctly
The Profiler MPack mistakenly uses the wrong flag/guard file to indicate that the Kafka ACL group has been setup. Whichever component (either Profiler or Indexing) that is executed first will complete successfully. The component to run next will not perform this setup task because the duplicated flag/guard file indicates that the setup was already completed successfully.
Most of the changes in this PR enhance the existing logging to help in the future when tracking down an issue like this. The actual fix is a one line.
### Testing
To test this change, you need to kerberize and environment and then ensure that both the Profiler and Indexing topologies are successfully running and consuming data after kerberization.
- [ ] Tested in Full Dev
- [ ] Tested in multi-node cluster
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1187
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/759.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #759
----
commit 5d9dba4ef57f3052697df1f156fc5a48d5ee7577
Author: Nick Allen <ni...@nickallen.org>
Date: 2017-09-14T19:44:36Z
METRON-1187 Indexing/Profiler Kafka ACL Groups Not Setup Correctly
----
> Indexing/Profiler Kafka ACL Groups Not Setup Correctly
> ------------------------------------------------------
>
> Key: METRON-1187
> URL: https://issues.apache.org/jira/browse/METRON-1187
> Project: Metron
> Issue Type: Bug
> Affects Versions: 0.4.0
> Reporter: Nick Allen
> Assignee: Nick Allen
> Fix For: Next + 1
>
>
> When kerberizing Metron using the MPack, either the Profiler or the Indexing Kafka ACL groups will not authorize the 'metron' user. This will only work correctly for the component which is executed first.
> This can lead to an error in either the Profiler or Indexing topology that looks like the following.
> {code}
> 2017-09-14 07:29:52.984 o.a.s.util [ERROR] Async loop died!
> org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: indexing
> {code}
> Manually checking confirms the problem.
> {code}
> [root@XXX ambari-server]# /usr/hdp/current/kafka-broker/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=${ZOOKEEPER} --list
> [2017-09-14 12:09:26,284] WARN read null data from /kafka-acl-changes/acl_changes_0000000004 when processing notification acl_changes_0000000004 (kafka.common.ZkNodeChangeNotificationListener)
> [2017-09-14 12:09:26,304] WARN read null data from /kafka-acl-changes/acl_changes_0000000005 when processing notification acl_changes_0000000005 (kafka.common.ZkNodeChangeNotificationListener)
> [2017-09-14 12:09:26,315] WARN read null data from /kafka-acl-changes/acl_changes_0000000006 when processing notification acl_changes_0000000006 (kafka.common.ZkNodeChangeNotificationListener)
> [2017-09-14 12:09:26,321] WARN read null data from /kafka-acl-changes/acl_changes_0000000007 when processing notification acl_changes_0000000007 (kafka.common.ZkNodeChangeNotificationListener)
> [2017-09-14 12:09:26,323] WARN read null data from /kafka-acl-changes/acl_changes_0000000008 when processing notification acl_changes_0000000008 (kafka.common.ZkNodeChangeNotificationListener)
> [2017-09-14 12:09:26,325] WARN read null data from /kafka-acl-changes/acl_changes_0000000009 when processing notification acl_changes_0000000009 (kafka.common.ZkNodeChangeNotificationListener)
> [2017-09-14 12:09:26,327] WARN read null data from /kafka-acl-changes/acl_changes_0000000010 when processing notification acl_changes_0000000010 (kafka.common.ZkNodeChangeNotificationListener)
> [2017-09-14 12:09:26,337] WARN read null data from /kafka-acl-changes/acl_changes_0000000011 when processing notification acl_changes_0000000011 (kafka.common.ZkNodeChangeNotificationListener)
> Current ACLs for resource `Group:bro_parser`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Topic:ambari_kafka_service_check`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Group:profiler`:
> User:metron has Allow permission for operations: All from hosts: *
> [2017-09-14 12:09:26,349] WARN read null data from /kafka-acl-changes/acl_changes_0000000012 when processing notification acl_changes_0000000012 (kafka.common.ZkNodeChangeNotificationListener)
> Current ACLs for resource `Group:metron-rest`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Topic:enrichments`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Topic:snort`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Topic:yaf`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Group:enrichments`:
> User:metron has Allow permission for operations: All from hosts: *
> [2017-09-14 12:09:26,351] WARN read null data from /kafka-acl-changes/acl_changes_0000000013 when processing notification acl_changes_0000000013 (kafka.common.ZkNodeChangeNotificationListener)
> Current ACLs for resource `Topic:__consumer_offsets`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Topic:bro`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Topic:escalation`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Group:yaf_parser`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Group:snort_parser`:
> User:metron has Allow permission for operations: All from hosts: *
> Current ACLs for resource `Topic:indexing`:
> User:metron has Allow permission for operations: All from hosts: *
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)