You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by mmiklavc <gi...@git.apache.org> on 2016/08/19 21:46:43 UTC
[GitHub] incubator-metron pull request #218: Metron 383 Create Ambari Service Definit...
GitHub user mmiklavc opened a pull request:
https://github.com/apache/incubator-metron/pull/218
Metron 383 Create Ambari Service Definition for Metron Parsers
More detailed instructions for testing are coming soon, but I wanted to get community eyes on this asap.
This PR addresses [https://issues.apache.org/jira/browse/METRON-383](https://issues.apache.org/jira/browse/METRON-383)
To test this PR:
- You need to build the rpm's and make them available in a "/localrepo" directory on the node you plan to use as Master for the parsers.
- Copy or mount the services and stacks content into the correct ambari-server resources directories. For example
- incubator-metron/metron-deployment/packaging/ambari/src/main/resources/stacks/HDP/2.3/services/PARSERS -> /var/lib/ambari-server/resources/stacks/HDP/2.3/services/PARSERS
- incubator-metron/metron-deployment/packaging/ambari/src/main/resources/common-services/PARSERS -> /var/lib/ambari-server/resources/common-services/PARSERS
- Use Ambari 2.4
- Install on a node with a Kafka Broker and ZK client (service advisor recommendations to come in a separate PR)
- You can check both a full install as well as an install of HDFS, Kakfa, Zookeeper, and Storm followed after by Metron Parsers.
Cleanup/delete service
- delete the service via Ambari UI or REST API
Run the following commands from the master node, swapping out the zookeeper quorum as appropriate:
```
#!/bin/bash
# kafka
for topic in bro parser_error parser_invalid snort yaf
do
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh \
--zookeeper amb1.service.consul:2181,amb2.service.consul:2181,amb3.service.consul:2181 \
--delete \
--topic $topic
done
# storm kill
for i in bro yaf snort
do
storm kill $i
done
# yum delete
yum -y erase metron-parsers metron-common
# final file delete
rm -rf /usr/metron
# zookeeper
/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server 127.0.0.1:2181
# once connected to the client do this:
rmr /metron
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mmiklavc/incubator-metron METRON-383
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/218.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #218
----
commit d4b8ea8c148f1753932f799e3c553847cdb3b05c
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-05T13:23:12Z
sample
commit 0193f467f6b79a81479d30798525d981757d3146
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-09T22:14:10Z
add parser and common-services
commit 67096076717087126bdcf1e07d5e28d5b55a52e7
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-10T22:12:04Z
finish commands. refactor
commit 09dc18b9266702e90f4a7f02ea33edac31be0317
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-12T16:00:13Z
fixes
commit b447901c078c2b943608e2b5304fb87fb2aaabad
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-12T16:18:18Z
config fixes
commit 724070de4696059b98a358d585ff724b2f2226f9
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-16T14:10:27Z
incremental progress. quicklinks. kafka home
commit edd7b60c0565ef725858aa32e299c1b20d1f481b
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-16T21:01:46Z
incremental fixes
commit 955680b20c6684c30631257a1068791a52e92a0a
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-17T21:09:44Z
check service configured on start. fix zk load. fix topology start
commit 310550291051f7d649f8f4a51a80ab722758837c
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-18T20:21:33Z
service check
commit 2a308376fb57ceaf2c875c0f0fb754c372d74f6f
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-18T21:40:12Z
trying to fix service check
commit e9abe68bba0b1b95302c99a29f5fb872dd60f1c0
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-19T15:25:07Z
cleanup/refactor. fix service check ordering
commit 8c1a093807c2ee18a6c9f315510bce2c2d0baf69
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-19T16:07:02Z
fix service check
commit ffae457b34e94f5fdbf22a638c62bf69d032057a
Author: Michael Miklavcic <mi...@gmail.com>
Date: 2016-08-19T21:34:14Z
METRON-383 Ambari install for Metron Parser topologies.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #218: Metron-383 Create Ambari Service Definition for...
Posted by cestella <gi...@git.apache.org>.
Github user cestella commented on the issue:
https://github.com/apache/incubator-metron/pull/218
+1 pending master is merged in and checks pass
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #218: Metron-383 Create Ambari Service Definition for...
Posted by cestella <gi...@git.apache.org>.
Github user cestella commented on the issue:
https://github.com/apache/incubator-metron/pull/218
This looks good; I'll go +1 providing @dlyle65535 will +1 with me as I'm no expert in Ambari.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #218: Metron-383 Create Ambari Service Definit...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-metron/pull/218
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---