You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@metron.apache.org by ni...@apache.org on 2018/08/31 19:20:03 UTC
[07/50] [abbrv] metron git commit: METRON-1701 Update General notes
on the installation of Pycapa on Kerberized cluster (MohanDV via nickwallen)
closes apache/metron#1136
METRON-1701 Update General notes on the installation of Pycapa on Kerberized cluster (MohanDV via nickwallen) closes apache/metron#1136
Project: http://git-wip-us.apache.org/repos/asf/metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/1b2bdff5
Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/1b2bdff5
Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/1b2bdff5
Branch: refs/remotes/apache/feature/METRON-1699-create-batch-profiler
Commit: 1b2bdff53106694a9223ecc97f7ce9d0a78b4b09
Parents: a1e25a4
Author: MohanDV <mo...@gmail.com>
Authored: Wed Aug 1 12:13:19 2018 -0400
Committer: nickallen <ni...@apache.org>
Committed: Wed Aug 1 12:13:19 2018 -0400
----------------------------------------------------------------------
metron-sensors/pycapa/README.md | 47 ++++++++++++++++++++++--------------
1 file changed, 29 insertions(+), 18 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/metron/blob/1b2bdff5/metron-sensors/pycapa/README.md
----------------------------------------------------------------------
diff --git a/metron-sensors/pycapa/README.md b/metron-sensors/pycapa/README.md
index 7e688a8..9f01d33 100644
--- a/metron-sensors/pycapa/README.md
+++ b/metron-sensors/pycapa/README.md
@@ -34,9 +34,9 @@ Pycapa performs network packet capture, both off-the-wire and from a Kafka topic
Installation
============
-General notes on the installation of Pycapa.
+General notes on the installation of Pycapa.
* Python 2.7 is required.
-* The following package dependencies are required and can be installed automatically with `pip`.
+* The following package dependencies are required and can be installed automatically with `pip`. The requirements are installed as part of step 4
* [confluent-kafka-python](https://github.com/confluentinc/confluent-kafka-python)
* [pcapy](https://github.com/CoreSecurity/pcapy)
* These instructions can be used directly on CentOS 7+.
@@ -54,8 +54,8 @@ General notes on the installation of Pycapa.
```
export PREFIX=/usr
- wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz -O - | tar -xz
- cd librdkafka-0.9.4/
+ wget https://github.com/edenhill/librdkafka/archive/v0.11.5.tar.gz -O - | tar -xz
+ cd librdkafka-0.11.5/
./configure --prefix=$PREFIX
make
make install
@@ -231,18 +231,24 @@ Capturing on 'Standard input'
### Kerberos
-The probe can be used in a Kerberized environment. Follow these additional steps to use Pycapa with Kerberos. The following assumptions have been made. These may need altered to fit your environment.
+The probe can be used in a Kerberized environment. The Python client README (https://github.com/confluentinc/confluent-kafka-python) has an important note for Kerberos case that the pre-built Linux wheels do NOT contain SASL Kerberos support. You will need to use the non-binary wheel to install confluent-kafka-python and build/install librdkafka separately. Follow these additional steps to use Pycapa with Kerberos. The following assumptions have been made. These may need altered to fit your environment.
* The Kafka broker is at `kafka1:6667`
* Zookeeper is at `zookeeper1:2181`
* The Kafka security protocol is `SASL_PLAINTEXT`
* The keytab used is located at `/etc/security/keytabs/metron.headless.keytab`
* The service principal is `metron@EXAMPLE.COM`
+
+
+1. If it is not, ensure that you have `libsasl` or `libsasl2` installed. On CentOS, this can be installed with the following command.
+ ```
+ yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi
+ ```
1. Build Librdkafka with SASL support (` --enable-sasl`) and install at your chosen $PREFIX.
```
- wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz -O - | tar -xz
- cd librdkafka-0.9.4/
+ wget https://github.com/edenhill/librdkafka/archive/v0.11.5.tar.gz -O - | tar -xz
+ cd librdkafka-0.11.5/
./configure --prefix=$PREFIX --enable-sasl
make
make install
@@ -250,14 +256,19 @@ The probe can be used in a Kerberized environment. Follow these additional step
1. Validate Librdkafka does indeed support SASL. Run the following command and ensure that `sasl` is returned as a built-in feature.
```
- $ examples/rdkafka_example -X builtin.features
- builtin.features = gzip,snappy,ssl,sasl,regex
- ```
-
- If it is not, ensure that you have `libsasl` or `libsasl2` installed. On CentOS, this can be installed with the following command.
- ```
- yum install -y cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi
+ $ examples/rdkafka_example -X builtin.features
+ builtin.features = gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins
```
+1. The source install of confluent-kafka.
+
+ If you have already installed, remove the binary wheel python client first, repeat until it says no longer installed
+ ```
+ pip uninstall -y confluent-kafka
+ ```
+
+ ```
+ pip install --no-binary :all: confluent-kafka
+ ```
1. Grant access to your Kafka topic. In this example the topic is simply named `pcap`.
```
@@ -279,8 +290,8 @@ The probe can be used in a Kerberized environment. Follow these additional step
* `security.protocol`
* `sasl.kerberos.keytab`
* `sasl.kerberos.principal`
-
- ```
+
+ ```
$ pycapa --producer \
--interface eth0 \
--kafka-broker kafka1:6667 \
@@ -292,8 +303,8 @@ The probe can be used in a Kerberized environment. Follow these additional step
INFO:root:Starting packet capture
INFO:root:Waiting for '1' message(s) to flush
INFO:root:'10' packet(s) in, '10' packet(s) out
- ```
-
+ ```
+
FAQs
====