You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Farrukh Naveed Anjum <an...@gmail.com> on 2019/12/05 15:30:53 UTC
Metron with Zeek not working.
Hi,
I am trying to use upgraded version of Bro that is Zeek. I am unable to
receive data into Kafka
@load packages/metron-bro-plugin-kafka/Apache/Kafka
redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG, SSL::LOG,
DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG, DNP3::LOG,
RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG, Software::LOG,
Intel::LOG, Notice::LOG, Signatures::LOG);
redef Kafka::send_all_active_logs = T;
redef Kafka::topic_name = "bro";
redef Kafka::tag_json = T;
redef Kafka::kafka_conf = table(
["metadata.broker.list"] = "localhost:6667",
["client.id"] = "bro"
);
I have 1 name node, 2 data nodes. Kafa does not seems to be recieving data
from either Zeek or Snort.
It keep sayings broker may not be avalable stuff. Any suggestion ?
--
*Best Regards*
Farrukh Naveed Anjum
*M:* +92 321 5083954 (WhatsApp Enabled)
*W:* https://www.farrukh.cc/
Re: Metron with Zeek not working.
Posted by Hema malini <nh...@gmail.com>.
Hi Farrukh,
You are welcome..
Thanks and Regards
Hema
On Fri, 6 Dec, 2019, 4:48 PM Farrukh Naveed Anjum, <an...@gmail.com>
wrote:
> Thanks for help Hema, It was port 6667 that i need to send data to. It is
> working fine with Zeek now.
>
>
>
> On Fri, Dec 6, 2019 at 1:14 PM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Hi,
>>
>> Please do find attached detailed configs
>>
>> On Fri, Dec 6, 2019 at 1:10 PM Farrukh Naveed Anjum <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Please do find the attached Settings I have I tried on 6667 but it does
>>> not work
>>>
>>> On Fri, Dec 6, 2019 at 12:22 PM Hema malini <nh...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please use the same listener port . I tried with 6067. Can u please
>>>> provide the settings you have used.
>>>>
>>>>
>>>> Thanks and Regards,
>>>> Hema
>>>> On Fri, 6 Dec, 2019, 11:48 AM Farrukh Naveed Anjum, <
>>>> anjum.farrukh@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Can you please suggest what port I should use. Metron Documentation
>>>>> says 6667 (For Kafka) I tried both 6667 and 9092 but did not receive data
>>>>> in kafka.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Dec 6, 2019 at 9:48 AM Hema malini <nh...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> This issue is basically due to Kafka configuration. Check your
>>>>>> listener port in Kafka configuration .Hope this helps.
>>>>>>
>>>>>> Thanks and Regards,
>>>>>> Hema
>>>>>>
>>>>>> On Fri, 6 Dec, 2019, 10:17 AM Hema malini, <nh...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Please check the listener property.Use the same port while consuming.
>>>>>>>
>>>>>>> Thanks and Regards,
>>>>>>> Hema
>>>>>>>
>>>>>>> On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <
>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Problem is i am not able to Push data into Kafka..
>>>>>>>>
>>>>>>>> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>>>>>>>>
>>>>>>>> Even with Tools following test fails
>>>>>>>>
>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create
>>>>>>>> --zookeeper namenodetest:2181 --replication-factor 1 --partitions 1 --topic
>>>>>>>> t1
>>>>>>>>
>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>>>>>>>> --broker-list namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>>>>>>>>
>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>>> --zookeeper namenodetest:2181 --topic t1
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I am unable to see the any message transmission between consumer /
>>>>>>>> producer.
>>>>>>>> It keep saying
>>>>>>>> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
>>>>>>>> Connection to node -2 could not be established. Broker may not be
>>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>>> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
>>>>>>>> Connection to node -1 could not be established. Broker may not be
>>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>>> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
>>>>>>>> Connection to node -3 could not be established. Broker may not be
>>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>>>
>>>>>>>>
>>>>>>>> Any suggesion what can I do ?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <
>>>>>>>> ottobackwards@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I don’t think we support newer versions of bro yet i.e. zeek.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>>>>>>>>> anjum.farrukh@gmail.com) wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I am trying to use upgraded version of Bro that is Zeek. I am
>>>>>>>>> unable to receive data into Kafka
>>>>>>>>>
>>>>>>>>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>>>>>>>>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG,
>>>>>>>>> SSL::LOG, DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG,
>>>>>>>>> DNP3::LOG, RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG,
>>>>>>>>> Software::LOG, Intel::LOG, Notice::LOG, Signatures::LOG);
>>>>>>>>> redef Kafka::send_all_active_logs = T;
>>>>>>>>> redef Kafka::topic_name = "bro";
>>>>>>>>> redef Kafka::tag_json = T;
>>>>>>>>> redef Kafka::kafka_conf = table(
>>>>>>>>> ["metadata.broker.list"] = "localhost:6667",
>>>>>>>>> ["client.id"] = "bro"
>>>>>>>>> );
>>>>>>>>>
>>>>>>>>> I have 1 name node, 2 data nodes. Kafa does not seems to be
>>>>>>>>> recieving data from either Zeek or Snort.
>>>>>>>>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Best Regards*
>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>>>>> *W:* https://www.farrukh.cc/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Best Regards*
>>>>>>>> Farrukh Naveed Anjum
>>>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>>>> *W:* https://www.farrukh.cc/
>>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> *Best Regards*
>>>>> Farrukh Naveed Anjum
>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>> *W:* https://www.farrukh.cc/
>>>>>
>>>>
>>>
>>> --
>>> *Best Regards*
>>> Farrukh Naveed Anjum
>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>> *W:* https://www.farrukh.cc/
>>>
>>
>>
>> --
>> *Best Regards*
>> Farrukh Naveed Anjum
>> *M:* +92 321 5083954 (WhatsApp Enabled)
>> *W:* https://www.farrukh.cc/
>>
>
>
> --
> *Best Regards*
> Farrukh Naveed Anjum
> *M:* +92 321 5083954 (WhatsApp Enabled)
> *W:* https://www.farrukh.cc/
>
Re: Metron with Zeek not working.
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Thanks for help Hema, It was port 6667 that i need to send data to. It is
working fine with Zeek now.
On Fri, Dec 6, 2019 at 1:14 PM Farrukh Naveed Anjum <an...@gmail.com>
wrote:
> Hi,
>
> Please do find attached detailed configs
>
> On Fri, Dec 6, 2019 at 1:10 PM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Hi,
>>
>> Please do find the attached Settings I have I tried on 6667 but it does
>> not work
>>
>> On Fri, Dec 6, 2019 at 12:22 PM Hema malini <nh...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Please use the same listener port . I tried with 6067. Can u please
>>> provide the settings you have used.
>>>
>>>
>>> Thanks and Regards,
>>> Hema
>>> On Fri, 6 Dec, 2019, 11:48 AM Farrukh Naveed Anjum, <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Can you please suggest what port I should use. Metron Documentation
>>>> says 6667 (For Kafka) I tried both 6667 and 9092 but did not receive data
>>>> in kafka.
>>>>
>>>>
>>>>
>>>> On Fri, Dec 6, 2019 at 9:48 AM Hema malini <nh...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> This issue is basically due to Kafka configuration. Check your
>>>>> listener port in Kafka configuration .Hope this helps.
>>>>>
>>>>> Thanks and Regards,
>>>>> Hema
>>>>>
>>>>> On Fri, 6 Dec, 2019, 10:17 AM Hema malini, <nh...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please check the listener property.Use the same port while consuming.
>>>>>>
>>>>>> Thanks and Regards,
>>>>>> Hema
>>>>>>
>>>>>> On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <
>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>
>>>>>>> Problem is i am not able to Push data into Kafka..
>>>>>>>
>>>>>>> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>>>>>>>
>>>>>>> Even with Tools following test fails
>>>>>>>
>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create
>>>>>>> --zookeeper namenodetest:2181 --replication-factor 1 --partitions 1 --topic
>>>>>>> t1
>>>>>>>
>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>>>>>>> --broker-list namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>>>>>>>
>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>> --zookeeper namenodetest:2181 --topic t1
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I am unable to see the any message transmission between consumer /
>>>>>>> producer.
>>>>>>> It keep saying
>>>>>>> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
>>>>>>> Connection to node -2 could not be established. Broker may not be
>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
>>>>>>> Connection to node -1 could not be established. Broker may not be
>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
>>>>>>> Connection to node -3 could not be established. Broker may not be
>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>>
>>>>>>>
>>>>>>> Any suggesion what can I do ?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <ot...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I don’t think we support newer versions of bro yet i.e. zeek.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>>>>>>>> anjum.farrukh@gmail.com) wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>> I am trying to use upgraded version of Bro that is Zeek. I am
>>>>>>>> unable to receive data into Kafka
>>>>>>>>
>>>>>>>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>>>>>>>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG,
>>>>>>>> SSL::LOG, DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG,
>>>>>>>> DNP3::LOG, RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG,
>>>>>>>> Software::LOG, Intel::LOG, Notice::LOG, Signatures::LOG);
>>>>>>>> redef Kafka::send_all_active_logs = T;
>>>>>>>> redef Kafka::topic_name = "bro";
>>>>>>>> redef Kafka::tag_json = T;
>>>>>>>> redef Kafka::kafka_conf = table(
>>>>>>>> ["metadata.broker.list"] = "localhost:6667",
>>>>>>>> ["client.id"] = "bro"
>>>>>>>> );
>>>>>>>>
>>>>>>>> I have 1 name node, 2 data nodes. Kafa does not seems to be
>>>>>>>> recieving data from either Zeek or Snort.
>>>>>>>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Best Regards*
>>>>>>>> Farrukh Naveed Anjum
>>>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>>>> *W:* https://www.farrukh.cc/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Best Regards*
>>>>>>> Farrukh Naveed Anjum
>>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>>> *W:* https://www.farrukh.cc/
>>>>>>>
>>>>>>
>>>>
>>>> --
>>>> *Best Regards*
>>>> Farrukh Naveed Anjum
>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>> *W:* https://www.farrukh.cc/
>>>>
>>>
>>
>> --
>> *Best Regards*
>> Farrukh Naveed Anjum
>> *M:* +92 321 5083954 (WhatsApp Enabled)
>> *W:* https://www.farrukh.cc/
>>
>
>
> --
> *Best Regards*
> Farrukh Naveed Anjum
> *M:* +92 321 5083954 (WhatsApp Enabled)
> *W:* https://www.farrukh.cc/
>
--
*Best Regards*
Farrukh Naveed Anjum
*M:* +92 321 5083954 (WhatsApp Enabled)
*W:* https://www.farrukh.cc/
Re: Metron with Zeek not working.
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Thanks it worked the same way
On Fri, Dec 6, 2019 at 4:20 PM Hema malini <nh...@gmail.com> wrote:
> Hi,
>
> In server.properties in each node ,u should find a property -
> listeners=PLAINTEXT://hostname:6667.
>
> Check that property in each node
>
> I used the below commands
>
> bin/kafka-console-producer.sh --broker-list ip1:6667,ip2:6667 --topic
> topicname
>
> bin/kafka-console-consumer.sh --bootstrap-server ip1:6667,ip2:6667 --topic
> topicname
> Hope this helps.
>
> Thanks and Regards,
> Hema
>
> On Fri, 6 Dec, 2019, 1:45 PM Farrukh Naveed Anjum, <
> anjum.farrukh@gmail.com> wrote:
>
>> Hi,
>>
>> Please do find attached detailed configs
>>
>> On Fri, Dec 6, 2019 at 1:10 PM Farrukh Naveed Anjum <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Please do find the attached Settings I have I tried on 6667 but it does
>>> not work
>>>
>>> On Fri, Dec 6, 2019 at 12:22 PM Hema malini <nh...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please use the same listener port . I tried with 6067. Can u please
>>>> provide the settings you have used.
>>>>
>>>>
>>>> Thanks and Regards,
>>>> Hema
>>>> On Fri, 6 Dec, 2019, 11:48 AM Farrukh Naveed Anjum, <
>>>> anjum.farrukh@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Can you please suggest what port I should use. Metron Documentation
>>>>> says 6667 (For Kafka) I tried both 6667 and 9092 but did not receive data
>>>>> in kafka.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Dec 6, 2019 at 9:48 AM Hema malini <nh...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> This issue is basically due to Kafka configuration. Check your
>>>>>> listener port in Kafka configuration .Hope this helps.
>>>>>>
>>>>>> Thanks and Regards,
>>>>>> Hema
>>>>>>
>>>>>> On Fri, 6 Dec, 2019, 10:17 AM Hema malini, <nh...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Please check the listener property.Use the same port while consuming.
>>>>>>>
>>>>>>> Thanks and Regards,
>>>>>>> Hema
>>>>>>>
>>>>>>> On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <
>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Problem is i am not able to Push data into Kafka..
>>>>>>>>
>>>>>>>> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>>>>>>>>
>>>>>>>> Even with Tools following test fails
>>>>>>>>
>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create
>>>>>>>> --zookeeper namenodetest:2181 --replication-factor 1 --partitions 1 --topic
>>>>>>>> t1
>>>>>>>>
>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>>>>>>>> --broker-list namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>>>>>>>>
>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>>> --zookeeper namenodetest:2181 --topic t1
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I am unable to see the any message transmission between consumer /
>>>>>>>> producer.
>>>>>>>> It keep saying
>>>>>>>> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
>>>>>>>> Connection to node -2 could not be established. Broker may not be
>>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>>> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
>>>>>>>> Connection to node -1 could not be established. Broker may not be
>>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>>> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
>>>>>>>> Connection to node -3 could not be established. Broker may not be
>>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>>>
>>>>>>>>
>>>>>>>> Any suggesion what can I do ?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <
>>>>>>>> ottobackwards@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I don’t think we support newer versions of bro yet i.e. zeek.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>>>>>>>>> anjum.farrukh@gmail.com) wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I am trying to use upgraded version of Bro that is Zeek. I am
>>>>>>>>> unable to receive data into Kafka
>>>>>>>>>
>>>>>>>>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>>>>>>>>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG,
>>>>>>>>> SSL::LOG, DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG,
>>>>>>>>> DNP3::LOG, RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG,
>>>>>>>>> Software::LOG, Intel::LOG, Notice::LOG, Signatures::LOG);
>>>>>>>>> redef Kafka::send_all_active_logs = T;
>>>>>>>>> redef Kafka::topic_name = "bro";
>>>>>>>>> redef Kafka::tag_json = T;
>>>>>>>>> redef Kafka::kafka_conf = table(
>>>>>>>>> ["metadata.broker.list"] = "localhost:6667",
>>>>>>>>> ["client.id"] = "bro"
>>>>>>>>> );
>>>>>>>>>
>>>>>>>>> I have 1 name node, 2 data nodes. Kafa does not seems to be
>>>>>>>>> recieving data from either Zeek or Snort.
>>>>>>>>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> *Best Regards*
>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>>>>> *W:* https://www.farrukh.cc/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Best Regards*
>>>>>>>> Farrukh Naveed Anjum
>>>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>>>> *W:* https://www.farrukh.cc/
>>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> *Best Regards*
>>>>> Farrukh Naveed Anjum
>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>> *W:* https://www.farrukh.cc/
>>>>>
>>>>
>>>
>>> --
>>> *Best Regards*
>>> Farrukh Naveed Anjum
>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>> *W:* https://www.farrukh.cc/
>>>
>>
>>
>> --
>> *Best Regards*
>> Farrukh Naveed Anjum
>> *M:* +92 321 5083954 (WhatsApp Enabled)
>> *W:* https://www.farrukh.cc/
>>
>
--
*Best Regards*
Farrukh Naveed Anjum
*M:* +92 321 5083954 (WhatsApp Enabled)
*W:* https://www.farrukh.cc/
Re: Metron with Zeek not working.
Posted by Hema malini <nh...@gmail.com>.
Hi,
In server.properties in each node ,u should find a property -
listeners=PLAINTEXT://hostname:6667.
Check that property in each node
I used the below commands
bin/kafka-console-producer.sh --broker-list ip1:6667,ip2:6667 --topic
topicname
bin/kafka-console-consumer.sh --bootstrap-server ip1:6667,ip2:6667 --topic
topicname
Hope this helps.
Thanks and Regards,
Hema
On Fri, 6 Dec, 2019, 1:45 PM Farrukh Naveed Anjum, <an...@gmail.com>
wrote:
> Hi,
>
> Please do find attached detailed configs
>
> On Fri, Dec 6, 2019 at 1:10 PM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Hi,
>>
>> Please do find the attached Settings I have I tried on 6667 but it does
>> not work
>>
>> On Fri, Dec 6, 2019 at 12:22 PM Hema malini <nh...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Please use the same listener port . I tried with 6067. Can u please
>>> provide the settings you have used.
>>>
>>>
>>> Thanks and Regards,
>>> Hema
>>> On Fri, 6 Dec, 2019, 11:48 AM Farrukh Naveed Anjum, <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Can you please suggest what port I should use. Metron Documentation
>>>> says 6667 (For Kafka) I tried both 6667 and 9092 but did not receive data
>>>> in kafka.
>>>>
>>>>
>>>>
>>>> On Fri, Dec 6, 2019 at 9:48 AM Hema malini <nh...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> This issue is basically due to Kafka configuration. Check your
>>>>> listener port in Kafka configuration .Hope this helps.
>>>>>
>>>>> Thanks and Regards,
>>>>> Hema
>>>>>
>>>>> On Fri, 6 Dec, 2019, 10:17 AM Hema malini, <nh...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please check the listener property.Use the same port while consuming.
>>>>>>
>>>>>> Thanks and Regards,
>>>>>> Hema
>>>>>>
>>>>>> On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <
>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>
>>>>>>> Problem is i am not able to Push data into Kafka..
>>>>>>>
>>>>>>> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>>>>>>>
>>>>>>> Even with Tools following test fails
>>>>>>>
>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create
>>>>>>> --zookeeper namenodetest:2181 --replication-factor 1 --partitions 1 --topic
>>>>>>> t1
>>>>>>>
>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>>>>>>> --broker-list namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>>>>>>>
>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>> --zookeeper namenodetest:2181 --topic t1
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I am unable to see the any message transmission between consumer /
>>>>>>> producer.
>>>>>>> It keep saying
>>>>>>> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
>>>>>>> Connection to node -2 could not be established. Broker may not be
>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
>>>>>>> Connection to node -1 could not be established. Broker may not be
>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
>>>>>>> Connection to node -3 could not be established. Broker may not be
>>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>>
>>>>>>>
>>>>>>> Any suggesion what can I do ?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <ot...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I don’t think we support newer versions of bro yet i.e. zeek.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>>>>>>>> anjum.farrukh@gmail.com) wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>> I am trying to use upgraded version of Bro that is Zeek. I am
>>>>>>>> unable to receive data into Kafka
>>>>>>>>
>>>>>>>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>>>>>>>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG,
>>>>>>>> SSL::LOG, DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG,
>>>>>>>> DNP3::LOG, RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG,
>>>>>>>> Software::LOG, Intel::LOG, Notice::LOG, Signatures::LOG);
>>>>>>>> redef Kafka::send_all_active_logs = T;
>>>>>>>> redef Kafka::topic_name = "bro";
>>>>>>>> redef Kafka::tag_json = T;
>>>>>>>> redef Kafka::kafka_conf = table(
>>>>>>>> ["metadata.broker.list"] = "localhost:6667",
>>>>>>>> ["client.id"] = "bro"
>>>>>>>> );
>>>>>>>>
>>>>>>>> I have 1 name node, 2 data nodes. Kafa does not seems to be
>>>>>>>> recieving data from either Zeek or Snort.
>>>>>>>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Best Regards*
>>>>>>>> Farrukh Naveed Anjum
>>>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>>>> *W:* https://www.farrukh.cc/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Best Regards*
>>>>>>> Farrukh Naveed Anjum
>>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>>> *W:* https://www.farrukh.cc/
>>>>>>>
>>>>>>
>>>>
>>>> --
>>>> *Best Regards*
>>>> Farrukh Naveed Anjum
>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>> *W:* https://www.farrukh.cc/
>>>>
>>>
>>
>> --
>> *Best Regards*
>> Farrukh Naveed Anjum
>> *M:* +92 321 5083954 (WhatsApp Enabled)
>> *W:* https://www.farrukh.cc/
>>
>
>
> --
> *Best Regards*
> Farrukh Naveed Anjum
> *M:* +92 321 5083954 (WhatsApp Enabled)
> *W:* https://www.farrukh.cc/
>
Re: Metron with Zeek not working.
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Hi,
Please do find attached detailed configs
On Fri, Dec 6, 2019 at 1:10 PM Farrukh Naveed Anjum <an...@gmail.com>
wrote:
> Hi,
>
> Please do find the attached Settings I have I tried on 6667 but it does
> not work
>
> On Fri, Dec 6, 2019 at 12:22 PM Hema malini <nh...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Please use the same listener port . I tried with 6067. Can u please
>> provide the settings you have used.
>>
>>
>> Thanks and Regards,
>> Hema
>> On Fri, 6 Dec, 2019, 11:48 AM Farrukh Naveed Anjum, <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Can you please suggest what port I should use. Metron Documentation says
>>> 6667 (For Kafka) I tried both 6667 and 9092 but did not receive data in
>>> kafka.
>>>
>>>
>>>
>>> On Fri, Dec 6, 2019 at 9:48 AM Hema malini <nh...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> This issue is basically due to Kafka configuration. Check your listener
>>>> port in Kafka configuration .Hope this helps.
>>>>
>>>> Thanks and Regards,
>>>> Hema
>>>>
>>>> On Fri, 6 Dec, 2019, 10:17 AM Hema malini, <nh...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Please check the listener property.Use the same port while consuming.
>>>>>
>>>>> Thanks and Regards,
>>>>> Hema
>>>>>
>>>>> On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <
>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>
>>>>>> Problem is i am not able to Push data into Kafka..
>>>>>>
>>>>>> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>>>>>>
>>>>>> Even with Tools following test fails
>>>>>>
>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create
>>>>>> --zookeeper namenodetest:2181 --replication-factor 1 --partitions 1 --topic
>>>>>> t1
>>>>>>
>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>>>>>> --broker-list namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>>>>>>
>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>> --zookeeper namenodetest:2181 --topic t1
>>>>>>
>>>>>>
>>>>>>
>>>>>> I am unable to see the any message transmission between consumer /
>>>>>> producer.
>>>>>> It keep saying
>>>>>> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
>>>>>> Connection to node -2 could not be established. Broker may not be
>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
>>>>>> Connection to node -1 could not be established. Broker may not be
>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
>>>>>> Connection to node -3 could not be established. Broker may not be
>>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>>
>>>>>>
>>>>>> Any suggesion what can I do ?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <ot...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I don’t think we support newer versions of bro yet i.e. zeek.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>>>>>>> anjum.farrukh@gmail.com) wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>> I am trying to use upgraded version of Bro that is Zeek. I am unable
>>>>>>> to receive data into Kafka
>>>>>>>
>>>>>>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>>>>>>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG,
>>>>>>> SSL::LOG, DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG,
>>>>>>> DNP3::LOG, RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG,
>>>>>>> Software::LOG, Intel::LOG, Notice::LOG, Signatures::LOG);
>>>>>>> redef Kafka::send_all_active_logs = T;
>>>>>>> redef Kafka::topic_name = "bro";
>>>>>>> redef Kafka::tag_json = T;
>>>>>>> redef Kafka::kafka_conf = table(
>>>>>>> ["metadata.broker.list"] = "localhost:6667",
>>>>>>> ["client.id"] = "bro"
>>>>>>> );
>>>>>>>
>>>>>>> I have 1 name node, 2 data nodes. Kafa does not seems to be
>>>>>>> recieving data from either Zeek or Snort.
>>>>>>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>>>>>>
>>>>>>> --
>>>>>>> *Best Regards*
>>>>>>> Farrukh Naveed Anjum
>>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>>> *W:* https://www.farrukh.cc/
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Best Regards*
>>>>>> Farrukh Naveed Anjum
>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>> *W:* https://www.farrukh.cc/
>>>>>>
>>>>>
>>>
>>> --
>>> *Best Regards*
>>> Farrukh Naveed Anjum
>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>> *W:* https://www.farrukh.cc/
>>>
>>
>
> --
> *Best Regards*
> Farrukh Naveed Anjum
> *M:* +92 321 5083954 (WhatsApp Enabled)
> *W:* https://www.farrukh.cc/
>
--
*Best Regards*
Farrukh Naveed Anjum
*M:* +92 321 5083954 (WhatsApp Enabled)
*W:* https://www.farrukh.cc/
Re: Metron with Zeek not working.
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Hi,
Please do find the attached Settings I have I tried on 6667 but it does not
work
On Fri, Dec 6, 2019 at 12:22 PM Hema malini <nh...@gmail.com> wrote:
> Hi,
>
> Please use the same listener port . I tried with 6067. Can u please
> provide the settings you have used.
>
>
> Thanks and Regards,
> Hema
> On Fri, 6 Dec, 2019, 11:48 AM Farrukh Naveed Anjum, <
> anjum.farrukh@gmail.com> wrote:
>
>> Hi,
>>
>> Can you please suggest what port I should use. Metron Documentation says
>> 6667 (For Kafka) I tried both 6667 and 9092 but did not receive data in
>> kafka.
>>
>>
>>
>> On Fri, Dec 6, 2019 at 9:48 AM Hema malini <nh...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> This issue is basically due to Kafka configuration. Check your listener
>>> port in Kafka configuration .Hope this helps.
>>>
>>> Thanks and Regards,
>>> Hema
>>>
>>> On Fri, 6 Dec, 2019, 10:17 AM Hema malini, <nh...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Please check the listener property.Use the same port while consuming.
>>>>
>>>> Thanks and Regards,
>>>> Hema
>>>>
>>>> On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <
>>>> anjum.farrukh@gmail.com> wrote:
>>>>
>>>>> Problem is i am not able to Push data into Kafka..
>>>>>
>>>>> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>>>>>
>>>>> Even with Tools following test fails
>>>>>
>>>>> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create --zookeeper
>>>>> namenodetest:2181 --replication-factor 1 --partitions 1 --topic t1
>>>>>
>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>>>>> --broker-list namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>>>>>
>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>> --zookeeper namenodetest:2181 --topic t1
>>>>>
>>>>>
>>>>>
>>>>> I am unable to see the any message transmission between consumer /
>>>>> producer.
>>>>> It keep saying
>>>>> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
>>>>> Connection to node -2 could not be established. Broker may not be
>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
>>>>> Connection to node -1 could not be established. Broker may not be
>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
>>>>> Connection to node -3 could not be established. Broker may not be
>>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>>
>>>>>
>>>>> Any suggesion what can I do ?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <ot...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I don’t think we support newer versions of bro yet i.e. zeek.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>>>>>> anjum.farrukh@gmail.com) wrote:
>>>>>>
>>>>>> Hi,
>>>>>> I am trying to use upgraded version of Bro that is Zeek. I am unable
>>>>>> to receive data into Kafka
>>>>>>
>>>>>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>>>>>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG,
>>>>>> SSL::LOG, DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG,
>>>>>> DNP3::LOG, RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG,
>>>>>> Software::LOG, Intel::LOG, Notice::LOG, Signatures::LOG);
>>>>>> redef Kafka::send_all_active_logs = T;
>>>>>> redef Kafka::topic_name = "bro";
>>>>>> redef Kafka::tag_json = T;
>>>>>> redef Kafka::kafka_conf = table(
>>>>>> ["metadata.broker.list"] = "localhost:6667",
>>>>>> ["client.id"] = "bro"
>>>>>> );
>>>>>>
>>>>>> I have 1 name node, 2 data nodes. Kafa does not seems to be recieving
>>>>>> data from either Zeek or Snort.
>>>>>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>>>>>
>>>>>> --
>>>>>> *Best Regards*
>>>>>> Farrukh Naveed Anjum
>>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>>> *W:* https://www.farrukh.cc/
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> *Best Regards*
>>>>> Farrukh Naveed Anjum
>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>> *W:* https://www.farrukh.cc/
>>>>>
>>>>
>>
>> --
>> *Best Regards*
>> Farrukh Naveed Anjum
>> *M:* +92 321 5083954 (WhatsApp Enabled)
>> *W:* https://www.farrukh.cc/
>>
>
--
*Best Regards*
Farrukh Naveed Anjum
*M:* +92 321 5083954 (WhatsApp Enabled)
*W:* https://www.farrukh.cc/
Re: Metron with Zeek not working.
Posted by Hema malini <nh...@gmail.com>.
Hi,
Please use the same listener port . I tried with 6067. Can u please provide
the settings you have used.
Thanks and Regards,
Hema
On Fri, 6 Dec, 2019, 11:48 AM Farrukh Naveed Anjum, <an...@gmail.com>
wrote:
> Hi,
>
> Can you please suggest what port I should use. Metron Documentation says
> 6667 (For Kafka) I tried both 6667 and 9092 but did not receive data in
> kafka.
>
>
>
> On Fri, Dec 6, 2019 at 9:48 AM Hema malini <nh...@gmail.com>
> wrote:
>
>> Hi,
>>
>> This issue is basically due to Kafka configuration. Check your listener
>> port in Kafka configuration .Hope this helps.
>>
>> Thanks and Regards,
>> Hema
>>
>> On Fri, 6 Dec, 2019, 10:17 AM Hema malini, <nh...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Please check the listener property.Use the same port while consuming.
>>>
>>> Thanks and Regards,
>>> Hema
>>>
>>> On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> Problem is i am not able to Push data into Kafka..
>>>>
>>>> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>>>>
>>>> Even with Tools following test fails
>>>>
>>>> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create --zookeeper
>>>> namenodetest:2181 --replication-factor 1 --partitions 1 --topic t1
>>>>
>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>>>> --broker-list namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>>>>
>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
>>>> namenodetest:2181 --topic t1
>>>>
>>>>
>>>>
>>>> I am unable to see the any message transmission between consumer /
>>>> producer.
>>>> It keep saying
>>>> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
>>>> Connection to node -2 could not be established. Broker may not be
>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
>>>> Connection to node -1 could not be established. Broker may not be
>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
>>>> Connection to node -3 could not be established. Broker may not be
>>>> available. (org.apache.kafka.clients.NetworkClient)
>>>>
>>>>
>>>> Any suggesion what can I do ?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <ot...@gmail.com>
>>>> wrote:
>>>>
>>>>> I don’t think we support newer versions of bro yet i.e. zeek.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>>>>> anjum.farrukh@gmail.com) wrote:
>>>>>
>>>>> Hi,
>>>>> I am trying to use upgraded version of Bro that is Zeek. I am unable
>>>>> to receive data into Kafka
>>>>>
>>>>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>>>>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG,
>>>>> SSL::LOG, DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG,
>>>>> DNP3::LOG, RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG,
>>>>> Software::LOG, Intel::LOG, Notice::LOG, Signatures::LOG);
>>>>> redef Kafka::send_all_active_logs = T;
>>>>> redef Kafka::topic_name = "bro";
>>>>> redef Kafka::tag_json = T;
>>>>> redef Kafka::kafka_conf = table(
>>>>> ["metadata.broker.list"] = "localhost:6667",
>>>>> ["client.id"] = "bro"
>>>>> );
>>>>>
>>>>> I have 1 name node, 2 data nodes. Kafa does not seems to be recieving
>>>>> data from either Zeek or Snort.
>>>>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>>>>
>>>>> --
>>>>> *Best Regards*
>>>>> Farrukh Naveed Anjum
>>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>>> *W:* https://www.farrukh.cc/
>>>>>
>>>>>
>>>>
>>>> --
>>>> *Best Regards*
>>>> Farrukh Naveed Anjum
>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>> *W:* https://www.farrukh.cc/
>>>>
>>>
>
> --
> *Best Regards*
> Farrukh Naveed Anjum
> *M:* +92 321 5083954 (WhatsApp Enabled)
> *W:* https://www.farrukh.cc/
>
Re: Metron with Zeek not working.
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Hi,
Can you please suggest what port I should use. Metron Documentation says
6667 (For Kafka) I tried both 6667 and 9092 but did not receive data in
kafka.
On Fri, Dec 6, 2019 at 9:48 AM Hema malini <nh...@gmail.com> wrote:
> Hi,
>
> This issue is basically due to Kafka configuration. Check your listener
> port in Kafka configuration .Hope this helps.
>
> Thanks and Regards,
> Hema
>
> On Fri, 6 Dec, 2019, 10:17 AM Hema malini, <nh...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Please check the listener property.Use the same port while consuming.
>>
>> Thanks and Regards,
>> Hema
>>
>> On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Problem is i am not able to Push data into Kafka..
>>>
>>> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>>>
>>> Even with Tools following test fails
>>>
>>> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create --zookeeper
>>> namenodetest:2181 --replication-factor 1 --partitions 1 --topic t1
>>>
>>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>>> --broker-list namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>>>
>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
>>> namenodetest:2181 --topic t1
>>>
>>>
>>>
>>> I am unable to see the any message transmission between consumer /
>>> producer.
>>> It keep saying
>>> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
>>> Connection to node -2 could not be established. Broker may not be
>>> available. (org.apache.kafka.clients.NetworkClient)
>>> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
>>> Connection to node -1 could not be established. Broker may not be
>>> available. (org.apache.kafka.clients.NetworkClient)
>>> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
>>> Connection to node -3 could not be established. Broker may not be
>>> available. (org.apache.kafka.clients.NetworkClient)
>>>
>>>
>>> Any suggesion what can I do ?
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <ot...@gmail.com>
>>> wrote:
>>>
>>>> I don’t think we support newer versions of bro yet i.e. zeek.
>>>>
>>>>
>>>>
>>>>
>>>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>>>> anjum.farrukh@gmail.com) wrote:
>>>>
>>>> Hi,
>>>> I am trying to use upgraded version of Bro that is Zeek. I am unable to
>>>> receive data into Kafka
>>>>
>>>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>>>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG, SSL::LOG,
>>>> DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG, DNP3::LOG,
>>>> RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG, Software::LOG,
>>>> Intel::LOG, Notice::LOG, Signatures::LOG);
>>>> redef Kafka::send_all_active_logs = T;
>>>> redef Kafka::topic_name = "bro";
>>>> redef Kafka::tag_json = T;
>>>> redef Kafka::kafka_conf = table(
>>>> ["metadata.broker.list"] = "localhost:6667",
>>>> ["client.id"] = "bro"
>>>> );
>>>>
>>>> I have 1 name node, 2 data nodes. Kafa does not seems to be recieving
>>>> data from either Zeek or Snort.
>>>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>>>
>>>> --
>>>> *Best Regards*
>>>> Farrukh Naveed Anjum
>>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>>> *W:* https://www.farrukh.cc/
>>>>
>>>>
>>>
>>> --
>>> *Best Regards*
>>> Farrukh Naveed Anjum
>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>> *W:* https://www.farrukh.cc/
>>>
>>
--
*Best Regards*
Farrukh Naveed Anjum
*M:* +92 321 5083954 (WhatsApp Enabled)
*W:* https://www.farrukh.cc/
Re: Metron with Zeek not working.
Posted by Hema malini <nh...@gmail.com>.
Hi,
This issue is basically due to Kafka configuration. Check your listener
port in Kafka configuration .Hope this helps.
Thanks and Regards,
Hema
On Fri, 6 Dec, 2019, 10:17 AM Hema malini, <nh...@gmail.com> wrote:
> Hi,
>
> Please check the listener property.Use the same port while consuming.
>
> Thanks and Regards,
> Hema
>
> On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <
> anjum.farrukh@gmail.com> wrote:
>
>> Problem is i am not able to Push data into Kafka..
>>
>> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>>
>> Even with Tools following test fails
>>
>> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create --zookeeper
>> namenodetest:2181 --replication-factor 1 --partitions 1 --topic t1
>>
>> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list
>> namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>>
>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
>> namenodetest:2181 --topic t1
>>
>>
>>
>> I am unable to see the any message transmission between consumer /
>> producer.
>> It keep saying
>> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
>> Connection to node -2 could not be established. Broker may not be
>> available. (org.apache.kafka.clients.NetworkClient)
>> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
>> Connection to node -1 could not be established. Broker may not be
>> available. (org.apache.kafka.clients.NetworkClient)
>> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
>> Connection to node -3 could not be established. Broker may not be
>> available. (org.apache.kafka.clients.NetworkClient)
>>
>>
>> Any suggesion what can I do ?
>>
>>
>>
>>
>>
>> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <ot...@gmail.com>
>> wrote:
>>
>>> I don’t think we support newer versions of bro yet i.e. zeek.
>>>
>>>
>>>
>>>
>>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>>> anjum.farrukh@gmail.com) wrote:
>>>
>>> Hi,
>>> I am trying to use upgraded version of Bro that is Zeek. I am unable to
>>> receive data into Kafka
>>>
>>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG, SSL::LOG,
>>> DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG, DNP3::LOG,
>>> RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG, Software::LOG,
>>> Intel::LOG, Notice::LOG, Signatures::LOG);
>>> redef Kafka::send_all_active_logs = T;
>>> redef Kafka::topic_name = "bro";
>>> redef Kafka::tag_json = T;
>>> redef Kafka::kafka_conf = table(
>>> ["metadata.broker.list"] = "localhost:6667",
>>> ["client.id"] = "bro"
>>> );
>>>
>>> I have 1 name node, 2 data nodes. Kafa does not seems to be recieving
>>> data from either Zeek or Snort.
>>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>>
>>> --
>>> *Best Regards*
>>> Farrukh Naveed Anjum
>>> *M:* +92 321 5083954 (WhatsApp Enabled)
>>> *W:* https://www.farrukh.cc/
>>>
>>>
>>
>> --
>> *Best Regards*
>> Farrukh Naveed Anjum
>> *M:* +92 321 5083954 (WhatsApp Enabled)
>> *W:* https://www.farrukh.cc/
>>
>
Re: Metron with Zeek not working.
Posted by Hema malini <nh...@gmail.com>.
Hi,
Please check the listener property.Use the same port while consuming.
Thanks and Regards,
Hema
On Fri, 6 Dec, 2019, 9:54 AM Farrukh Naveed Anjum, <an...@gmail.com>
wrote:
> Problem is i am not able to Push data into Kafka..
>
> I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
>
> Even with Tools following test fails
>
> /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create --zookeeper
> namenodetest:2181 --replication-factor 1 --partitions 1 --topic t1
>
> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list
> namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
>
> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
> namenodetest:2181 --topic t1
>
>
>
> I am unable to see the any message transmission between consumer /
> producer.
> It keep saying
> [2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
> Connection to node -2 could not be established. Broker may not be
> available. (org.apache.kafka.clients.NetworkClient)
> [2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
> Connection to node -1 could not be established. Broker may not be
> available. (org.apache.kafka.clients.NetworkClient)
> [2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
> Connection to node -3 could not be established. Broker may not be
> available. (org.apache.kafka.clients.NetworkClient)
>
>
> Any suggesion what can I do ?
>
>
>
>
>
> On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <ot...@gmail.com>
> wrote:
>
>> I don’t think we support newer versions of bro yet i.e. zeek.
>>
>>
>>
>>
>> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
>> anjum.farrukh@gmail.com) wrote:
>>
>> Hi,
>> I am trying to use upgraded version of Bro that is Zeek. I am unable to
>> receive data into Kafka
>>
>> @load packages/metron-bro-plugin-kafka/Apache/Kafka
>> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG, SSL::LOG,
>> DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG, DNP3::LOG,
>> RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG, Software::LOG,
>> Intel::LOG, Notice::LOG, Signatures::LOG);
>> redef Kafka::send_all_active_logs = T;
>> redef Kafka::topic_name = "bro";
>> redef Kafka::tag_json = T;
>> redef Kafka::kafka_conf = table(
>> ["metadata.broker.list"] = "localhost:6667",
>> ["client.id"] = "bro"
>> );
>>
>> I have 1 name node, 2 data nodes. Kafa does not seems to be recieving
>> data from either Zeek or Snort.
>> It keep sayings broker may not be avalable stuff. Any suggestion ?
>>
>> --
>> *Best Regards*
>> Farrukh Naveed Anjum
>> *M:* +92 321 5083954 (WhatsApp Enabled)
>> *W:* https://www.farrukh.cc/
>>
>>
>
> --
> *Best Regards*
> Farrukh Naveed Anjum
> *M:* +92 321 5083954 (WhatsApp Enabled)
> *W:* https://www.farrukh.cc/
>
Re: Metron with Zeek not working.
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Problem is i am not able to Push data into Kafka..
I have 1 Name Node, 2 Data Node (All have Kafka Broker + Zookeeper).
Even with Tools following test fails
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --create --zookeeper
namenodetest:2181 --replication-factor 1 --partitions 1 --topic t1
/usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list
namenodetest:9092,datanode1:9092,datanode2:9092 --topic t1
/usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
namenodetest:2181 --topic t1
I am unable to see the any message transmission between consumer / producer.
It keep saying
[2019-12-06 09:23:23,030] WARN [Producer clientId=console-producer]
Connection to node -2 could not be established. Broker may not be
available. (org.apache.kafka.clients.NetworkClient)
[2019-12-06 09:23:23,182] WARN [Producer clientId=console-producer]
Connection to node -1 could not be established. Broker may not be
available. (org.apache.kafka.clients.NetworkClient)
[2019-12-06 09:23:23,434] WARN [Producer clientId=console-producer]
Connection to node -3 could not be established. Broker may not be
available. (org.apache.kafka.clients.NetworkClient)
Any suggesion what can I do ?
On Thu, Dec 5, 2019 at 10:40 PM Otto Fowler <ot...@gmail.com> wrote:
> I don’t think we support newer versions of bro yet i.e. zeek.
>
>
>
>
> On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
> anjum.farrukh@gmail.com) wrote:
>
> Hi,
> I am trying to use upgraded version of Bro that is Zeek. I am unable to
> receive data into Kafka
>
> @load packages/metron-bro-plugin-kafka/Apache/Kafka
> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG, SSL::LOG,
> DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG, DNP3::LOG,
> RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG, Software::LOG,
> Intel::LOG, Notice::LOG, Signatures::LOG);
> redef Kafka::send_all_active_logs = T;
> redef Kafka::topic_name = "bro";
> redef Kafka::tag_json = T;
> redef Kafka::kafka_conf = table(
> ["metadata.broker.list"] = "localhost:6667",
> ["client.id"] = "bro"
> );
>
> I have 1 name node, 2 data nodes. Kafa does not seems to be recieving data
> from either Zeek or Snort.
> It keep sayings broker may not be avalable stuff. Any suggestion ?
>
> --
> *Best Regards*
> Farrukh Naveed Anjum
> *M:* +92 321 5083954 (WhatsApp Enabled)
> *W:* https://www.farrukh.cc/
>
>
--
*Best Regards*
Farrukh Naveed Anjum
*M:* +92 321 5083954 (WhatsApp Enabled)
*W:* https://www.farrukh.cc/
Re: Metron with Zeek not working.
Posted by Otto Fowler <ot...@gmail.com>.
I don’t think we support newer versions of bro yet i.e. zeek.
On December 5, 2019 at 10:31:12, Farrukh Naveed Anjum (
anjum.farrukh@gmail.com) wrote:
Hi,
I am trying to use upgraded version of Bro that is Zeek. I am unable to
receive data into Kafka
@load packages/metron-bro-plugin-kafka/Apache/Kafka
redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG, SSL::LOG,
DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG, DNP3::LOG,
RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG, Software::LOG,
Intel::LOG, Notice::LOG, Signatures::LOG);
redef Kafka::send_all_active_logs = T;
redef Kafka::topic_name = "bro";
redef Kafka::tag_json = T;
redef Kafka::kafka_conf = table(
["metadata.broker.list"] = "localhost:6667",
["client.id"] = "bro"
);
I have 1 name node, 2 data nodes. Kafa does not seems to be recieving data
from either Zeek or Snort.
It keep sayings broker may not be avalable stuff. Any suggestion ?
--
*Best Regards*
Farrukh Naveed Anjum
*M:* +92 321 5083954 (WhatsApp Enabled)
*W:* https://www.farrukh.cc/
Re: Metron with Zeek not working.
Posted by Nick Allen <ni...@nickallen.org>.
Are you able to push data in kafka using Kafka's console producer/consumer
tools?
On Thu, Dec 5, 2019 at 10:31 AM Farrukh Naveed Anjum <
anjum.farrukh@gmail.com> wrote:
> Hi,
> I am trying to use upgraded version of Bro that is Zeek. I am unable to
> receive data into Kafka
>
> @load packages/metron-bro-plugin-kafka/Apache/Kafka
> redef Kafka::logs_to_send = set(SSH::LOG, RDP::LOG, KRB::LOG, SSL::LOG,
> DHCP::LOG, Cluster::LOG, Syslog::LOG, SNMP::LOG, Reporter::LOG, DNP3::LOG,
> RADIUS::LOG, Tunnel::LOG, Conn::LOG, HTTP::LOG, DNS::LOG, Software::LOG,
> Intel::LOG, Notice::LOG, Signatures::LOG);
> redef Kafka::send_all_active_logs = T;
> redef Kafka::topic_name = "bro";
> redef Kafka::tag_json = T;
> redef Kafka::kafka_conf = table(
> ["metadata.broker.list"] = "localhost:6667",
> ["client.id"] = "bro"
> );
>
> I have 1 name node, 2 data nodes. Kafa does not seems to be recieving data
> from either Zeek or Snort.
> It keep sayings broker may not be avalable stuff. Any suggestion ?
>
> --
> *Best Regards*
> Farrukh Naveed Anjum
> *M:* +92 321 5083954 (WhatsApp Enabled)
> *W:* https://www.farrukh.cc/
>