You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@metron.apache.org by Syed Hammad Tahir <ms...@itu.edu.pk> on 2017/11/13 08:16:13 UTC

Pushing snort logs in metron

Hi all, I am starting this topic again from scratch.

*Requirement: *Push snort logs to kafka topic and then see them in metron
(kibana dashboard)

*Source if logs: *
https://raw.githubusercontent.com/apache/metron/master/metron-deployment/roles/sensor-stubs/files/snort.out

*Procedure followed:*  Saved these logs in a txt file and then ran the
following command


sudo cat snort.out |
/usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list
node1:6667 --topic snort

*Issues faced:*

1- Not able to see the data in kibana dashboard


2- Getting errors in storm topologies as evident from these screenshots


[image: Inline image 1]

I am not getting any error in any topology under the topic of snort here,
same goes for enrichments but when I go to indexing,

[image: Inline image 2]

[image: Inline image 3]

when I click hdfsindexingbolt:

[image: Inline image 4]

when I click indexingbolt:

[image: Inline image 5]

Log file for the error port 6704 has also been attached
 indexing-5-1509036400_6704_worker(1).log
<https://drive.google.com/a/itu.edu.pk/file/d/1jrmznRbGOzZF6qwnO1OYyHmSIwqSCc6Q/view?usp=drive_web>



*Please help me . *

Re: Pushing snort logs in metron

Posted by Syed Hammad Tahir <ms...@itu.edu.pk>.

Already deployed full dev environment. My installation is this one
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68718548

On Mon, Nov 13, 2017 at 6:46 PM, Nick Allen <ni...@nickallen.org> wrote:

> *> Requirement: *Push snort logs to kafka topic and then see them in
> metron (kibana dashboard)
>
> By default, the Full Dev environment gets deployed and does exactly this.
> Can you deploy Full Dev and then compare your broken environment to Full
> Dev?
>
>
>
>
>
>
> On Mon, Nov 13, 2017 at 3:16 AM Syed Hammad Tahir <ms...@itu.edu.pk>
> wrote:
>
>> Hi all, I am starting this topic again from scratch.
>>
>> *Requirement: *Push snort logs to kafka topic and then see them in
>> metron (kibana dashboard)
>>
>> *Source if logs: *https://raw.githubusercontent.com/apache/metron/master/
>> metron-deployment/roles/sensor-stubs/files/snort.out
>>
>> *Procedure followed:*  Saved these logs in a txt file and then ran the
>> following command
>>
>>
>> sudo cat snort.out | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh
>> --broker-list node1:6667 --topic snort
>>
>> *Issues faced:*
>>
>> 1- Not able to see the data in kibana dashboard
>>
>>
>> 2- Getting errors in storm topologies as evident from these screenshots
>>
>>
>> [image: Inline image 1]
>>
>> I am not getting any error in any topology under the topic of snort here,
>> same goes for enrichments but when I go to indexing,
>>
>> [image: Inline image 2]
>>
>> [image: Inline image 3]
>>
>> when I click hdfsindexingbolt:
>>
>> [image: Inline image 4]
>>
>> when I click indexingbolt:
>>
>> [image: Inline image 5]
>>
>> Log file for the error port 6704 has also been attached
>>  indexing-5-1509036400_6704_worker(1).log
>> <https://drive.google.com/a/itu.edu.pk/file/d/1jrmznRbGOzZF6qwnO1OYyHmSIwqSCc6Q/view?usp=drive_web>
>> 
>>
>>
>> *Please help me . *
>>
>>
>>

Re: Pushing snort logs in metron

Posted by Nick Allen <ni...@nickallen.org>.

*> Requirement: *Push snort logs to kafka topic and then see them in metron
(kibana dashboard)

By default, the Full Dev environment gets deployed and does exactly this.
Can you deploy Full Dev and then compare your broken environment to Full
Dev?






On Mon, Nov 13, 2017 at 3:16 AM Syed Hammad Tahir <ms...@itu.edu.pk>
wrote:

> Hi all, I am starting this topic again from scratch.
>
> *Requirement: *Push snort logs to kafka topic and then see them in metron
> (kibana dashboard)
>
> *Source if logs: *
> https://raw.githubusercontent.com/apache/metron/master/metron-deployment/roles/sensor-stubs/files/snort.out
>
> *Procedure followed:*  Saved these logs in a txt file and then ran the
> following command
>
>
> sudo cat snort.out |
> /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list
> node1:6667 --topic snort
>
> *Issues faced:*
>
> 1- Not able to see the data in kibana dashboard
>
>
> 2- Getting errors in storm topologies as evident from these screenshots
>
>
> [image: Inline image 1]
>
> I am not getting any error in any topology under the topic of snort here,
> same goes for enrichments but when I go to indexing,
>
> [image: Inline image 2]
>
> [image: Inline image 3]
>
> when I click hdfsindexingbolt:
>
> [image: Inline image 4]
>
> when I click indexingbolt:
>
> [image: Inline image 5]
>
> Log file for the error port 6704 has also been attached
>  indexing-5-1509036400_6704_worker(1).log
> <https://drive.google.com/a/itu.edu.pk/file/d/1jrmznRbGOzZF6qwnO1OYyHmSIwqSCc6Q/view?usp=drive_web>
> 
>
>
> *Please help me . *
>
>
>