You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by Farrukh Naveed Anjum <an...@gmail.com> on 2019/03/24 14:32:48 UTC
Metron 0.7.1 (METRON INDEXING) bug
Hi,
I am experiencing the bug in meanwhile getting data from KAFKA to METRON...
It does not show any thing in management console of metron. Neither creates
the index.
However, I can confirm I am able to see Snort, Bro Data coming to the Kafka.
/usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
localhost:2181 --topic snort --from-beginning
Can some one please guide me where shall I look for problem. There is no
logging for enrichment, bro, indexing processes. I have checked Storm UI
but no error there.
--
With Regards
Farrukh Naveed Anjum
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Ali Nazemian <al...@gmail.com>.
You need to first make sure delete.topic.enable is set to true. Make sure
there is no producer and consumer registered to the topic. (Stop all storm
topologies if you can and make sure all of them are stopped entirely.)
Then, you can use kafka topic cli to delete the topic by using --delete
[topic_name]. Kafka will mark the topic to be deleted and then it will be
deleted in the background shortly. Please be mindful if you don't manage to
stop all consumers and producers, Kafka may think the topic is deleted, but
it is actually not deleted from Zookeeper and log dir. In that case, you
need to delete log dir manually and delete the zookeeper corresponding
znodes as well. This task some times cause some issues. Therefore, double
check all the producers and consumers to make sure nothing is active on the
topic you want to delete.
Cheers,
Ali
On Thu, Mar 28, 2019 at 2:18 AM Farrukh Naveed Anjum <
anjum.farrukh@gmail.com> wrote:
> How can I delete topics and purge Data in Kafka ?
>
> On Wed, Mar 27, 2019, 8:16 PM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Its in the KAFKA ERROR
>>
>> On Mon, Mar 25, 2019, 6:08 PM Michael Miklavcic <
>> michael.miklavcic@gmail.com> wrote:
>>
>>> Check out your error index in es or solr
>>>
>>> On Mon, Mar 25, 2019, 12:10 AM Farrukh Naveed Anjum <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> Hi,
>>>> Please do find the attached screenshots, Real Problem with new STORM is
>>>> it does not give any ERROR. I am unable to track where is the problem ? Why
>>>> it is not working?
>>>>
>>>> Please help
>>>>
>>>> On Sun, Mar 24, 2019 at 10:56 PM Farrukh Naveed Anjum <
>>>> anjum.farrukh@gmail.com> wrote:
>>>>
>>>>> Any reason i am not getting error in Storm UI
>>>>>
>>>>> On Sun, Mar 24, 2019, 10:50 PM Farrukh Naveed Anjum <
>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>
>>>>>> It is 1
>>>>>>
>>>>>> On Sun, Mar 24, 2019, 10:47 PM Michael Miklavcic <
>>>>>> michael.miklavcic@gmail.com> wrote:
>>>>>>
>>>>>>> Check out the storm UI. I'm not in front of a computer or I'd try to
>>>>>>> share more specifics. If your available slots show as 0 on the main Storm
>>>>>>> UI page, you should be able to change it via the storm settings in Ambari.
>>>>>>>
>>>>>>> On Sun, Mar 24, 2019, 10:39 AM Farrukh Naveed Anjum <
>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Where should i look slots are there
>>>>>>>>
>>>>>>>> On Sun, Mar 24, 2019, 10:37 PM Michael Miklavcic <
>>>>>>>> michael.miklavcic@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> To my knowledge, we haven't seen this issue before. If you're not
>>>>>>>>> getting anything in enrichment I'd look for exceptions in the parser logs.
>>>>>>>>> It sounds like messages aren't making it to the enrichment topic. Afa the
>>>>>>>>> indexing topology you might double check that you have enough Storm slots
>>>>>>>>> available.
>>>>>>>>>
>>>>>>>>> On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <
>>>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi, Can you please approve and answer this question? I know user
>>>>>>>>>> support is optional and voluntary
>>>>>>>>>>
>>>>>>>>>> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
>>>>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Furthermore, Metron Indexing Process Shows Stopped in Ambari
>>>>>>>>>>> after starting for few minutes. But in Storm UI
>>>>>>>>>>> random_access_indexing
>>>>>>>>>>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>>>>>>>>>>> and batch_indexing topologies keeps running. God knows what is problem.
>>>>>>>>>>> with this..
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I build 0.7.0 successfully and deployed a month ago but right no
>>>>>>>>>>> neither 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>>>>>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I am experiencing the bug in meanwhile getting data from KAFKA
>>>>>>>>>>>> to METRON... It does not show any thing in management console of metron.
>>>>>>>>>>>> Neither creates the index.
>>>>>>>>>>>>
>>>>>>>>>>>> However, I can confirm I am able to see Snort, Bro Data coming
>>>>>>>>>>>> to the Kafka.
>>>>>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>>>>>>> --zookeeper localhost:2181 --topic snort --from-beginning
>>>>>>>>>>>>
>>>>>>>>>>>> Can some one please guide me where shall I look for problem.
>>>>>>>>>>>> There is no logging for enrichment, bro, indexing processes. I have checked
>>>>>>>>>>>> Storm UI but no error there.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> With Regards
>>>>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> With Regards
>>>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> With Regards
>>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>>>
>>>>>>>>>
>>>>
>>>> --
>>>> With Regards
>>>> Farrukh Naveed Anjum
>>>>
>>>
--
A.Nazemian
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
How can I delete topics and purge Data in Kafka ?
On Wed, Mar 27, 2019, 8:16 PM Farrukh Naveed Anjum <an...@gmail.com>
wrote:
> Its in the KAFKA ERROR
>
> On Mon, Mar 25, 2019, 6:08 PM Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
>> Check out your error index in es or solr
>>
>> On Mon, Mar 25, 2019, 12:10 AM Farrukh Naveed Anjum <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Hi,
>>> Please do find the attached screenshots, Real Problem with new STORM is
>>> it does not give any ERROR. I am unable to track where is the problem ? Why
>>> it is not working?
>>>
>>> Please help
>>>
>>> On Sun, Mar 24, 2019 at 10:56 PM Farrukh Naveed Anjum <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> Any reason i am not getting error in Storm UI
>>>>
>>>> On Sun, Mar 24, 2019, 10:50 PM Farrukh Naveed Anjum <
>>>> anjum.farrukh@gmail.com> wrote:
>>>>
>>>>> It is 1
>>>>>
>>>>> On Sun, Mar 24, 2019, 10:47 PM Michael Miklavcic <
>>>>> michael.miklavcic@gmail.com> wrote:
>>>>>
>>>>>> Check out the storm UI. I'm not in front of a computer or I'd try to
>>>>>> share more specifics. If your available slots show as 0 on the main Storm
>>>>>> UI page, you should be able to change it via the storm settings in Ambari.
>>>>>>
>>>>>> On Sun, Mar 24, 2019, 10:39 AM Farrukh Naveed Anjum <
>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>
>>>>>>> Where should i look slots are there
>>>>>>>
>>>>>>> On Sun, Mar 24, 2019, 10:37 PM Michael Miklavcic <
>>>>>>> michael.miklavcic@gmail.com> wrote:
>>>>>>>
>>>>>>>> To my knowledge, we haven't seen this issue before. If you're not
>>>>>>>> getting anything in enrichment I'd look for exceptions in the parser logs.
>>>>>>>> It sounds like messages aren't making it to the enrichment topic. Afa the
>>>>>>>> indexing topology you might double check that you have enough Storm slots
>>>>>>>> available.
>>>>>>>>
>>>>>>>> On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <
>>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi, Can you please approve and answer this question? I know user
>>>>>>>>> support is optional and voluntary
>>>>>>>>>
>>>>>>>>> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
>>>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Furthermore, Metron Indexing Process Shows Stopped in Ambari
>>>>>>>>>> after starting for few minutes. But in Storm UI
>>>>>>>>>> random_access_indexing
>>>>>>>>>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>>>>>>>>>> and batch_indexing topologies keeps running. God knows what is problem.
>>>>>>>>>> with this..
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I build 0.7.0 successfully and deployed a month ago but right no
>>>>>>>>>> neither 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>>>>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I am experiencing the bug in meanwhile getting data from KAFKA
>>>>>>>>>>> to METRON... It does not show any thing in management console of metron.
>>>>>>>>>>> Neither creates the index.
>>>>>>>>>>>
>>>>>>>>>>> However, I can confirm I am able to see Snort, Bro Data coming
>>>>>>>>>>> to the Kafka.
>>>>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>>>>>> --zookeeper localhost:2181 --topic snort --from-beginning
>>>>>>>>>>>
>>>>>>>>>>> Can some one please guide me where shall I look for problem.
>>>>>>>>>>> There is no logging for enrichment, bro, indexing processes. I have checked
>>>>>>>>>>> Storm UI but no error there.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> With Regards
>>>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> With Regards
>>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> With Regards
>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>>
>>>>>>>>
>>>
>>> --
>>> With Regards
>>> Farrukh Naveed Anjum
>>>
>>
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Its in the KAFKA ERROR
On Mon, Mar 25, 2019, 6:08 PM Michael Miklavcic <mi...@gmail.com>
wrote:
> Check out your error index in es or solr
>
> On Mon, Mar 25, 2019, 12:10 AM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Hi,
>> Please do find the attached screenshots, Real Problem with new STORM is
>> it does not give any ERROR. I am unable to track where is the problem ? Why
>> it is not working?
>>
>> Please help
>>
>> On Sun, Mar 24, 2019 at 10:56 PM Farrukh Naveed Anjum <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Any reason i am not getting error in Storm UI
>>>
>>> On Sun, Mar 24, 2019, 10:50 PM Farrukh Naveed Anjum <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> It is 1
>>>>
>>>> On Sun, Mar 24, 2019, 10:47 PM Michael Miklavcic <
>>>> michael.miklavcic@gmail.com> wrote:
>>>>
>>>>> Check out the storm UI. I'm not in front of a computer or I'd try to
>>>>> share more specifics. If your available slots show as 0 on the main Storm
>>>>> UI page, you should be able to change it via the storm settings in Ambari.
>>>>>
>>>>> On Sun, Mar 24, 2019, 10:39 AM Farrukh Naveed Anjum <
>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>
>>>>>> Where should i look slots are there
>>>>>>
>>>>>> On Sun, Mar 24, 2019, 10:37 PM Michael Miklavcic <
>>>>>> michael.miklavcic@gmail.com> wrote:
>>>>>>
>>>>>>> To my knowledge, we haven't seen this issue before. If you're not
>>>>>>> getting anything in enrichment I'd look for exceptions in the parser logs.
>>>>>>> It sounds like messages aren't making it to the enrichment topic. Afa the
>>>>>>> indexing topology you might double check that you have enough Storm slots
>>>>>>> available.
>>>>>>>
>>>>>>> On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <
>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi, Can you please approve and answer this question? I know user
>>>>>>>> support is optional and voluntary
>>>>>>>>
>>>>>>>> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
>>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Furthermore, Metron Indexing Process Shows Stopped in Ambari after
>>>>>>>>> starting for few minutes. But in Storm UI random_access_indexing
>>>>>>>>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>>>>>>>>> and batch_indexing topologies keeps running. God knows what is problem.
>>>>>>>>> with this..
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I build 0.7.0 successfully and deployed a month ago but right no
>>>>>>>>> neither 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>>>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I am experiencing the bug in meanwhile getting data from KAFKA to
>>>>>>>>>> METRON... It does not show any thing in management console of metron.
>>>>>>>>>> Neither creates the index.
>>>>>>>>>>
>>>>>>>>>> However, I can confirm I am able to see Snort, Bro Data coming to
>>>>>>>>>> the Kafka.
>>>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>>>>> --zookeeper localhost:2181 --topic snort --from-beginning
>>>>>>>>>>
>>>>>>>>>> Can some one please guide me where shall I look for problem.
>>>>>>>>>> There is no logging for enrichment, bro, indexing processes. I have checked
>>>>>>>>>> Storm UI but no error there.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> With Regards
>>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> With Regards
>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> With Regards
>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>
>>>>>>>
>>
>> --
>> With Regards
>> Farrukh Naveed Anjum
>>
>
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Michael Miklavcic <mi...@gmail.com>.
Check out your error index in es or solr
On Mon, Mar 25, 2019, 12:10 AM Farrukh Naveed Anjum <an...@gmail.com>
wrote:
> Hi,
> Please do find the attached screenshots, Real Problem with new STORM is it
> does not give any ERROR. I am unable to track where is the problem ? Why it
> is not working?
>
> Please help
>
> On Sun, Mar 24, 2019 at 10:56 PM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Any reason i am not getting error in Storm UI
>>
>> On Sun, Mar 24, 2019, 10:50 PM Farrukh Naveed Anjum <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> It is 1
>>>
>>> On Sun, Mar 24, 2019, 10:47 PM Michael Miklavcic <
>>> michael.miklavcic@gmail.com> wrote:
>>>
>>>> Check out the storm UI. I'm not in front of a computer or I'd try to
>>>> share more specifics. If your available slots show as 0 on the main Storm
>>>> UI page, you should be able to change it via the storm settings in Ambari.
>>>>
>>>> On Sun, Mar 24, 2019, 10:39 AM Farrukh Naveed Anjum <
>>>> anjum.farrukh@gmail.com> wrote:
>>>>
>>>>> Where should i look slots are there
>>>>>
>>>>> On Sun, Mar 24, 2019, 10:37 PM Michael Miklavcic <
>>>>> michael.miklavcic@gmail.com> wrote:
>>>>>
>>>>>> To my knowledge, we haven't seen this issue before. If you're not
>>>>>> getting anything in enrichment I'd look for exceptions in the parser logs.
>>>>>> It sounds like messages aren't making it to the enrichment topic. Afa the
>>>>>> indexing topology you might double check that you have enough Storm slots
>>>>>> available.
>>>>>>
>>>>>> On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <
>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>
>>>>>>> Hi, Can you please approve and answer this question? I know user
>>>>>>> support is optional and voluntary
>>>>>>>
>>>>>>> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Furthermore, Metron Indexing Process Shows Stopped in Ambari after
>>>>>>>> starting for few minutes. But in Storm UI random_access_indexing
>>>>>>>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>>>>>>>> and batch_indexing topologies keeps running. God knows what is problem.
>>>>>>>> with this..
>>>>>>>>
>>>>>>>>
>>>>>>>> I build 0.7.0 successfully and deployed a month ago but right no
>>>>>>>> neither 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I am experiencing the bug in meanwhile getting data from KAFKA to
>>>>>>>>> METRON... It does not show any thing in management console of metron.
>>>>>>>>> Neither creates the index.
>>>>>>>>>
>>>>>>>>> However, I can confirm I am able to see Snort, Bro Data coming to
>>>>>>>>> the Kafka.
>>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>>>> --zookeeper localhost:2181 --topic snort --from-beginning
>>>>>>>>>
>>>>>>>>> Can some one please guide me where shall I look for problem. There
>>>>>>>>> is no logging for enrichment, bro, indexing processes. I have checked Storm
>>>>>>>>> UI but no error there.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> With Regards
>>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> With Regards
>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> With Regards
>>>>>>> Farrukh Naveed Anjum
>>>>>>>
>>>>>>
>
> --
> With Regards
> Farrukh Naveed Anjum
>
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Hi,
Please do find the attached screenshots, Real Problem with new STORM is it
does not give any ERROR. I am unable to track where is the problem ? Why it
is not working?
Please help
On Sun, Mar 24, 2019 at 10:56 PM Farrukh Naveed Anjum <
anjum.farrukh@gmail.com> wrote:
> Any reason i am not getting error in Storm UI
>
> On Sun, Mar 24, 2019, 10:50 PM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> It is 1
>>
>> On Sun, Mar 24, 2019, 10:47 PM Michael Miklavcic <
>> michael.miklavcic@gmail.com> wrote:
>>
>>> Check out the storm UI. I'm not in front of a computer or I'd try to
>>> share more specifics. If your available slots show as 0 on the main Storm
>>> UI page, you should be able to change it via the storm settings in Ambari.
>>>
>>> On Sun, Mar 24, 2019, 10:39 AM Farrukh Naveed Anjum <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> Where should i look slots are there
>>>>
>>>> On Sun, Mar 24, 2019, 10:37 PM Michael Miklavcic <
>>>> michael.miklavcic@gmail.com> wrote:
>>>>
>>>>> To my knowledge, we haven't seen this issue before. If you're not
>>>>> getting anything in enrichment I'd look for exceptions in the parser logs.
>>>>> It sounds like messages aren't making it to the enrichment topic. Afa the
>>>>> indexing topology you might double check that you have enough Storm slots
>>>>> available.
>>>>>
>>>>> On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <
>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>
>>>>>> Hi, Can you please approve and answer this question? I know user
>>>>>> support is optional and voluntary
>>>>>>
>>>>>> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>
>>>>>>> Furthermore, Metron Indexing Process Shows Stopped in Ambari after
>>>>>>> starting for few minutes. But in Storm UI random_access_indexing
>>>>>>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>>>>>>> and batch_indexing topologies keeps running. God knows what is problem.
>>>>>>> with this..
>>>>>>>
>>>>>>>
>>>>>>> I build 0.7.0 successfully and deployed a month ago but right no
>>>>>>> neither 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I am experiencing the bug in meanwhile getting data from KAFKA to
>>>>>>>> METRON... It does not show any thing in management console of metron.
>>>>>>>> Neither creates the index.
>>>>>>>>
>>>>>>>> However, I can confirm I am able to see Snort, Bro Data coming to
>>>>>>>> the Kafka.
>>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>>> --zookeeper localhost:2181 --topic snort --from-beginning
>>>>>>>>
>>>>>>>> Can some one please guide me where shall I look for problem. There
>>>>>>>> is no logging for enrichment, bro, indexing processes. I have checked Storm
>>>>>>>> UI but no error there.
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> With Regards
>>>>>>>> Farrukh Naveed Anjum
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> With Regards
>>>>>>> Farrukh Naveed Anjum
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> With Regards
>>>>>> Farrukh Naveed Anjum
>>>>>>
>>>>>
--
With Regards
Farrukh Naveed Anjum
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Any reason i am not getting error in Storm UI
On Sun, Mar 24, 2019, 10:50 PM Farrukh Naveed Anjum <an...@gmail.com>
wrote:
> It is 1
>
> On Sun, Mar 24, 2019, 10:47 PM Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
>> Check out the storm UI. I'm not in front of a computer or I'd try to
>> share more specifics. If your available slots show as 0 on the main Storm
>> UI page, you should be able to change it via the storm settings in Ambari.
>>
>> On Sun, Mar 24, 2019, 10:39 AM Farrukh Naveed Anjum <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Where should i look slots are there
>>>
>>> On Sun, Mar 24, 2019, 10:37 PM Michael Miklavcic <
>>> michael.miklavcic@gmail.com> wrote:
>>>
>>>> To my knowledge, we haven't seen this issue before. If you're not
>>>> getting anything in enrichment I'd look for exceptions in the parser logs.
>>>> It sounds like messages aren't making it to the enrichment topic. Afa the
>>>> indexing topology you might double check that you have enough Storm slots
>>>> available.
>>>>
>>>> On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <
>>>> anjum.farrukh@gmail.com> wrote:
>>>>
>>>>> Hi, Can you please approve and answer this question? I know user
>>>>> support is optional and voluntary
>>>>>
>>>>> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>
>>>>>> Furthermore, Metron Indexing Process Shows Stopped in Ambari after
>>>>>> starting for few minutes. But in Storm UI random_access_indexing
>>>>>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>>>>>> and batch_indexing topologies keeps running. God knows what is problem.
>>>>>> with this..
>>>>>>
>>>>>>
>>>>>> I build 0.7.0 successfully and deployed a month ago but right no
>>>>>> neither 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am experiencing the bug in meanwhile getting data from KAFKA to
>>>>>>> METRON... It does not show any thing in management console of metron.
>>>>>>> Neither creates the index.
>>>>>>>
>>>>>>> However, I can confirm I am able to see Snort, Bro Data coming to
>>>>>>> the Kafka.
>>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>>> --zookeeper localhost:2181 --topic snort --from-beginning
>>>>>>>
>>>>>>> Can some one please guide me where shall I look for problem. There
>>>>>>> is no logging for enrichment, bro, indexing processes. I have checked Storm
>>>>>>> UI but no error there.
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> With Regards
>>>>>>> Farrukh Naveed Anjum
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> With Regards
>>>>>> Farrukh Naveed Anjum
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> With Regards
>>>>> Farrukh Naveed Anjum
>>>>>
>>>>
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
It is 1
On Sun, Mar 24, 2019, 10:47 PM Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:
> Check out the storm UI. I'm not in front of a computer or I'd try to share
> more specifics. If your available slots show as 0 on the main Storm UI
> page, you should be able to change it via the storm settings in Ambari.
>
> On Sun, Mar 24, 2019, 10:39 AM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Where should i look slots are there
>>
>> On Sun, Mar 24, 2019, 10:37 PM Michael Miklavcic <
>> michael.miklavcic@gmail.com> wrote:
>>
>>> To my knowledge, we haven't seen this issue before. If you're not
>>> getting anything in enrichment I'd look for exceptions in the parser logs.
>>> It sounds like messages aren't making it to the enrichment topic. Afa the
>>> indexing topology you might double check that you have enough Storm slots
>>> available.
>>>
>>> On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> Hi, Can you please approve and answer this question? I know user
>>>> support is optional and voluntary
>>>>
>>>> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
>>>> anjum.farrukh@gmail.com> wrote:
>>>>
>>>>> Furthermore, Metron Indexing Process Shows Stopped in Ambari after
>>>>> starting for few minutes. But in Storm UI random_access_indexing
>>>>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>>>>> and batch_indexing topologies keeps running. God knows what is problem.
>>>>> with this..
>>>>>
>>>>>
>>>>> I build 0.7.0 successfully and deployed a month ago but right no
>>>>> neither 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>>>>> anjum.farrukh@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am experiencing the bug in meanwhile getting data from KAFKA to
>>>>>> METRON... It does not show any thing in management console of metron.
>>>>>> Neither creates the index.
>>>>>>
>>>>>> However, I can confirm I am able to see Snort, Bro Data coming to the
>>>>>> Kafka.
>>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>>> --zookeeper localhost:2181 --topic snort --from-beginning
>>>>>>
>>>>>> Can some one please guide me where shall I look for problem. There is
>>>>>> no logging for enrichment, bro, indexing processes. I have checked Storm UI
>>>>>> but no error there.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> With Regards
>>>>>> Farrukh Naveed Anjum
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> With Regards
>>>>> Farrukh Naveed Anjum
>>>>>
>>>>
>>>>
>>>> --
>>>> With Regards
>>>> Farrukh Naveed Anjum
>>>>
>>>
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Michael Miklavcic <mi...@gmail.com>.
Check out the storm UI. I'm not in front of a computer or I'd try to share
more specifics. If your available slots show as 0 on the main Storm UI
page, you should be able to change it via the storm settings in Ambari.
On Sun, Mar 24, 2019, 10:39 AM Farrukh Naveed Anjum <an...@gmail.com>
wrote:
> Where should i look slots are there
>
> On Sun, Mar 24, 2019, 10:37 PM Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
>> To my knowledge, we haven't seen this issue before. If you're not getting
>> anything in enrichment I'd look for exceptions in the parser logs. It
>> sounds like messages aren't making it to the enrichment topic. Afa the
>> indexing topology you might double check that you have enough Storm slots
>> available.
>>
>> On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Hi, Can you please approve and answer this question? I know user support
>>> is optional and voluntary
>>>
>>> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> Furthermore, Metron Indexing Process Shows Stopped in Ambari after
>>>> starting for few minutes. But in Storm UI random_access_indexing
>>>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>>>> and batch_indexing topologies keeps running. God knows what is problem.
>>>> with this..
>>>>
>>>>
>>>> I build 0.7.0 successfully and deployed a month ago but right no
>>>> neither 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>>>
>>>>
>>>>
>>>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>>>> anjum.farrukh@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am experiencing the bug in meanwhile getting data from KAFKA to
>>>>> METRON... It does not show any thing in management console of metron.
>>>>> Neither creates the index.
>>>>>
>>>>> However, I can confirm I am able to see Snort, Bro Data coming to the
>>>>> Kafka.
>>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh
>>>>> --zookeeper localhost:2181 --topic snort --from-beginning
>>>>>
>>>>> Can some one please guide me where shall I look for problem. There is
>>>>> no logging for enrichment, bro, indexing processes. I have checked Storm UI
>>>>> but no error there.
>>>>>
>>>>>
>>>>> --
>>>>> With Regards
>>>>> Farrukh Naveed Anjum
>>>>>
>>>>
>>>>
>>>> --
>>>> With Regards
>>>> Farrukh Naveed Anjum
>>>>
>>>
>>>
>>> --
>>> With Regards
>>> Farrukh Naveed Anjum
>>>
>>
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Where should i look slots are there
On Sun, Mar 24, 2019, 10:37 PM Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:
> To my knowledge, we haven't seen this issue before. If you're not getting
> anything in enrichment I'd look for exceptions in the parser logs. It
> sounds like messages aren't making it to the enrichment topic. Afa the
> indexing topology you might double check that you have enough Storm slots
> available.
>
> On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Hi, Can you please approve and answer this question? I know user support
>> is optional and voluntary
>>
>> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Furthermore, Metron Indexing Process Shows Stopped in Ambari after
>>> starting for few minutes. But in Storm UI random_access_indexing
>>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>>> and batch_indexing topologies keeps running. God knows what is problem.
>>> with this..
>>>
>>>
>>> I build 0.7.0 successfully and deployed a month ago but right no neither
>>> 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>>
>>>
>>>
>>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>>> anjum.farrukh@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am experiencing the bug in meanwhile getting data from KAFKA to
>>>> METRON... It does not show any thing in management console of metron.
>>>> Neither creates the index.
>>>>
>>>> However, I can confirm I am able to see Snort, Bro Data coming to the
>>>> Kafka.
>>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
>>>> localhost:2181 --topic snort --from-beginning
>>>>
>>>> Can some one please guide me where shall I look for problem. There is
>>>> no logging for enrichment, bro, indexing processes. I have checked Storm UI
>>>> but no error there.
>>>>
>>>>
>>>> --
>>>> With Regards
>>>> Farrukh Naveed Anjum
>>>>
>>>
>>>
>>> --
>>> With Regards
>>> Farrukh Naveed Anjum
>>>
>>
>>
>> --
>> With Regards
>> Farrukh Naveed Anjum
>>
>
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Michael Miklavcic <mi...@gmail.com>.
To my knowledge, we haven't seen this issue before. If you're not getting
anything in enrichment I'd look for exceptions in the parser logs. It
sounds like messages aren't making it to the enrichment topic. Afa the
indexing topology you might double check that you have enough Storm slots
available.
On Sun, Mar 24, 2019, 10:26 AM Farrukh Naveed Anjum <an...@gmail.com>
wrote:
> Hi, Can you please approve and answer this question? I know user support
> is optional and voluntary
>
> On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Furthermore, Metron Indexing Process Shows Stopped in Ambari after
>> starting for few minutes. But in Storm UI random_access_indexing
>> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
>> and batch_indexing topologies keeps running. God knows what is problem.
>> with this..
>>
>>
>> I build 0.7.0 successfully and deployed a month ago but right no neither
>> 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>>
>>
>>
>> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
>> anjum.farrukh@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am experiencing the bug in meanwhile getting data from KAFKA to
>>> METRON... It does not show any thing in management console of metron.
>>> Neither creates the index.
>>>
>>> However, I can confirm I am able to see Snort, Bro Data coming to the
>>> Kafka.
>>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
>>> localhost:2181 --topic snort --from-beginning
>>>
>>> Can some one please guide me where shall I look for problem. There is no
>>> logging for enrichment, bro, indexing processes. I have checked Storm UI
>>> but no error there.
>>>
>>>
>>> --
>>> With Regards
>>> Farrukh Naveed Anjum
>>>
>>
>>
>> --
>> With Regards
>> Farrukh Naveed Anjum
>>
>
>
> --
> With Regards
> Farrukh Naveed Anjum
>
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Hi, Can you please approve and answer this question? I know user support is
optional and voluntary
On Sun, Mar 24, 2019 at 7:35 PM Farrukh Naveed Anjum <
anjum.farrukh@gmail.com> wrote:
> Furthermore, Metron Indexing Process Shows Stopped in Ambari after
> starting for few minutes. But in Storm UI random_access_indexing
> <http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
> and batch_indexing topologies keeps running. God knows what is problem.
> with this..
>
>
> I build 0.7.0 successfully and deployed a month ago but right no neither
> 0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
>
>
>
> On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
> anjum.farrukh@gmail.com> wrote:
>
>> Hi,
>>
>> I am experiencing the bug in meanwhile getting data from KAFKA to
>> METRON... It does not show any thing in management console of metron.
>> Neither creates the index.
>>
>> However, I can confirm I am able to see Snort, Bro Data coming to the
>> Kafka.
>> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
>> localhost:2181 --topic snort --from-beginning
>>
>> Can some one please guide me where shall I look for problem. There is no
>> logging for enrichment, bro, indexing processes. I have checked Storm UI
>> but no error there.
>>
>>
>> --
>> With Regards
>> Farrukh Naveed Anjum
>>
>
>
> --
> With Regards
> Farrukh Naveed Anjum
>
--
With Regards
Farrukh Naveed Anjum
Re: Metron 0.7.1 (METRON INDEXING) bug
Posted by Farrukh Naveed Anjum <an...@gmail.com>.
Furthermore, Metron Indexing Process Shows Stopped in Ambari after starting
for few minutes. But in Storm UI random_access_indexing
<http://192.168.1.149:8744/topology.html?id=random_access_indexing-5-1553437501>
and batch_indexing topologies keeps running. God knows what is problem.
with this..
I build 0.7.0 successfully and deployed a month ago but right no neither
0.7.0 and 0.7.1 are working. I installed it on baremetal machine.
On Sun, Mar 24, 2019 at 7:32 PM Farrukh Naveed Anjum <
anjum.farrukh@gmail.com> wrote:
> Hi,
>
> I am experiencing the bug in meanwhile getting data from KAFKA to
> METRON... It does not show any thing in management console of metron.
> Neither creates the index.
>
> However, I can confirm I am able to see Snort, Bro Data coming to the
> Kafka.
> /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper
> localhost:2181 --topic snort --from-beginning
>
> Can some one please guide me where shall I look for problem. There is no
> logging for enrichment, bro, indexing processes. I have checked Storm UI
> but no error there.
>
>
> --
> With Regards
> Farrukh Naveed Anjum
>
--
With Regards
Farrukh Naveed Anjum