You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by tkg_cangkul <yu...@gmail.com> on 2017/02/13 10:28:15 UTC
failed grok parser metron squid
hi i've try to add new telemetry data source with squid by following
this tutorial :
https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
but when i start parser topology squid i've found some error message
like this :
i've check that file on hdfs. this is the value of that file :
SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed} %{IPV4:ip_src_addr}
%{WORD:action}/%{NUMBER:code} %{NUMBER:bytes} %{WORD:method}
%{NOTSPACE:url} - %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
%{WORD:UNWANTED}\/%{WORD:UNWANTED
pls give me your suggestion.
Thanks before.
Re: failed grok parser metron squid
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
Evidently I made up 0.2.2 - that was never a release - sorry about that, I
guess that was the one we bumped to 0.3.0. Regardless, 0.2.1 is over 4
months old, and thus very different from the current state of the project.
Jon
On Mon, Feb 13, 2017 at 10:36 PM Zeolla@GMail.com <ze...@gmail.com> wrote:
> Would you mind sharing what you changed in order to fix the previous
> issue? It may be helpful to others in the future.
>
> What version of Metron are you using, it looks like 0.2.1? That is rather
> dated at this point - there has been a 0.2.2 and 0.3.0 release, and 0.3.1
> is currently being voted on for a release hopefully this week.
>
> What are the contents of
> /usr/metron/$METRON_VERSION/config/zookeeper/parsers/squid.json?
> Can you provide the output of
> `/usr/metron/$METRON_VERSION/bin/zk_load_configs.sh -m DUMP -z $zk:2181`?
> What topic is StreamToMetron configured to send to in NiFi?
>
> Jon
>
> On Mon, Feb 13, 2017 at 10:12 PM tkg_cangkul <yu...@gmail.com>
> wrote:
>
> hi, i think i've been solved this problem. i don't see any error message
> anymore. but why there is still no emitted data on my storm squid topology?
>
> i've been check my kafka producer and try to send again from nifi but
> there is still no proccessed data on my topology?
>
>
>
>
>
>
> On 13/02/17 18:19, Zeolla@GMail.com wrote:
>
> OK, sorry just wanted to check the simple things. I'll try to take
> another look at this later today when I'm at a computer unless it gets
> resolved by then.
>
> On Mon, Feb 13, 2017, 6:11 AM Youzha <yu...@gmail.com> wrote:
>
> yeah i've set the permission to 777
>
> On Mon, Feb 13, 2017 at 6:08 PM Zeolla@GMail.com <ze...@gmail.com> wrote:
>
> Did you check the permissions on the file are correct?
>
>
>
> On Mon, Feb 13, 2017, 5:59 AM tkg_cangkul <yu...@gmail.com> wrote:
>
>
>
>
>
>
>
>
>
>
>
> hi zeolla,
>
>
>
>
>
> this is the output of hdfs cat command :
>
>
>
>
>
>
>
>
>
>
>
> i've test it with this
>
> http://grokconstructor.appspot.com/do/match#result
>
>
> and the result was match
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 13/02/17 17:54, Zeolla@GMail.com
>
> wrote:
>
>
>
>
>
>
> You appear to be missing a final }. Does it work
>
> with the standard grok pattern from the tutorial? Can you show
>
> the output of:
>
>
> hadoop fs -cat /apps/metron/patterns/squid
>
>
> Jon
>
>
>
>
>
>
>
> On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul <yu...@gmail.com>
>
> wrote:
>
>
>
>
>
>
> hi
>
> i've try to add new telemetry data source with squid by
>
> following this tutorial :
>
>
>
>
>
>
> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>
>
>
>
>
> but when i start parser topology squid i've found some error
>
> message like this :
>
>
>
>
>
>
>
>
>
>
>
> i've check that file on hdfs. this is the value of that file
>
> :
>
>
>
>
>
> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed}
>
> %{IPV4:ip_src_addr} %{WORD:action}/%{NUMBER:code}
>
> %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} -
>
> %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
>
> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>
>
>
>
>
> pls give me your suggestion.
>
>
>
>
>
> Thanks before.
>
>
>
>
>
>
>
>
> --
>
>
>
>
>
>
> Jon
>
>
> Sent from my mobile device
>
>
>
>
>
>
>
>
>
> --
>
> Jon
>
>
> Sent from my mobile device
>
>
>
>
> --
>
> Jon
>
> Sent from my mobile device
>
>
> --
>
> Jon
>
> Sent from my mobile device
>
--
Jon
Sent from my mobile device
Re: failed grok parser metron squid
Posted by tkg_cangkul <yu...@gmail.com>.
i just changed the path of my patterns. i thought if that is no need to
use a full path on hdfs. i've change it into /patterns/squid and the
error message gone.
i'm using metron 0.2.1 because i've try to install 0.3.0 but there is no
log file that created on my storm log dir. when i try with metron 0.2.1
the error logs is appear.
this is the value of my squid.json :
{
"parserClassName": "org.apache.metron.parsers.GrokParser",
"sensorTopic": "squid",
"parserConfig": {
"grokPath": "/patterns/squid",
"patternLabel": "SQUID_DELIMITED",
"timestampField": "timestamp"
},
"fieldTransformations" : [
{
"transformation" : "STELLAR"
,"output" : [ "full_hostname", "domain_without_subdomains" ]
,"config" : {
"full_hostname" : "URL_TO_HOST(url)"
,"domain_without_subdomains" :
"DOMAIN_REMOVE_SUBDOMAINS(full_hostname)"
}
}
]
}
and the output of load config is :
On 14/02/17 10:36, Zeolla@GMail.com wrote:
> Would you mind sharing what you changed in order to fix the previous
> issue? It may be helpful to others in the future.
>
> What version of Metron are you using, it looks like 0.2.1? That is
> rather dated at this point - there has been a 0.2.2 and 0.3.0 release,
> and 0.3.1 is currently being voted on for a release hopefully this week.
>
> What are the contents of
> /usr/metron/$METRON_VERSION/config/zookeeper/parsers/squid.json?
> Can you provide the output of
> `/usr/metron/$METRON_VERSION/bin/zk_load_configs.sh -m DUMP -z $zk:2181`?
> What topic is StreamToMetron configured to send to in NiFi?
>
> Jon
>
> On Mon, Feb 13, 2017 at 10:12 PM tkg_cangkul <yuza.rasfar@gmail.com
> <ma...@gmail.com>> wrote:
>
> hi, i think i've been solved this problem. i don't see any error
> message anymore. but why there is still no emitted data on my
> storm squid topology?
>
> i've been check my kafka producer and try to send again from nifi
> but there is still no proccessed data on my topology?
>
>
>
>
>
>
> On 13/02/17 18:19, Zeolla@GMail.com <ma...@GMail.com> wrote:
>>
>> OK, sorry just wanted to check the simple things. I'll try to
>> take another look at this later today when I'm at a computer
>> unless it gets resolved by then.
>>
>>
>> On Mon, Feb 13, 2017, 6:11 AM Youzha <yuza.rasfar@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> yeah i've set the permission to 777
>>
>> On Mon, Feb 13, 2017 at 6:08 PM Zeolla@GMail.com
>> <ma...@GMail.com> <zeolla@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> Did you check the permissions on the file are correct?
>>
>>
>>
>>
>> On Mon, Feb 13, 2017, 5:59 AM tkg_cangkul
>> <yuza.rasfar@gmail.com <ma...@gmail.com>> wrote:
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> hi zeolla,
>>
>>
>>
>>
>>
>> this is the output of hdfs cat command :
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> i've test it with this
>>
>> http://grokconstructor.appspot.com/do/match#result
>>
>>
>> and the result was match
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On 13/02/17 17:54, Zeolla@GMail.com
>> <ma...@GMail.com>
>>
>> wrote:
>>
>>
>>
>>
>>>
>>>
>>> You appear to be missing a final }. Does it work
>>>
>>> with the standard grok pattern from the tutorial?
>>> Can you show
>>>
>>> the output of:
>>>
>>>
>>> hadoop fs -cat /apps/metron/patterns/squid
>>>
>>>
>>>
>>> Jon
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul
>>> <yuza.rasfar@gmail.com <ma...@gmail.com>>
>>>
>>> wrote:
>>>
>>>
>>>
>>>
>>>
>>>
>>> hi
>>>
>>> i've try to add new telemetry data source with
>>> squid by
>>>
>>> following this tutorial :
>>>
>>>
>>>
>>>
>>>
>>> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>>>
>>>
>>>
>>>
>>>
>>> but when i start parser topology squid i've
>>> found some error
>>>
>>> message like this :
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> i've check that file on hdfs. this is the value
>>> of that file
>>>
>>> :
>>>
>>>
>>>
>>>
>>>
>>> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed}
>>>
>>> %{IPV4:ip_src_addr} %{WORD:action}/%{NUMBER:code}
>>>
>>> %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} -
>>>
>>> %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
>>>
>>> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>>>
>>>
>>>
>>>
>>>
>>> pls give me your suggestion.
>>>
>>>
>>>
>>>
>>>
>>> Thanks before.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>>
>>>
>>> Jon
>>>
>>>
>>>
>>> Sent from my mobile device
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>> --
>>
>> Jon
>>
>>
>>
>> Sent from my mobile device
>>
>>
>>
>>
>>
>> --
>>
>> Jon
>>
>> Sent from my mobile device
>>
>
> --
>
> Jon
>
> Sent from my mobile device
>
Re: failed grok parser metron squid
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
Would you mind sharing what you changed in order to fix the previous
issue? It may be helpful to others in the future.
What version of Metron are you using, it looks like 0.2.1? That is rather
dated at this point - there has been a 0.2.2 and 0.3.0 release, and 0.3.1
is currently being voted on for a release hopefully this week.
What are the contents of
/usr/metron/$METRON_VERSION/config/zookeeper/parsers/squid.json?
Can you provide the output of
`/usr/metron/$METRON_VERSION/bin/zk_load_configs.sh -m DUMP -z $zk:2181`?
What topic is StreamToMetron configured to send to in NiFi?
Jon
On Mon, Feb 13, 2017 at 10:12 PM tkg_cangkul <yu...@gmail.com> wrote:
> hi, i think i've been solved this problem. i don't see any error message
> anymore. but why there is still no emitted data on my storm squid topology?
>
> i've been check my kafka producer and try to send again from nifi but
> there is still no proccessed data on my topology?
>
>
>
>
>
>
> On 13/02/17 18:19, Zeolla@GMail.com wrote:
>
> OK, sorry just wanted to check the simple things. I'll try to take
> another look at this later today when I'm at a computer unless it gets
> resolved by then.
>
> On Mon, Feb 13, 2017, 6:11 AM Youzha <yu...@gmail.com> wrote:
>
> yeah i've set the permission to 777
>
> On Mon, Feb 13, 2017 at 6:08 PM Zeolla@GMail.com <ze...@gmail.com> wrote:
>
> Did you check the permissions on the file are correct?
>
>
>
> On Mon, Feb 13, 2017, 5:59 AM tkg_cangkul <yu...@gmail.com> wrote:
>
>
>
>
>
>
>
>
>
>
>
> hi zeolla,
>
>
>
>
>
> this is the output of hdfs cat command :
>
>
>
>
>
>
>
>
>
>
>
> i've test it with this
>
> http://grokconstructor.appspot.com/do/match#result
>
>
> and the result was match
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 13/02/17 17:54, Zeolla@GMail.com
>
> wrote:
>
>
>
>
>
>
> You appear to be missing a final }. Does it work
>
> with the standard grok pattern from the tutorial? Can you show
>
> the output of:
>
>
> hadoop fs -cat /apps/metron/patterns/squid
>
>
> Jon
>
>
>
>
>
>
>
> On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul <yu...@gmail.com>
>
> wrote:
>
>
>
>
>
>
> hi
>
> i've try to add new telemetry data source with squid by
>
> following this tutorial :
>
>
>
>
>
>
> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>
>
>
>
>
> but when i start parser topology squid i've found some error
>
> message like this :
>
>
>
>
>
>
>
>
>
>
>
> i've check that file on hdfs. this is the value of that file
>
> :
>
>
>
>
>
> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed}
>
> %{IPV4:ip_src_addr} %{WORD:action}/%{NUMBER:code}
>
> %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} -
>
> %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
>
> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>
>
>
>
>
> pls give me your suggestion.
>
>
>
>
>
> Thanks before.
>
>
>
>
>
>
>
>
> --
>
>
>
>
>
>
> Jon
>
>
> Sent from my mobile device
>
>
>
>
>
>
>
>
>
> --
>
> Jon
>
>
> Sent from my mobile device
>
>
>
>
> --
>
> Jon
>
> Sent from my mobile device
>
>
> --
Jon
Sent from my mobile device
Re: failed grok parser metron squid
Posted by tkg_cangkul <yu...@gmail.com>.
hi, i think i've been solved this problem. i don't see any error message
anymore. but why there is still no emitted data on my storm squid topology?
i've been check my kafka producer and try to send again from nifi but
there is still no proccessed data on my topology?
On 13/02/17 18:19, Zeolla@GMail.com wrote:
>
> OK, sorry just wanted to check the simple things. I'll try to take
> another look at this later today when I'm at a computer unless it gets
> resolved by then.
>
>
> On Mon, Feb 13, 2017, 6:11 AM Youzha <yuza.rasfar@gmail.com
> <ma...@gmail.com>> wrote:
>
> yeah i've set the permission to 777
>
> On Mon, Feb 13, 2017 at 6:08 PM Zeolla@GMail.com <zeolla@gmail.com
> <ma...@gmail.com>> wrote:
>
> Did you check the permissions on the file are correct?
>
>
>
>
> On Mon, Feb 13, 2017, 5:59 AM tkg_cangkul
> <yuza.rasfar@gmail.com <ma...@gmail.com>> wrote:
>
>
>
>
>
>
>
>
>
>
>
> hi zeolla,
>
>
>
>
>
> this is the output of hdfs cat command :
>
>
>
>
>
>
>
>
>
>
>
> i've test it with this
>
> http://grokconstructor.appspot.com/do/match#result
>
>
> and the result was match
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 13/02/17 17:54, Zeolla@GMail.com <ma...@GMail.com>
>
> wrote:
>
>
>
>
>>
>>
>> You appear to be missing a final }. Does it work
>>
>> with the standard grok pattern from the tutorial? Can
>> you show
>>
>> the output of:
>>
>>
>> hadoop fs -cat /apps/metron/patterns/squid
>>
>>
>>
>> Jon
>>
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul
>> <yuza.rasfar@gmail.com <ma...@gmail.com>>
>>
>> wrote:
>>
>>
>>
>>
>>
>>
>> hi
>>
>> i've try to add new telemetry data source with squid by
>>
>> following this tutorial :
>>
>>
>>
>>
>>
>> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>>
>>
>>
>>
>>
>> but when i start parser topology squid i've found
>> some error
>>
>> message like this :
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> i've check that file on hdfs. this is the value of
>> that file
>>
>> :
>>
>>
>>
>>
>>
>> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed}
>>
>> %{IPV4:ip_src_addr} %{WORD:action}/%{NUMBER:code}
>>
>> %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} -
>>
>> %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
>>
>> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>>
>>
>>
>>
>>
>> pls give me your suggestion.
>>
>>
>>
>>
>>
>> Thanks before.
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>>
>>
>>
>>
>>
>> Jon
>>
>>
>>
>> Sent from my mobile device
>>
>>
>>
>>
>>
>
>
>
>
>
> --
>
> Jon
>
>
>
> Sent from my mobile device
>
>
>
>
>
> --
>
> Jon
>
> Sent from my mobile device
>
Re: failed grok parser metron squid
Posted by tkg_cangkul <yu...@gmail.com>.
ok, i appreciate for your help sir. thx again for your help.
On 13/02/17 18:19, Zeolla@GMail.com wrote:
>
> OK, sorry just wanted to check the simple things. I'll try to take
> another look at this later today when I'm at a computer unless it gets
> resolved by then.
>
>
> On Mon, Feb 13, 2017, 6:11 AM Youzha <yuza.rasfar@gmail.com
> <ma...@gmail.com>> wrote:
>
> yeah i've set the permission to 777
>
> On Mon, Feb 13, 2017 at 6:08 PM Zeolla@GMail.com <zeolla@gmail.com
> <ma...@gmail.com>> wrote:
>
> Did you check the permissions on the file are correct?
>
>
>
>
> On Mon, Feb 13, 2017, 5:59 AM tkg_cangkul
> <yuza.rasfar@gmail.com <ma...@gmail.com>> wrote:
>
>
>
>
>
>
>
>
>
>
>
> hi zeolla,
>
>
>
>
>
> this is the output of hdfs cat command :
>
>
>
>
>
>
>
>
>
>
>
> i've test it with this
>
> http://grokconstructor.appspot.com/do/match#result
>
>
> and the result was match
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 13/02/17 17:54, Zeolla@GMail.com <ma...@GMail.com>
>
> wrote:
>
>
>
>
>>
>>
>> You appear to be missing a final }. Does it work
>>
>> with the standard grok pattern from the tutorial? Can
>> you show
>>
>> the output of:
>>
>>
>> hadoop fs -cat /apps/metron/patterns/squid
>>
>>
>>
>> Jon
>>
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul
>> <yuza.rasfar@gmail.com <ma...@gmail.com>>
>>
>> wrote:
>>
>>
>>
>>
>>
>>
>> hi
>>
>> i've try to add new telemetry data source with squid by
>>
>> following this tutorial :
>>
>>
>>
>>
>>
>> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>>
>>
>>
>>
>>
>> but when i start parser topology squid i've found
>> some error
>>
>> message like this :
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> i've check that file on hdfs. this is the value of
>> that file
>>
>> :
>>
>>
>>
>>
>>
>> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed}
>>
>> %{IPV4:ip_src_addr} %{WORD:action}/%{NUMBER:code}
>>
>> %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} -
>>
>> %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
>>
>> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>>
>>
>>
>>
>>
>> pls give me your suggestion.
>>
>>
>>
>>
>>
>> Thanks before.
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>>
>>
>>
>>
>>
>> Jon
>>
>>
>>
>> Sent from my mobile device
>>
>>
>>
>>
>>
>
>
>
>
>
> --
>
> Jon
>
>
>
> Sent from my mobile device
>
>
>
>
>
> --
>
> Jon
>
> Sent from my mobile device
>
Re: failed grok parser metron squid
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
OK, sorry just wanted to check the simple things. I'll try to take another
look at this later today when I'm at a computer unless it gets resolved by
then.
On Mon, Feb 13, 2017, 6:11 AM Youzha <yu...@gmail.com> wrote:
> yeah i've set the permission to 777
>
> On Mon, Feb 13, 2017 at 6:08 PM Zeolla@GMail.com <ze...@gmail.com> wrote:
>
> Did you check the permissions on the file are correct?
>
>
>
> On Mon, Feb 13, 2017, 5:59 AM tkg_cangkul <yu...@gmail.com> wrote:
>
>
>
>
>
>
>
>
>
>
>
> hi zeolla,
>
>
>
>
>
> this is the output of hdfs cat command :
>
>
>
>
>
>
>
>
>
>
>
> i've test it with this
>
> http://grokconstructor.appspot.com/do/match#result
>
>
> and the result was match
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 13/02/17 17:54, Zeolla@GMail.com
>
> wrote:
>
>
>
>
>
>
> You appear to be missing a final }. Does it work
>
> with the standard grok pattern from the tutorial? Can you show
>
> the output of:
>
>
> hadoop fs -cat /apps/metron/patterns/squid
>
>
> Jon
>
>
>
>
>
>
>
> On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul <yu...@gmail.com>
>
> wrote:
>
>
>
>
>
>
> hi
>
> i've try to add new telemetry data source with squid by
>
> following this tutorial :
>
>
>
>
>
>
> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>
>
>
>
>
> but when i start parser topology squid i've found some error
>
> message like this :
>
>
>
>
>
>
>
>
>
>
>
> i've check that file on hdfs. this is the value of that file
>
> :
>
>
>
>
>
> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed}
>
> %{IPV4:ip_src_addr} %{WORD:action}/%{NUMBER:code}
>
> %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} -
>
> %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
>
> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>
>
>
>
>
> pls give me your suggestion.
>
>
>
>
>
> Thanks before.
>
>
>
>
>
>
>
>
> --
>
>
>
>
>
>
> Jon
>
>
> Sent from my mobile device
>
>
>
>
>
>
>
>
>
> --
>
> Jon
>
>
> Sent from my mobile device
>
>
>
>
> --
Jon
Sent from my mobile device
Re: failed grok parser metron squid
Posted by Youzha <yu...@gmail.com>.
yeah i've set the permission to 777
On Mon, Feb 13, 2017 at 6:08 PM Zeolla@GMail.com <ze...@gmail.com> wrote:
> Did you check the permissions on the file are correct?
>
>
>
> On Mon, Feb 13, 2017, 5:59 AM tkg_cangkul <yu...@gmail.com> wrote:
>
>
>
>
>
>
>
>
>
>
>
> hi zeolla,
>
>
>
>
>
> this is the output of hdfs cat command :
>
>
>
>
>
>
>
>
>
>
>
> i've test it with this
>
> http://grokconstructor.appspot.com/do/match#result
>
>
> and the result was match
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 13/02/17 17:54, Zeolla@GMail.com
>
> wrote:
>
>
>
>
>
>
> You appear to be missing a final }. Does it work
>
> with the standard grok pattern from the tutorial? Can you show
>
> the output of:
>
>
> hadoop fs -cat /apps/metron/patterns/squid
>
>
> Jon
>
>
>
>
>
>
>
> On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul <yu...@gmail.com>
>
> wrote:
>
>
>
>
>
>
> hi
>
> i've try to add new telemetry data source with squid by
>
> following this tutorial :
>
>
>
>
>
>
> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>
>
>
>
>
> but when i start parser topology squid i've found some error
>
> message like this :
>
>
>
>
>
>
>
>
>
>
>
> i've check that file on hdfs. this is the value of that file
>
> :
>
>
>
>
>
> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed}
>
> %{IPV4:ip_src_addr} %{WORD:action}/%{NUMBER:code}
>
> %{NUMBER:bytes} %{WORD:method} %{NOTSPACE:url} -
>
> %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
>
> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>
>
>
>
>
> pls give me your suggestion.
>
>
>
>
>
> Thanks before.
>
>
>
>
>
>
>
>
> --
>
>
>
>
>
>
> Jon
>
>
> Sent from my mobile device
>
>
>
>
>
>
>
>
>
> --
>
> Jon
>
>
> Sent from my mobile device
>
>
>
>
>
Re: failed grok parser metron squid
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
Did you check the permissions on the file are correct?
On Mon, Feb 13, 2017, 5:59 AM tkg_cangkul <yu...@gmail.com> wrote:
> hi zeolla,
>
> this is the output of hdfs cat command :
>
>
>
> i've test it with this http://grokconstructor.appspot.com/do/match#result
> and the result was match
>
>
>
>
>
> On 13/02/17 17:54, Zeolla@GMail.com wrote:
>
> You appear to be missing a final }. Does it work with the standard grok
> pattern from the tutorial? Can you show the output of:
> hadoop fs -cat /apps/metron/patterns/squid
>
> Jon
>
> On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul <yu...@gmail.com> wrote:
>
> hi i've try to add new telemetry data source with squid by following this
> tutorial :
>
>
> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>
> but when i start parser topology squid i've found some error message like
> this :
>
>
>
> i've check that file on hdfs. this is the value of that file :
>
> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed} %{IPV4:ip_src_addr}
> %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes} %{WORD:method}
> %{NOTSPACE:url} - %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>
> pls give me your suggestion.
>
> Thanks before.
>
> --
>
> Jon
>
> Sent from my mobile device
>
>
> --
Jon
Sent from my mobile device
Re: failed grok parser metron squid
Posted by tkg_cangkul <yu...@gmail.com>.
hi zeolla,
this is the output of hdfs cat command :
i've test it with this http://grokconstructor.appspot.com/do/match#result
and the result was match
On 13/02/17 17:54, Zeolla@GMail.com wrote:
>
> You appear to be missing a final }. Does it work with the standard
> grok pattern from the tutorial? Can you show the output of:
> hadoop fs -cat /apps/metron/patterns/squid
>
> Jon
>
>
> On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul <yuza.rasfar@gmail.com
> <ma...@gmail.com>> wrote:
>
> hi i've try to add new telemetry data source with squid by
> following this tutorial :
>
> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>
> but when i start parser topology squid i've found some error
> message like this :
>
>
>
> i've check that file on hdfs. this is the value of that file :
>
> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed}
> %{IPV4:ip_src_addr} %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes}
> %{WORD:method} %{NOTSPACE:url} -
> %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>
> pls give me your suggestion.
>
> Thanks before.
>
> --
>
> Jon
>
> Sent from my mobile device
>
Re: failed grok parser metron squid
Posted by "Zeolla@GMail.com" <ze...@gmail.com>.
You appear to be missing a final }. Does it work with the standard grok
pattern from the tutorial? Can you show the output of:
hadoop fs -cat /apps/metron/patterns/squid
Jon
On Mon, Feb 13, 2017, 5:28 AM tkg_cangkul <yu...@gmail.com> wrote:
> hi i've try to add new telemetry data source with squid by following this
> tutorial :
>
>
> https://cwiki.apache.org/confluence/display/METRON/Adding+a+New+Telemetry+Data+Source
>
> but when i start parser topology squid i've found some error message like
> this :
>
>
>
> i've check that file on hdfs. this is the value of that file :
>
> SQUID_DELIMITED %{NUMBER:timestamp} %{INT:elapsed} %{IPV4:ip_src_addr}
> %{WORD:action}/%{NUMBER:code} %{NUMBER:bytes} %{WORD:method}
> %{NOTSPACE:url} - %{WORD:UNWANTED}\/%{IPV4:ip_dst_addr}
> %{WORD:UNWANTED}\/%{WORD:UNWANTED
>
> pls give me your suggestion.
>
> Thanks before.
>
--
Jon
Sent from my mobile device