Using Metron profiler

[Hema malini <nh...@gmail.com>]

Hi,

I was exploring Metron profiler and Metron MaaS. Is it possible to use
different models based on different profiler? For instance , user entity
behaviour analytics . How can I apply different end point for each user?

I am confused on why we are using MaaS in the parsing . Analytics should be
applied after enrichment in the Metron. What if I need to use enriched
fields for Machine learning models. I couldn't understand the exact purpose
of applying the models in parsing.Please correct me if I am wrong in
understanding the functionality of MaaS.


Thanks and Regards,
Hema

Re: Using Metron profiler

[Hema malini <nh...@gmail.com>]

Hi Nick,

Thanks for the clarification ..I was using field transformation .
Definitely will change that .

In MaaS, sometimes it is not able to accept the request when we have models
like pytorch. Though I allocate 2gb for the models it is not able to serve
the request properly. When I checked the logs in the yarn for that
application id , it is receiving null request . What is the recommended way
to load model file in  pickle format to be hosted in yarn and how can I
host multiple model.

As of now using Rest_get for the hosted models. Using Rest_get is fine or
will it cause any issues.

Thanks and regards,
Hema



On Wed, 1 Apr, 2020, 7:23 PM Nick Allen, <ni...@nickallen.org> wrote:

>
> > I am confused on why we are using MaaS in the parsing . Analytics
> should be applied after enrichment in the Metron.
>
> I am not sure where you are getting that understanding. In most cases,
> analytics would usually be applied during the enrichment phase.  It would
> not be applied during parsing.
>
> It would certainly be possible to perform some enrichment functions during
> parsing as part of what is called "field transformations", but it is not
> advised.
>
>
> > How can I apply different end point for each user?
>
> In enrichment you would just create a Stellar expression that constructs
> the endpoint (the model that you want to call) based on the username.
> Assuming you have a message containing a field called "username", you could
> do something like this. This is just one way to do it.
>
> user_risk_score := REST_GET( FORMAT("http://hostname/models/risk?user=%s",
> username) )
>
>
>
> > Please correct me if I am wrong in understanding the functionality of
> MaaS.
>
> MaaS is just one way to host machine learning models on YARN. Metron gives
> you the tools to call out to models hosted on MaaS or wherever they may be
> hosted through a simple HTTP request.
>
>
> Hope this helps some.
>
>
> On Wed, Apr 1, 2020 at 1:34 AM Hema malini <nh...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I was exploring Metron profiler and Metron MaaS. Is it possible to use
>> different models based on different profiler? For instance , user entity
>> behaviour analytics . How can I apply different end point for each user?
>>
>> I am confused on why we are using MaaS in the parsing . Analytics should
>> be applied after enrichment in the Metron. What if I need to use enriched
>> fields for Machine learning models. I couldn't understand the exact purpose
>> of applying the models in parsing.Please correct me if I am wrong in
>> understanding the functionality of MaaS.
>>
>>
>> Thanks and Regards,
>> Hema
>>
>>
>>
>>

Re: Using Metron profiler

[Nick Allen <ni...@nickallen.org>]

> I am confused on why we are using MaaS in the parsing . Analytics should
be applied after enrichment in the Metron.

I am not sure where you are getting that understanding. In most cases,
analytics would usually be applied during the enrichment phase.  It would
not be applied during parsing.

It would certainly be possible to perform some enrichment functions during
parsing as part of what is called "field transformations", but it is not
advised.


> How can I apply different end point for each user?

In enrichment you would just create a Stellar expression that constructs
the endpoint (the model that you want to call) based on the username.
Assuming you have a message containing a field called "username", you could
do something like this. This is just one way to do it.

user_risk_score := REST_GET( FORMAT("http://hostname/models/risk?user=%s",
username) )



> Please correct me if I am wrong in understanding the functionality of
MaaS.

MaaS is just one way to host machine learning models on YARN. Metron gives
you the tools to call out to models hosted on MaaS or wherever they may be
hosted through a simple HTTP request.


Hope this helps some.


On Wed, Apr 1, 2020 at 1:34 AM Hema malini <nh...@gmail.com> wrote:

> Hi,
>
> I was exploring Metron profiler and Metron MaaS. Is it possible to use
> different models based on different profiler? For instance , user entity
> behaviour analytics . How can I apply different end point for each user?
>
> I am confused on why we are using MaaS in the parsing . Analytics should
> be applied after enrichment in the Metron. What if I need to use enriched
> fields for Machine learning models. I couldn't understand the exact purpose
> of applying the models in parsing.Please correct me if I am wrong in
> understanding the functionality of MaaS.
>
>
> Thanks and Regards,
> Hema
>
>
>
>