You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/11/18 15:06:58 UTC
[jira] [Commented] (METRON-575) State from different profiles can
be co-mingled incorrectly
[ https://issues.apache.org/jira/browse/METRON-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676910#comment-15676910 ]
ASF GitHub Bot commented on METRON-575:
---------------------------------------
GitHub user nickwallen opened a pull request:
https://github.com/apache/incubator-metron/pull/362
METRON-575 State from different profiles can be co-mingled incorrectly
Added a keyed cache to the ProfileBuilderBolt so that it can maintain state for multiple profiles. Previously, the same StellarExecutor was being used across multiple different Profile/Entity pairs, which was incorrect.
The cache is also used to define a finite lifetime for a profile. A new property was added to allow the user to control the lifetime of a profile.
> `profiler.profile.ttl`: If a message has not been applied to a Profile in this number of periods, the Profile will be forgotten and its resources will be cleaned up. If the profile TTL is `10` and the period duration is `15 minutes`, then any Profile that has not had a message applied in the previous 10 periods, which is the same as `150 minutes`, will be eligible for deletion.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/incubator-metron METRON-575
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/362.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #362
----
commit 8d94fc5bcaaf1a8537639e59392d1aed8091a9ed
Author: Nick Allen <ni...@nickallen.org>
Date: 2016-11-17T23:27:36Z
METRON-575 State from different profiles can be co-mingled incorrectly
----
> State from different profiles can be co-mingled incorrectly
> -----------------------------------------------------------
>
> Key: METRON-575
> URL: https://issues.apache.org/jira/browse/METRON-575
> Project: Metron
> Issue Type: Bug
> Reporter: Nick Allen
> Assignee: Nick Allen
>
> The ProfileBuilderBolt incorrectly assumes that it will only ever see a single [profile, entity] pair. The bolt maintains a single StellarExecutor that is responsible for executing the init, update, result expressions. This assumption is incorrect as Storm's field grouping only guarantees that the same profile/entity pairs will go to the same task. Storm does not guarantee that a task only receives a single profile/entity pair.
> The easiest fix is to maintain a cache that maps a profile/entity to its state. This would follow what is currently done in the Join bolt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)