You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/06/22 20:35:16 UTC
[jira] [Commented] (METRON-253) Reloading parser configurations
from zookeeper doesn't call init()
[ https://issues.apache.org/jira/browse/METRON-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345111#comment-15345111 ]
ASF GitHub Bot commented on METRON-253:
---------------------------------------
GitHub user cestella opened a pull request:
https://github.com/apache/incubator-metron/pull/170
METRON-253: Reloading parser configurations from zookeeper doesn't call init()
This results in a NPE if you do not specify an output column (this output column gets inferred as part of init())
To validate this:
* Get a running topology
* Run `/usr/metron/0.2.0BETA/bin/zk_load_configs.sh -m PUSH -i /usr/metron/0.2.0BETA/config/zookeeper -z node1:2181`
* Wait a full 2 minutes
* Check the `parser_error` topic via the kafka console consumer and ensure you don't see any weird NPEs
The problem here is that init() is not getting run on the sensor parser config, so some syntactic sugar in the FieldTransformer isn't getting updated. Specifically, if you only specify `input` and not `output`, then it will NPE.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cestella/incubator-metron yaf_npe
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/170.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #170
----
commit b0d391c143e3f14a16cadcac6ab91e6863b1be8d
Author: cstella <ce...@gmail.com>
Date: 2016-06-22T20:27:31Z
METRON-253: Reloading parser configurations from zookeeper doesn't call init()
----
> Reloading parser configurations from zookeeper doesn't call init()
> ------------------------------------------------------------------
>
> Key: METRON-253
> URL: https://issues.apache.org/jira/browse/METRON-253
> Project: Metron
> Issue Type: Bug
> Reporter: Casey Stella
> Assignee: Casey Stella
>
> This results in a NPE if you do not specify an output column (this output column gets inferred as part of init()):
> {"exception":"java.lang.NullPointerException","hostname":"ip-10-0-0-119.us-west-2.compute.internal","stack":"java.lang.NullPointerException\n\tat org.apache.metron.common.field.transformation.SimpleFieldTransformation.map(SimpleFieldTransformation.java:38)\n\tat org.apache.metron.common.configuration.FieldTransformer.transform(FieldTransformer.java:107)\n\tat org.apache.metron.common.configuration.FieldTransformer.transformAndUpdate(FieldTransformer.java:112)\n\tat org.apache.metron.parsers.bolt.ParserBolt.execute(ParserBolt.java:123)\n\tat backtype.storm.daemon.executor$fn__5495$tuple_action_fn__5497.invoke(executor.clj:670)\n\tat backtype.storm.daemon.executor$mk_task_receiver$fn__5418.invoke(executor.clj:426)\n\tat backtype.storm.disruptor$clojure_handler$reify__4994.onEvent(disruptor.clj:58)\n\tat backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)\n\tat backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)\n\tat backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)\n\tat backtype.storm.daemon.executor$fn__5495$fn__5508$fn__5559.invoke(executor.clj:808)\n\tat backtype.storm.util$async_loop$fn__543.invoke(util.clj:475)\n\tat clojure.lang.AFn.run(AFn.java:22)\n\tat java.lang.Thread.run(Thread.java:745)\n","time":1466617043534,"rawMessage_bytes":[50,48,49,54,45,48,54,45,50,50,32,49,55,58,51,53,58,53,49,46,56,56,56,124,50,48,49,54,45,48,54,45,50,50,32,49,55,58,51,55,58,48,57,46,55,49,51,124,32,32,55,55,46,56,50,53,124,32,32,32,48,46,49,55,51,124,32,32,54,124,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,49,57,50,46,49,54,56,46,49,51,56,46,49,53,56,124,52,57,50,48,54,124,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,57,53,46,49,54,51,46,49,50,49,46,50,48,52,124,32,32,32,56,48,124,32,32,32,32,32,32,32,83,124,32,32,32,32,32,65,80,82,124,32,32,32,32,32,32,65,83,124,32,32,32,32,32,32,65,80,124,97,56,48,100,97,97,54,101,124,98,57,51,97,48,52,100,102,124,48,48,48,124,48,48,48,124,32,32,32,32,32,32,49,48,124,32,32,32,32,50,50,54,53,124,32,32,32,32,32,32,49,50,124,32,32,32,32,55,49,57,52,124,32,32,32,32,48,124],"message":null,"rawMessage":"2016-06-22 17:35:51.888|2016-06-22 17:37:09.713| 77.825| 0.173| 6| 192.168.138.158|49206| 95.163.121.204| 80| S| APR| AS| AP|a80daa6e|b93a04df|000|000| 10| 2265| 12| 7194| 0|","source.type":"yaf_error"}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)