You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "Justin Leet (JIRA)" <ji...@apache.org> on 2018/05/22 19:24:00 UTC
[jira] [Assigned] (METRON-1551) Profiler Should Not Use Java
Serialization
[ https://issues.apache.org/jira/browse/METRON-1551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Justin Leet reassigned METRON-1551:
-----------------------------------
Assignee: Nick Allen
> Profiler Should Not Use Java Serialization
> ------------------------------------------
>
> Key: METRON-1551
> URL: https://issues.apache.org/jira/browse/METRON-1551
> Project: Metron
> Issue Type: Bug
> Reporter: Nick Allen
> Assignee: Nick Allen
> Priority: Major
>
> When running the Profiler in a topology where serialization occurs, the following error happens. This can occur when the number of workers is greater than 1.
> The topology should not be using Java serialization for serializing tuple values as this will negatively impact performance.
> {code}
> 2018-05-09 10:48:35.136 o.a.s.d.executor [ERROR]
> java.lang.RuntimeException: java.lang.RuntimeException: java.io.NotSerializableException: org.apache.metron.common.configuration.profiler.ProfileResult
> at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:485) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:451) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.disruptor$consume_loop_STAR_$fn__7183.invoke(disruptor.clj:83) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.util$async_loop$fn__553.invoke(util.clj:484) [storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
> Caused by: java.lang.RuntimeException: java.io.NotSerializableException: org.apache.metron.common.configuration.profiler.ProfileResult
> at org.apache.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:41) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534) ~[kryo-3.0.3.jar:?]
> at org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.daemon.worker$mk_transfer_fn$transfer_fn__7805.invoke(worker.clj:193) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__7430.invoke(executor.clj:309) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.disruptor$clojure_handler$reify__7166.onEvent(disruptor.clj:40) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> ... 6 more
> Caused by: java.io.NotSerializableException: org.apache.metron.common.configuration.profiler.ProfileResult
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184) ~[?:1.8.0_162]
> at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) ~[?:1.8.0_162]
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) ~[?:1.8.0_162]
> at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) ~[?:1.8.0_162]
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[?:1.8.0_162]
> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[?:1.8.0_162]
> at org.apache.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:38) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40) ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534) ~[kryo-3.0.3.jar:?]
> at org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.daemon.worker$mk_transfer_fn$transfer_fn__7805.invoke(worker.clj:193) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__7430.invoke(executor.clj:309) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.disruptor$clojure_handler$reify__7166.onEvent(disruptor.clj:40) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:472) ~[storm-core-1.1.0.2.6.4.0-91.jar:1.1.0.2.6.4.0-91]
> ... 6 more
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)