You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Dmitriy Ivanov <da...@gridfore.com> on 2018/02/20 12:04:43 UTC
Ignite Streamer usage in UriDeploymentSpi
Hi. I use UriDeploymentSpi to deploy users code and ComputeTasks.
Simple ComputeTasks work well, but in ComputeTask with IgniteStreamer I
have ClassNotFoundException.
There is users transformations code in application which should be applied
to data on each node.
So I put transform execution into StreamReceiver.
Each transform is a user java class which was deployed along with
ComputeTask in one gar file
In StreamReceiver constructor or readResovle method I am trying to get
instance of user transform class but I have ClassNotFoundException.
I try to get instanse that way:
try {
try {
return (Transform) Class.forName(transform, true,
Thread.currentThread().getContextClassLoader()).newInstance();
} catch (ClassNotFoundException e) {
return (Transform) Class.forName(transform).newInstance();
}
}
This is my stackTrace:
SEVERE: Failed to unmarshal message
[nodeId=0000493d-03cd-4ccb-85e5-3c4b17793d7f, req=DataStreamerRequest
[reqId=77, cacheName=ODS_CDR.MSC_CDR, ignoreDepOwnership=true,
skipStore=false, keepBinary=true, depMode=SHARED,
sampleClsName=wf.LoadMscCdr.ETLTransform, userVer=0, ldrParticipants=null,
clsLdrId=019b403b161-0000493d-03cd-4ccb-85e5-3c4b17793d7f,
forceLocDep=false, topVer=AffinityTopologyVersion [topVer=5,
minorTopVer=2], partId=-2147483648]]
class org.apache.ignite.IgniteCheckedException: Failed to deserialize
object [typeName=com.gridfore.dmp.etl.receivers.impl.DataBinaryReceiver]
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9859)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:289)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:59)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:89)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at
org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:505)
at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
deserialize object
[typeName=com.gridfore.dmp.etl.receivers.impl.DataBinaryReceiver]
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:874)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1762)
at
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1714)
at
org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:310)
at
org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9853)
... 9 more
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed to
execute readResolve() method on
com.gridfore.dmp.etl.receivers.impl.DataBinaryReceiver@3a40ad0e
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:866)
... 15 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:855)
... 15 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
wf.LoadMscCdr.ETLTransform
at
com.gridfore.dmp.workflow.job.TaskAdapter.getTransform(TaskAdapter.java:18)
at
com.gridfore.dmp.workflow.job.source.SourceTaskAdapter.getTransform(SourceTaskAdapter.java:127)
at
com.gridfore.dmp.etl.transform.CompileAllTransforms.compile(CompileAllTransforms.java:28)
at
com.gridfore.dmp.etl.receivers.BinaryReceiver.initTransforms(BinaryReceiver.java:47)
at
com.gridfore.dmp.etl.receivers.BinaryReceiver.readResolve(BinaryReceiver.java:60)
... 19 more
Caused by: java.lang.ClassNotFoundException: wf.LoadMscCdr.ETLTransform
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
com.gridfore.dmp.workflow.job.TaskAdapter.getTransform(TaskAdapter.java:15)
... 23 more
I try to use IgniteStreamer deployClass() method but it doesn't help:
Optional.ofNullable(sourceTaskAdapter.getTransform())
.ifPresent(it -> dataStreamer.deployClass(it.getClass()));
Is it possible in my particular case? Maybe it should be used for another
cases?
PeerClassloading is on.
Could you help me with that problem? How can I instansiate my users
classes in StreamReceiver?
Dmitry.
Re: Ignite Streamer usage in UriDeploymentSpi
Posted by daivanov <da...@gridfore.com>.
Hi, Slava. Thanks for your answer.
Yes, it looks similar.
But I think there are some difference.
Because I use UriDeploymentSpi i assume that I will have proper classloader
in each node for my executed ComputeTask.
In future I will remove peerClassloading because I load classes in each node
by UriDeploymentSpi and user classes located on each node.
So my main problem is how to get proper classloader on Receiver side for
deployed Gar unit when I have it on Streamer
And one more question: for which case IgniteStreamer deployClass method
should be used and why it takes only one argument?
Dmitry.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ignite Streamer usage in UriDeploymentSpi
Posted by "slava.koptilin" <sl...@gmail.com>.
Hi Dmitry,
It looks like the issue you described is similar to the following
https://issues.apache.org/jira/browse/IGNITE-3935
The fix will be available in ApacheIgnite 2.4.
In meanwhile, could you please check your case with the latest code from the
master branch?
Thanks!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/