You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by mp <mj...@gmail.com> on 2015/11/02 08:59:37 UTC

Re: Distributed queue problem with peerClassLoading enabled

Hi Artem,

Thanks for your investigation! I will then eagerly wait for the resolution
of this bug.

Unfortunately the workarounds you suggested probably are not applicable to
my main use case for Ignite, unless I misunderstood them.

Here's my main use case (only for the development environment):
I'd like the developers to be able to quickly verify/test their custom
ad-hoc code on CPU-intensive computations (for this I can dedicate a couple
of powerful multi-core, huge RAM machines). Therefore I assume that
peerClassLoading is a must. I just want their modified/new classes to be
immediately running on the cluster, whenever they run their tests.
I tried the CONTINUOUS deployment mode as you suggested. Unfortunately, in
this deployment mode, once the class is deployed on the server node, it
seems that any changes in the class code on the client node will not be
seen on the server. To test this I repeated the same "steps to reproduce"
from my previous mail, with the CONTINUOUS deployment mode on both the
server and the client node. As you wrote, in this mode the 2nd client test
passed. Then I changed the Task class on the client node to write a
different message to the console, eg:

    System.err.println("task called 2");

When the test runs, the new message is printed only on the client node, but
the server node still has the old version of the class and prints out the
old, obsolete message. Then I tried to change the serialVersionUID of the
class to somehow "force" the redeployment on the class but this caused a
problem on the client side, when attempting to fill the queue (please see
below for stack trace). This is especially strange because the queue was
empty, so how come the old serialVersionUID prevents filling the queue? I
thought that maybe the queue, even though empty, preserved some internal
state and this caused the problem. To check this I tried to call "close" on
the queue (if exists), and then recreating it again before filling:

      IgniteQueue<Task> queue = ignite.queue("myTaskQueue", 0, null);
      if (queue != null)
      {
        queue.close();
      }
      queue = ignite.queue("myTaskQueue", 0, new CollectionConfiguration());
      queue.clear();
      queue.addAll(Collections.nCopies(100, new Task()));


Any hints/suggestions?

Thanks!
-Mateusz


And here's the stack trace of the problem when attempting to fill the queue
(I think you could easily reproduce it on your side):

Exception in thread "main" class org.apache.ignite.IgniteException: Failed
to find class with given class loader for unmarshalling (make sure same
version of all classes are available on all nodes or enable
peer-class-loading):
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader@545d1fea
at
org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:881)
at
org.apache.ignite.internal.processors.datastructures.GridAtomicCacheQueueImpl.addAll(GridAtomicCacheQueueImpl.java:186)
at
org.apache.ignite.internal.processors.datastructures.GridCacheQueueProxy.addAll(GridCacheQueueProxy.java:146)
at ignitetest.Test.main(Test.java:31)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to find
class with given class loader for unmarshalling (make sure same version of
all classes are available on all nodes or enable peer-class-loading):
org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CacheClassLoader@545d1fea
at
org.apache.ignite.marshaller.optimized.OptimizedMarshaller.unmarshal(OptimizedMarshaller.java:246)
at
org.apache.ignite.internal.processors.cacheobject.IgniteCacheObjectProcessorImpl.unmarshal(IgniteCacheObjectProcessorImpl.java:101)
at
org.apache.ignite.internal.processors.cache.portable.CacheObjectPortableProcessorImpl.unmarshal(CacheObjectPortableProcessorImpl.java:730)
at
org.apache.ignite.internal.processors.cache.CacheObjectImpl.finishUnmarshal(CacheObjectImpl.java:104)
at
org.apache.ignite.internal.processors.cache.GridCacheMessage.finishUnmarshalCacheObjects(GridCacheMessage.java:534)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateRequest.finishUnmarshal(GridNearAtomicUpdateRequest.java:578)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1034)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:269)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:198)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:77)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:160)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:811)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$1500(GridIoManager.java:106)
at
org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:774)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: Optimized stream class
checksum mismatch (is same version of marshalled class present on all
nodes?) [expected=1, actual=2, cls=class ignitetest.Task]
at
org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.verifyChecksum(OptimizedClassDescriptor.java:824)
at
org.apache.ignite.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:806)
at
org.apache.ignite.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:315)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:365)
at
org.apache.ignite.marshaller.optimized.OptimizedMarshaller.unmarshal(OptimizedMarshaller.java:240)
... 16 more



On Fri, Oct 30, 2015 at 6:02 PM, Artem Shutak <as...@gridgain.com> wrote:

> Mateusz,
>
> I reproduced the issue and filed a bug:
> https://issues.apache.org/jira/browse/IGNITE-1823. You can track a
> progress
> of the issue there. Also you are welcome to fix it by yourself and provide
> a
> patch.
>
> I can suggest you next workarounds:
> - Use another DeploymentMode. For example in this case Ignite works fine
> with CONTINUOUS deployment mode.
> - Don't use peer class loading and add jars with custom tasks to classpath
> of server nodes.
>
> Thanks,
> Artem.
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1787.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Mateusz,

In general Compute Engine may help you with this 
(https://apacheignite.readme.io/v1.4/docs/compute-grid).
It takes peer class loading and different deployment modes into account.

--
Denis

On 11/3/2015 2:05 PM, mp wrote:
> Hi Denis,
>
> Thanks a lot for the clarifications.
>
> So, is there any way in Ignite to achieve my goal: different versions 
> of the same class running on the cluster (they can be deployed at the 
> same time from different client nodes)?
> If yes, I can think of a way in which I can achieve my goals without 
> using the distributed queue.
>
> Cheers,
> -Mateusz
>
>
> On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <dmagda@gridgain.com 
> <ma...@gridgain.com>> wrote:
>
>     Hi Mateusz,
>
>     Unfortunately I don't see any way to specify userVersion
>     programmatically.
>     In any case CONTINIOUS and SHARED mode is not suitable for your
>     task because
>     the developers may want to have different version of the same
>     class. Please
>     refer to DeploymentMode documentation for more info on different
>     mode types.
>
>     Do you really need to use IgniteQueue in your tests because, yes,
>     neither
>     PRIVATE nor ISOLATED modes are supported for user caches.
>
>     --
>     Denis
>
>
>
>     --
>     View this message in context:
>     http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
>     Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
>


Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Hi Mateusz,

Please see inline

On 2/17/2016 11:30 AM, mp wrote:
> Denis,
>
> Please see below for answers.
>
> Cheers,
> -Mateusz
>
> On Tue, Feb 16, 2016 at 10:18 PM, Denis Magda <dmagda@gridgain.com 
> <ma...@gridgain.com>> wrote:
>
>     Hi Mateusz,
>
>     I've revisited the whole discussion from the beginning and should
>     say that the solution based on the distributed queue won't work
>     for you even if all the issues listed below are fixed.
>
>     Presently you're placing in the queue tasks coming form different
>     nodes with different class versions. Even if the tasks are stored
>     in the queue in a binary form they have to be deserialized to an
>     original form before execution. This will lead to
>     ClassNotFoundExceptions in your case.
>
>
> No, I'm not filling the queue with different class versions. My test 
> case is quite simple. Please check the original description specified 
> in 
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1780.html
>
> In 1.4.0 it works as follows:
>
> 1. A server node is started.
> 2. Client node (which acts as a server as well) is started, creates a 
> queue, fills it with 100 tasks, and both nodes are polling tasks from 
> the queue. When the queue is empty (all tasks called), the test ends. 
> At this point the client node leaves the cluster as well.
> 3. Then you run the same test again *without any code modifications*, 
> ie, exactly the same class on the same client node. I assume that even 
> if the server node already cached the class during the 1st test run, 
> it should be no problem for the 2nd test run. But unfortunately it 
> fails with "ignitetest.Task cannot be cast to ignitetest.Task", which 
> I believe is due to class loaders being different.
>
> In 1.5.0 even the 1st run of the test fails due to a problem you 
> reported in https://issues.apache.org/jira/browse/IGNITE-2339
>
It will pass the 1st run if you switch to OptimizedMarshaller that is a 
default one in 1.4.0. Just set IgniteConfiguration.setMarshaller(new 
OptimizedMarshaller()) and you will see that everything will work the 
same in 1.5.0.
However the second run will fail as in case with 1.4.0.

> Actually, what I would like to achieve is the following (in 3 steps):
>
> 1. Have my original test pass, regardless of how many times I call the 
> test. This is a baseline.
> 2. Modify my test so that after I finish each test run, I can *modify* 
> the code of Task class, and my change is reflected in the subsequent 
> test run, ie, the server node will process the Task class with its new 
> definition.
> 3. Multiple client nodes can have their own Task class definitions 
> that run on the cluster at the same time, assuming that they create 
> *different* queues for their tasks. Actually they will also broadcast 
> their own IgniteCallables to the cluster (note that in particular they 
> can broadcast the same IgniteCallable class, each in its own different 
> version). But I'm always assuming that a given queue is 
> created/maintained/filled by one client node therefore it will never 
> mix objects with different versions of the same class.
>
>
>     My suggestion is the same is it was before. Avoid usage of the
>     distributed queue but rather start sending ignite compute tasks
>     for execution from the beginning.
>
>     Will this work for you?
>
>
> In my setup, it is simply more logical to model the computation as 
> IgniteCallables being broadcast to the cluster and then they pull 
> tasks from a distributed queue. Actually, the distributed queue (and 
> other distributed structures provided by Ignite) was one of main 
> reasons that attracted me to try it out. And I really like them and 
> their simple usage :)
> Without the use of distributed structures I think managing my 
> computations would probably be more cumbersome. But I will of course 
> think about it, and maybe I will come up with a suitable model.

Seems that I got the idea why you had decided to use a distributed queue 
in this particular case. This way you try to send several jobs for 
execution at once and wants them to be executed uniformly by all nodes 
that are available.
Is this assumption correct?

If I'm right then I will suggest you create your own ComputeTask that 
will be split and mapped to "less" loaded nodes using a load balancer.
I've prepared a code sample for you [1]. Please go through it. If 
anything is unclear then let me know.

Here [2] you can read more on load balancing that is implemented and 
used by compute engine out of the box.

[1] https://gist.github.com/dmagda/3a78e7d01d9fffd7bc66
[2] https://apacheignite.readme.io/docs/load-balancing

Regards,
Denis

>
>
>     --
>     Denis
>
>
>
>     On 2/16/2016 11:33 AM, mp wrote:
>>     Hi Denis,
>>
>>     Many thanks! I look forward to 1.6 then.
>>     Please also consider the following statement made by Dmitriy on
>>     Nov 03, 2015 (see his message in the thread):
>>
>>     "With that in mind, we will be removing the requirement for
>>     caches to work only with SHARED and CONTINUOUS deployment modes,
>>     so you will be able to use PRIVATE or ISOLATED deployment modes
>>     to deploy your computations."
>>
>>     As far as I understand, the above planned change is not covered
>>     by any Jira ticket.
>>
>>     Cheers,
>>     -Mateusz
>>
>>
>>
>>     On Fri, Feb 12, 2016 at 10:35 PM, Denis Magda
>>     <dmagda@gridgain.com <ma...@gridgain.com>> wrote:
>>
>>         Hi Mateusz,
>>
>>         I assigned both tickets that you have problems with on
>>         myself. They will be fixed as a part of the next release.
>>         https://issues.apache.org/jira/browse/IGNITE-2339
>>         https://issues.apache.org/jira/browse/IGNITE-1823
>>
>>         There is one more issue that was reproduced locally and
>>         refers to unexpected cache undeployment when the binary
>>         marshaller is used.
>>         https://issues.apache.org/jira/browse/IGNITE-2647
>>
>>         Thanks for your patience and still showing the interest in
>>         Ignite.
>>
>>         Regards,
>>         Denis
>>
>>
>>         On 2/12/2016 4:41 PM, mp wrote:
>>>         Hi Denis,
>>>
>>>         But my test still fails in version 1.5 with default (ie,
>>>         binary) marshaller. See my message from January 7, and your
>>>         reply in which you mentioned a new Jira ticked for a bug
>>>         concerning the new binary marshaller:
>>>         https://issues.apache.org/jira/browse/IGNITE-2339
>>>
>>>         Basically, my test case (see
>>>         https://issues.apache.org/jira/browse/IGNITE-1823 ) fails in
>>>         all of the scenarios I tried:
>>>
>>>         1. Binary marshaller + default deployment mode
>>>         2. Binary marshaller + shared deployment mode
>>>         3. Binary marshaller + private deployment mode
>>>         4. Optimized marshaller + default deployment mode
>>>         5. Optimized marshaller + shared deployment mode
>>>         6. Optimized marshaller + private deployment mode
>>>
>>>         Would you have any hint/advice on how I could proceed? Is
>>>         there any chance of fixing the issues related to my test case?
>>>
>>>         Thanks for your help,
>>>         -Mateusz
>>>
>>>
>>>         On Wed, Feb 10, 2016 at 4:46 PM, Denis Magda
>>>         <dmagda@gridgain.com <ma...@gridgain.com>> wrote:
>>>
>>>             Hi Mateusz,
>>>
>>>             In version 1.5 we released the binary objects [1] format
>>>             that allows to store cache in class version independent
>>>             form. Thus you don't need to have any classes on server
>>>             side.
>>>             This ability allows dynamic change to an objects
>>>             structure, and even allows multiple clients with
>>>             different versions of class definitions to co-exist.
>>>
>>>             In my understanding if you switch to this format you
>>>             will be able to support your use case.
>>>
>>>             If something is unclear don't hesitate to ask.
>>>
>>>             [1] https://apacheignite.readme.io/docs/binary-marshaller
>>>
>>>             --
>>>             Denis
>>>
>>>
>>>             On 2/10/2016 4:06 PM, mp wrote:
>>>>             Hi Denis,
>>>>
>>>>             Thanks for your reply.
>>>>             So, summing up, it seems that in the context of my use
>>>>             case, version 1.5 does not differ from 1.4? Which means
>>>>             that I still cannot achieve my goal: different versions
>>>>             of the same class (from different clients) running on
>>>>             the cluster at the same time?
>>>>
>>>>             As far as I understand this involves:
>>>>             1. https://issues.apache.org/jira/browse/IGNITE-1823
>>>>             2. https://issues.apache.org/jira/browse/IGNITE-2339
>>>>             3. Removing the requirement for caches to work only
>>>>             with SHARED and CONTINUOUS deployment modes (this was
>>>>             announced by Dmitriy in
>>>>             http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1829.html
>>>>             )
>>>>
>>>>             Is there any chance the above use case will be possible
>>>>             in near future (any upcoming version)?
>>>>
>>>>             I really like the API and concept of Ignite. If only I
>>>>             could achieve the above scenario...
>>>>
>>>>             Cheers,
>>>>             -Mateusz
>>>>
>>>>
>>>>
>>>>             On Thu, Jan 7, 2016 at 5:25 PM, Denis Magda
>>>>             <dmagda@gridgain.com <ma...@gridgain.com>> wrote:
>>>>
>>>>                 Mateusz,
>>>>
>>>>                 It doesn’t work for now because peerClassLoading
>>>>                 doesn’t work for objects that are stored in the
>>>>                 binary format in a cache.
>>>>                 Since starting from 1.5 BinaryMarshaller is a
>>>>                 default one all the objects are stored in a such
>>>>                 format in caches by default.
>>>>
>>>>                 If you prefer to turn off such a behavior you can
>>>>                 set IgniteConfiguration.setMarshaller(new
>>>>                 OptimizedMarshaller()) for every node and your test
>>>>                 should work as before.
>>>>
>>>>                 —
>>>>                 Denis
>>>>
>>>>>                 On 7 янв. 2016 г., at 17:09, mp <mjjp00@gmail.com
>>>>>                 <ma...@gmail.com>> wrote:
>>>>>
>>>>>                 Hello Denis,
>>>>>
>>>>>                 Thanks a lot for your reply!
>>>>>                 Concerning point 2: does it mean that
>>>>>                 "peerClassLoading" simply does not work in 1.5?
>>>>>                 It used to work (partially) in 1.4 (details
>>>>>                 described earlier in the message thread).
>>>>>
>>>>>                 Cheers,
>>>>>                 -Mateusz
>>>>>
>>>>>
>>>>>
>>>>>                 On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda
>>>>>                 <dmagda@gridgain.com <ma...@gridgain.com>>
>>>>>                 wrote:
>>>>>
>>>>>                     Hi Mateusz,
>>>>>
>>>>>                     1. It seems that distributed cache is still
>>>>>                     *not* available in
>>>>>                     PRIVATE/ISOLATED modes. Is this correct?
>>>>>
>>>>>                     Right, it hasn't been fixed yet. I've just
>>>>>                     followed up the related discussion on the dev
>>>>>                     list. Please follow it to see the most
>>>>>                     up-to-date information
>>>>>                     http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
>>>>>
>>>>>                     2. When I run my simple test code in the
>>>>>                     default SHARED mode (the same as
>>>>>                     specified in
>>>>>                     https://issues.apache.org/jira/browse/IGNITE-1823
>>>>>                     jira issue),
>>>>>                     I still get an error. However the cause
>>>>>                     exception seems to be different.
>>>>>                     Please see attached server log.
>>>>>
>>>>>                     The reason is that there is an attempt to
>>>>>                     deserialize a binary object stored on a server
>>>>>                     node and the server node doesn't have object's
>>>>>                     class definition in its class path.
>>>>>                     I've opened a ticket
>>>>>                     https://issues.apache.org/jira/browse/IGNITE-2339
>>>>>
>>>>>                     As a workaround you can put a class definition
>>>>>                     on server's class path and the problem will
>>>>>                     disappear.
>>>>>
>>>>>                     Regards,
>>>>>                     Denis
>>>>>
>>>>>                     On 1/7/2016 1:30 PM, mjjp wrote:
>>>>>
>>>>>                         Hello,
>>>>>
>>>>>                         I have just downloaded 1.5.0-final to
>>>>>                         check if my problem has been resolved.
>>>>>                         Either I'm doing something wrong, or
>>>>>                         version 1.5 has the same behavior in
>>>>>                         this context:
>>>>>
>>>>>                         1. It seems that distributed cache is
>>>>>                         still *not* available in
>>>>>                         PRIVATE/ISOLATED modes. Is this correct?
>>>>>
>>>>>                         2. When I run my simple test code in the
>>>>>                         default SHARED mode (the same as
>>>>>                         specified in
>>>>>                         https://issues.apache.org/jira/browse/IGNITE-1823
>>>>>                         jira issue),
>>>>>                         I still get an error. However the cause
>>>>>                         exception seems to be different.
>>>>>                         Please see attached server log.
>>>>>
>>>>>                         Would you be able to check the attached
>>>>>                         log to verify if this is an expected
>>>>>                         behavior in 1.5?
>>>>>
>>>>>                         Cheers,
>>>>>                         -Mateusz
>>>>>
>>>>>                         ignite-fd14d572.log
>>>>>                         <http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                         --
>>>>>                         View this message in context:
>>>>>                         http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
>>>>>                         Sent from the Apache Ignite Users mailing
>>>>>                         list archive at Nabble.com
>>>>>                         <http://nabble.com>.
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>


Re: Distributed queue problem with peerClassLoading enabled

Posted by mp <mj...@gmail.com>.
Denis,

Please see below for answers.

Cheers,
-Mateusz

On Tue, Feb 16, 2016 at 10:18 PM, Denis Magda <dm...@gridgain.com> wrote:

> Hi Mateusz,
>
> I've revisited the whole discussion from the beginning and should say that
> the solution based on the distributed queue won't work for you even if all
> the issues listed below are fixed.
>
> Presently you're placing in the queue tasks coming form different nodes
> with different class versions. Even if the tasks are stored in the queue in
> a binary form they have to be deserialized to an original form before
> execution. This will lead to ClassNotFoundExceptions in your case.
>

No, I'm not filling the queue with different class versions. My test case
is quite simple. Please check the original description specified in
http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1780.html

In 1.4.0 it works as follows:

1. A server node is started.
2. Client node (which acts as a server as well) is started, creates a
queue, fills it with 100 tasks, and both nodes are polling tasks from the
queue. When the queue is empty (all tasks called), the test ends. At this
point the client node leaves the cluster as well.
3. Then you run the same test again *without any code modifications*, ie,
exactly the same class on the same client node. I assume that even if the
server node already cached the class during the 1st test run, it should be
no problem for the 2nd test run. But unfortunately it fails with
"ignitetest.Task cannot be cast to ignitetest.Task", which I believe is due
to class loaders being different.

In 1.5.0 even the 1st run of the test fails due to a problem you reported
in https://issues.apache.org/jira/browse/IGNITE-2339

Actually, what I would like to achieve is the following (in 3 steps):

1. Have my original test pass, regardless of how many times I call the
test. This is a baseline.
2. Modify my test so that after I finish each test run, I can *modify* the
code of Task class, and my change is reflected in the subsequent test run,
ie, the server node will process the Task class with its new definition.
3. Multiple client nodes can have their own Task class definitions that run
on the cluster at the same time, assuming that they create *different*
queues for their tasks. Actually they will also broadcast their own
IgniteCallables to the cluster (note that in particular they can broadcast
the same IgniteCallable class, each in its own different version). But I'm
always assuming that a given queue is created/maintained/filled by one
client node therefore it will never mix objects with different versions of
the same class.



>
> My suggestion is the same is it was before. Avoid usage of the distributed
> queue but rather start sending ignite compute tasks for execution from the
> beginning.
>
> Will this work for you?
>

In my setup, it is simply more logical to model the computation as
IgniteCallables being broadcast to the cluster and then they pull tasks
from a distributed queue. Actually, the distributed queue (and other
distributed structures provided by Ignite) was one of main reasons that
attracted me to try it out. And I really like them and their simple usage :)
Without the use of distributed structures I think managing my computations
would probably be more cumbersome. But I will of course think about it, and
maybe I will come up with a suitable model.



>
> --
> Denis
>
>
>
> On 2/16/2016 11:33 AM, mp wrote:
>
> Hi Denis,
>
> Many thanks! I look forward to 1.6 then.
> Please also consider the following statement made by Dmitriy on Nov 03,
> 2015 (see his message in the thread):
>
> "With that in mind, we will be removing the requirement for caches to work
> only with SHARED and CONTINUOUS deployment modes, so you will be able to
> use PRIVATE or ISOLATED deployment modes to deploy your computations."
>
> As far as I understand, the above planned change is not covered by any
> Jira ticket.
>
> Cheers,
> -Mateusz
>
>
>
> On Fri, Feb 12, 2016 at 10:35 PM, Denis Magda <dm...@gridgain.com> wrote:
>
>> Hi Mateusz,
>>
>> I assigned both tickets that you have problems with on myself. They will
>> be fixed as a part of the next release.
>> https://issues.apache.org/jira/browse/IGNITE-2339
>> https://issues.apache.org/jira/browse/IGNITE-1823
>>
>> There is one more issue that was reproduced locally and refers to
>> unexpected cache undeployment when the binary marshaller is used.
>> https://issues.apache.org/jira/browse/IGNITE-2647
>>
>> Thanks for your patience and still showing the interest in Ignite.
>>
>> Regards,
>> Denis
>>
>>
>> On 2/12/2016 4:41 PM, mp wrote:
>>
>> Hi Denis,
>>
>> But my test still fails in version 1.5 with default (ie, binary)
>> marshaller. See my message from January 7, and your reply in which you
>> mentioned a new Jira ticked for a bug concerning the new binary marshaller:
>> <https://issues.apache.org/jira/browse/IGNITE-2339>
>> https://issues.apache.org/jira/browse/IGNITE-2339
>>
>> Basically, my test case (see
>> <https://issues.apache.org/jira/browse/IGNITE-1823>
>> https://issues.apache.org/jira/browse/IGNITE-1823 ) fails in all of the
>> scenarios I tried:
>>
>> 1. Binary marshaller + default deployment mode
>> 2. Binary marshaller + shared deployment mode
>> 3. Binary marshaller + private deployment mode
>> 4. Optimized marshaller + default deployment mode
>> 5. Optimized marshaller + shared deployment mode
>> 6. Optimized marshaller + private deployment mode
>>
>> Would you have any hint/advice on how I could proceed? Is there any
>> chance of fixing the issues related to my test case?
>>
>> Thanks for your help,
>> -Mateusz
>>
>>
>> On Wed, Feb 10, 2016 at 4:46 PM, Denis Magda < <dm...@gridgain.com>
>> dmagda@gridgain.com> wrote:
>>
>>> Hi Mateusz,
>>>
>>> In version 1.5 we released the binary objects [1] format that allows to
>>> store cache in class version independent form. Thus you don't need to have
>>> any classes on server side.
>>> This ability allows dynamic change to an objects structure, and even
>>> allows multiple clients with different versions of class definitions to
>>> co-exist.
>>>
>>> In my understanding if you switch to this format you will be able to
>>> support your use case.
>>>
>>> If something is unclear don't hesitate to ask.
>>>
>>> [1] https://apacheignite.readme.io/docs/binary-marshaller
>>>
>>> --
>>> Denis
>>>
>>>
>>> On 2/10/2016 4:06 PM, mp wrote:
>>>
>>> Hi Denis,
>>>
>>> Thanks for your reply.
>>> So, summing up, it seems that in the context of my use case, version 1.5
>>> does not differ from 1.4? Which means that I still cannot achieve my goal:
>>> different versions of the same class (from different clients) running on
>>> the cluster at the same time?
>>>
>>> As far as I understand this involves:
>>> 1. https://issues.apache.org/jira/browse/IGNITE-1823
>>> 2. https://issues.apache.org/jira/browse/IGNITE-2339
>>> 3. Removing the requirement for caches to work only with SHARED and
>>> CONTINUOUS deployment modes (this was announced by Dmitriy in
>>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1829.html
>>> )
>>>
>>> Is there any chance the above use case will be possible in near future
>>> (any upcoming version)?
>>>
>>> I really like the API and concept of Ignite. If only I could achieve the
>>> above scenario...
>>>
>>> Cheers,
>>> -Mateusz
>>>
>>>
>>>
>>> On Thu, Jan 7, 2016 at 5:25 PM, Denis Magda < <dm...@gridgain.com>
>>> dmagda@gridgain.com> wrote:
>>>
>>>> Mateusz,
>>>>
>>>> It doesn’t work for now because peerClassLoading doesn’t work for
>>>> objects that are stored in the binary format in a cache.
>>>> Since starting from 1.5 BinaryMarshaller is a default one all the
>>>> objects are stored in a such format in caches by default.
>>>>
>>>> If you prefer to turn off such a behavior you can set
>>>> IgniteConfiguration.setMarshaller(new OptimizedMarshaller()) for every node
>>>> and your test should work as before.
>>>>
>>>> —
>>>> Denis
>>>>
>>>> On 7 янв. 2016 г., at 17:09, mp < <mj...@gmail.com>
>>>> wrote:
>>>>
>>>> Hello Denis,
>>>>
>>>> Thanks a lot for your reply!
>>>> Concerning point 2: does it mean that "peerClassLoading" simply does
>>>> not work in 1.5?
>>>> It used to work (partially) in 1.4 (details described earlier in the
>>>> message thread).
>>>>
>>>> Cheers,
>>>> -Mateusz
>>>>
>>>>
>>>>
>>>> On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda < <dm...@gridgain.com>
>>>> dmagda@gridgain.com> wrote:
>>>>
>>>>> Hi Mateusz,
>>>>>
>>>>> 1. It seems that distributed cache is still *not* available in
>>>>> PRIVATE/ISOLATED modes. Is this correct?
>>>>>
>>>>> Right, it hasn't been fixed yet. I've just followed up the related
>>>>> discussion on the dev list. Please follow it to see the most up-to-date
>>>>> information
>>>>>
>>>>> <http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html>
>>>>> http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
>>>>>
>>>>> 2. When I run my simple test code in the default SHARED mode (the same
>>>>> as
>>>>> specified in <https://issues.apache.org/jira/browse/IGNITE-1823>
>>>>> https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
>>>>> I still get an error. However the cause exception seems to be
>>>>> different.
>>>>> Please see attached server log.
>>>>>
>>>>> The reason is that there is an attempt to deserialize a binary object
>>>>> stored on a server node and the server node doesn't have object's class
>>>>> definition in its class path.
>>>>> I've opened a ticket
>>>>> <https://issues.apache.org/jira/browse/IGNITE-2339>
>>>>> https://issues.apache.org/jira/browse/IGNITE-2339
>>>>>
>>>>> As a workaround you can put a class definition on server's class path
>>>>> and the problem will disappear.
>>>>>
>>>>> Regards,
>>>>> Denis
>>>>>
>>>>> On 1/7/2016 1:30 PM, mjjp wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I have just downloaded 1.5.0-final to check if my problem has been
>>>>>> resolved.
>>>>>> Either I'm doing something wrong, or version 1.5 has the same
>>>>>> behavior in
>>>>>> this context:
>>>>>>
>>>>>> 1. It seems that distributed cache is still *not* available in
>>>>>> PRIVATE/ISOLATED modes. Is this correct?
>>>>>>
>>>>>> 2. When I run my simple test code in the default SHARED mode (the
>>>>>> same as
>>>>>> specified in <https://issues.apache.org/jira/browse/IGNITE-1823>
>>>>>> https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
>>>>>> I still get an error. However the cause exception seems to be
>>>>>> different.
>>>>>> Please see attached server log.
>>>>>>
>>>>>> Would you be able to check the attached log to verify if this is an
>>>>>> expected
>>>>>> behavior in 1.5?
>>>>>>
>>>>>> Cheers,
>>>>>> -Mateusz
>>>>>>
>>>>>> ignite-fd14d572.log
>>>>>> <
>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log>
>>>>>> http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html>
>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
>>>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com
>>>>>> <http://nabble.com>.
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Hi Mateusz,

I've revisited the whole discussion from the beginning and should say 
that the solution based on the distributed queue won't work for you even 
if all the issues listed below are fixed.

Presently you're placing in the queue tasks coming form different nodes 
with different class versions. Even if the tasks are stored in the queue 
in a binary form they have to be deserialized to an original form before 
execution. This will lead to ClassNotFoundExceptions in your case.

My suggestion is the same is it was before. Avoid usage of the 
distributed queue but rather start sending ignite compute tasks for 
execution from the beginning.

Will this work for you?

--
Denis


On 2/16/2016 11:33 AM, mp wrote:
> Hi Denis,
>
> Many thanks! I look forward to 1.6 then.
> Please also consider the following statement made by Dmitriy on Nov 
> 03, 2015 (see his message in the thread):
>
> "With that in mind, we will be removing the requirement for caches to 
> work only with SHARED and CONTINUOUS deployment modes, so you will be 
> able to use PRIVATE or ISOLATED deployment modes to deploy your 
> computations."
>
> As far as I understand, the above planned change is not covered by any 
> Jira ticket.
>
> Cheers,
> -Mateusz
>
>
>
> On Fri, Feb 12, 2016 at 10:35 PM, Denis Magda <dmagda@gridgain.com 
> <ma...@gridgain.com>> wrote:
>
>     Hi Mateusz,
>
>     I assigned both tickets that you have problems with on myself.
>     They will be fixed as a part of the next release.
>     https://issues.apache.org/jira/browse/IGNITE-2339
>     https://issues.apache.org/jira/browse/IGNITE-1823
>
>     There is one more issue that was reproduced locally and refers to
>     unexpected cache undeployment when the binary marshaller is used.
>     https://issues.apache.org/jira/browse/IGNITE-2647
>
>     Thanks for your patience and still showing the interest in Ignite.
>
>     Regards,
>     Denis
>
>
>     On 2/12/2016 4:41 PM, mp wrote:
>>     Hi Denis,
>>
>>     But my test still fails in version 1.5 with default (ie, binary)
>>     marshaller. See my message from January 7, and your reply in
>>     which you mentioned a new Jira ticked for a bug concerning the
>>     new binary marshaller:
>>     https://issues.apache.org/jira/browse/IGNITE-2339
>>
>>     Basically, my test case (see
>>     https://issues.apache.org/jira/browse/IGNITE-1823 ) fails in all
>>     of the scenarios I tried:
>>
>>     1. Binary marshaller + default deployment mode
>>     2. Binary marshaller + shared deployment mode
>>     3. Binary marshaller + private deployment mode
>>     4. Optimized marshaller + default deployment mode
>>     5. Optimized marshaller + shared deployment mode
>>     6. Optimized marshaller + private deployment mode
>>
>>     Would you have any hint/advice on how I could proceed? Is there
>>     any chance of fixing the issues related to my test case?
>>
>>     Thanks for your help,
>>     -Mateusz
>>
>>
>>     On Wed, Feb 10, 2016 at 4:46 PM, Denis Magda <dmagda@gridgain.com
>>     <ma...@gridgain.com>> wrote:
>>
>>         Hi Mateusz,
>>
>>         In version 1.5 we released the binary objects [1] format that
>>         allows to store cache in class version independent form. Thus
>>         you don't need to have any classes on server side.
>>         This ability allows dynamic change to an objects structure,
>>         and even allows multiple clients with different versions of
>>         class definitions to co-exist.
>>
>>         In my understanding if you switch to this format you will be
>>         able to support your use case.
>>
>>         If something is unclear don't hesitate to ask.
>>
>>         [1] https://apacheignite.readme.io/docs/binary-marshaller
>>
>>         --
>>         Denis
>>
>>
>>         On 2/10/2016 4:06 PM, mp wrote:
>>>         Hi Denis,
>>>
>>>         Thanks for your reply.
>>>         So, summing up, it seems that in the context of my use case,
>>>         version 1.5 does not differ from 1.4? Which means that I
>>>         still cannot achieve my goal: different versions of the same
>>>         class (from different clients) running on the cluster at the
>>>         same time?
>>>
>>>         As far as I understand this involves:
>>>         1. https://issues.apache.org/jira/browse/IGNITE-1823
>>>         2. https://issues.apache.org/jira/browse/IGNITE-2339
>>>         3. Removing the requirement for caches to work only with
>>>         SHARED and CONTINUOUS deployment modes (this was announced
>>>         by Dmitriy in
>>>         http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1829.html
>>>         )
>>>
>>>         Is there any chance the above use case will be possible in
>>>         near future (any upcoming version)?
>>>
>>>         I really like the API and concept of Ignite. If only I could
>>>         achieve the above scenario...
>>>
>>>         Cheers,
>>>         -Mateusz
>>>
>>>
>>>
>>>         On Thu, Jan 7, 2016 at 5:25 PM, Denis Magda
>>>         <dmagda@gridgain.com <ma...@gridgain.com>> wrote:
>>>
>>>             Mateusz,
>>>
>>>             It doesn’t work for now because peerClassLoading doesn’t
>>>             work for objects that are stored in the binary format in
>>>             a cache.
>>>             Since starting from 1.5 BinaryMarshaller is a default
>>>             one all the objects are stored in a such format in
>>>             caches by default.
>>>
>>>             If you prefer to turn off such a behavior you can set
>>>             IgniteConfiguration.setMarshaller(new
>>>             OptimizedMarshaller()) for every node and your test
>>>             should work as before.
>>>
>>>             —
>>>             Denis
>>>
>>>>             On 7 янв. 2016 г., at 17:09, mp <mjjp00@gmail.com
>>>>             <ma...@gmail.com>> wrote:
>>>>
>>>>             Hello Denis,
>>>>
>>>>             Thanks a lot for your reply!
>>>>             Concerning point 2: does it mean that
>>>>             "peerClassLoading" simply does not work in 1.5?
>>>>             It used to work (partially) in 1.4 (details described
>>>>             earlier in the message thread).
>>>>
>>>>             Cheers,
>>>>             -Mateusz
>>>>
>>>>
>>>>
>>>>             On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda
>>>>             <dmagda@gridgain.com <ma...@gridgain.com>> wrote:
>>>>
>>>>                 Hi Mateusz,
>>>>
>>>>                 1. It seems that distributed cache is still *not*
>>>>                 available in
>>>>                 PRIVATE/ISOLATED modes. Is this correct?
>>>>
>>>>                 Right, it hasn't been fixed yet. I've just followed
>>>>                 up the related discussion on the dev list. Please
>>>>                 follow it to see the most up-to-date information
>>>>                 http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
>>>>
>>>>                 2. When I run my simple test code in the default
>>>>                 SHARED mode (the same as
>>>>                 specified in
>>>>                 https://issues.apache.org/jira/browse/IGNITE-1823
>>>>                 jira issue),
>>>>                 I still get an error. However the cause exception
>>>>                 seems to be different.
>>>>                 Please see attached server log.
>>>>
>>>>                 The reason is that there is an attempt to
>>>>                 deserialize a binary object stored on a server node
>>>>                 and the server node doesn't have object's class
>>>>                 definition in its class path.
>>>>                 I've opened a ticket
>>>>                 https://issues.apache.org/jira/browse/IGNITE-2339
>>>>
>>>>                 As a workaround you can put a class definition on
>>>>                 server's class path and the problem will disappear.
>>>>
>>>>                 Regards,
>>>>                 Denis
>>>>
>>>>                 On 1/7/2016 1:30 PM, mjjp wrote:
>>>>
>>>>                     Hello,
>>>>
>>>>                     I have just downloaded 1.5.0-final to check if
>>>>                     my problem has been resolved.
>>>>                     Either I'm doing something wrong, or version
>>>>                     1.5 has the same behavior in
>>>>                     this context:
>>>>
>>>>                     1. It seems that distributed cache is still
>>>>                     *not* available in
>>>>                     PRIVATE/ISOLATED modes. Is this correct?
>>>>
>>>>                     2. When I run my simple test code in the
>>>>                     default SHARED mode (the same as
>>>>                     specified in
>>>>                     https://issues.apache.org/jira/browse/IGNITE-1823
>>>>                     jira issue),
>>>>                     I still get an error. However the cause
>>>>                     exception seems to be different.
>>>>                     Please see attached server log.
>>>>
>>>>                     Would you be able to check the attached log to
>>>>                     verify if this is an expected
>>>>                     behavior in 1.5?
>>>>
>>>>                     Cheers,
>>>>                     -Mateusz
>>>>
>>>>                     ignite-fd14d572.log
>>>>                     <http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log>
>>>>
>>>>
>>>>
>>>>
>>>>                     --
>>>>                     View this message in context:
>>>>                     http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
>>>>                     Sent from the Apache Ignite Users mailing list
>>>>                     archive at Nabble.com <http://nabble.com>.
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>


Re: Distributed queue problem with peerClassLoading enabled

Posted by mp <mj...@gmail.com>.
Hi Denis,

Many thanks! I look forward to 1.6 then.
Please also consider the following statement made by Dmitriy on Nov 03,
2015 (see his message in the thread):

"With that in mind, we will be removing the requirement for caches to work
only with SHARED and CONTINUOUS deployment modes, so you will be able to
use PRIVATE or ISOLATED deployment modes to deploy your computations."

As far as I understand, the above planned change is not covered by any Jira
ticket.

Cheers,
-Mateusz



On Fri, Feb 12, 2016 at 10:35 PM, Denis Magda <dm...@gridgain.com> wrote:

> Hi Mateusz,
>
> I assigned both tickets that you have problems with on myself. They will
> be fixed as a part of the next release.
> https://issues.apache.org/jira/browse/IGNITE-2339
> https://issues.apache.org/jira/browse/IGNITE-1823
>
> There is one more issue that was reproduced locally and refers to
> unexpected cache undeployment when the binary marshaller is used.
> https://issues.apache.org/jira/browse/IGNITE-2647
>
> Thanks for your patience and still showing the interest in Ignite.
>
> Regards,
> Denis
>
>
> On 2/12/2016 4:41 PM, mp wrote:
>
> Hi Denis,
>
> But my test still fails in version 1.5 with default (ie, binary)
> marshaller. See my message from January 7, and your reply in which you
> mentioned a new Jira ticked for a bug concerning the new binary marshaller:
> <https://issues.apache.org/jira/browse/IGNITE-2339>
> https://issues.apache.org/jira/browse/IGNITE-2339
>
> Basically, my test case (see
> https://issues.apache.org/jira/browse/IGNITE-1823 ) fails in all of the
> scenarios I tried:
>
> 1. Binary marshaller + default deployment mode
> 2. Binary marshaller + shared deployment mode
> 3. Binary marshaller + private deployment mode
> 4. Optimized marshaller + default deployment mode
> 5. Optimized marshaller + shared deployment mode
> 6. Optimized marshaller + private deployment mode
>
> Would you have any hint/advice on how I could proceed? Is there any chance
> of fixing the issues related to my test case?
>
> Thanks for your help,
> -Mateusz
>
>
> On Wed, Feb 10, 2016 at 4:46 PM, Denis Magda <dm...@gridgain.com> wrote:
>
>> Hi Mateusz,
>>
>> In version 1.5 we released the binary objects [1] format that allows to
>> store cache in class version independent form. Thus you don't need to have
>> any classes on server side.
>> This ability allows dynamic change to an objects structure, and even
>> allows multiple clients with different versions of class definitions to
>> co-exist.
>>
>> In my understanding if you switch to this format you will be able to
>> support your use case.
>>
>> If something is unclear don't hesitate to ask.
>>
>> [1] https://apacheignite.readme.io/docs/binary-marshaller
>>
>> --
>> Denis
>>
>>
>> On 2/10/2016 4:06 PM, mp wrote:
>>
>> Hi Denis,
>>
>> Thanks for your reply.
>> So, summing up, it seems that in the context of my use case, version 1.5
>> does not differ from 1.4? Which means that I still cannot achieve my goal:
>> different versions of the same class (from different clients) running on
>> the cluster at the same time?
>>
>> As far as I understand this involves:
>> 1. https://issues.apache.org/jira/browse/IGNITE-1823
>> 2. https://issues.apache.org/jira/browse/IGNITE-2339
>> 3. Removing the requirement for caches to work only with SHARED and
>> CONTINUOUS deployment modes (this was announced by Dmitriy in
>> <http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1829.html>
>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1829.html
>> )
>>
>> Is there any chance the above use case will be possible in near future
>> (any upcoming version)?
>>
>> I really like the API and concept of Ignite. If only I could achieve the
>> above scenario...
>>
>> Cheers,
>> -Mateusz
>>
>>
>>
>> On Thu, Jan 7, 2016 at 5:25 PM, Denis Magda < <dm...@gridgain.com>
>> dmagda@gridgain.com> wrote:
>>
>>> Mateusz,
>>>
>>> It doesn’t work for now because peerClassLoading doesn’t work for
>>> objects that are stored in the binary format in a cache.
>>> Since starting from 1.5 BinaryMarshaller is a default one all the
>>> objects are stored in a such format in caches by default.
>>>
>>> If you prefer to turn off such a behavior you can set
>>> IgniteConfiguration.setMarshaller(new OptimizedMarshaller()) for every node
>>> and your test should work as before.
>>>
>>> —
>>> Denis
>>>
>>> On 7 янв. 2016 г., at 17:09, mp < <mj...@gmail.com>
>>> wrote:
>>>
>>> Hello Denis,
>>>
>>> Thanks a lot for your reply!
>>> Concerning point 2: does it mean that "peerClassLoading" simply does not
>>> work in 1.5?
>>> It used to work (partially) in 1.4 (details described earlier in the
>>> message thread).
>>>
>>> Cheers,
>>> -Mateusz
>>>
>>>
>>>
>>> On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda < <dm...@gridgain.com>
>>> dmagda@gridgain.com> wrote:
>>>
>>>> Hi Mateusz,
>>>>
>>>> 1. It seems that distributed cache is still *not* available in
>>>> PRIVATE/ISOLATED modes. Is this correct?
>>>>
>>>> Right, it hasn't been fixed yet. I've just followed up the related
>>>> discussion on the dev list. Please follow it to see the most up-to-date
>>>> information
>>>>
>>>> <http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html>
>>>> http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
>>>>
>>>> 2. When I run my simple test code in the default SHARED mode (the same
>>>> as
>>>> specified in <https://issues.apache.org/jira/browse/IGNITE-1823>
>>>> https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
>>>> I still get an error. However the cause exception seems to be different.
>>>> Please see attached server log.
>>>>
>>>> The reason is that there is an attempt to deserialize a binary object
>>>> stored on a server node and the server node doesn't have object's class
>>>> definition in its class path.
>>>> I've opened a ticket
>>>> <https://issues.apache.org/jira/browse/IGNITE-2339>
>>>> https://issues.apache.org/jira/browse/IGNITE-2339
>>>>
>>>> As a workaround you can put a class definition on server's class path
>>>> and the problem will disappear.
>>>>
>>>> Regards,
>>>> Denis
>>>>
>>>> On 1/7/2016 1:30 PM, mjjp wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I have just downloaded 1.5.0-final to check if my problem has been
>>>>> resolved.
>>>>> Either I'm doing something wrong, or version 1.5 has the same behavior
>>>>> in
>>>>> this context:
>>>>>
>>>>> 1. It seems that distributed cache is still *not* available in
>>>>> PRIVATE/ISOLATED modes. Is this correct?
>>>>>
>>>>> 2. When I run my simple test code in the default SHARED mode (the same
>>>>> as
>>>>> specified in <https://issues.apache.org/jira/browse/IGNITE-1823>
>>>>> https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
>>>>> I still get an error. However the cause exception seems to be
>>>>> different.
>>>>> Please see attached server log.
>>>>>
>>>>> Would you be able to check the attached log to verify if this is an
>>>>> expected
>>>>> behavior in 1.5?
>>>>>
>>>>> Cheers,
>>>>> -Mateusz
>>>>>
>>>>> ignite-fd14d572.log
>>>>> <
>>>>> <http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log>
>>>>> http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html>
>>>>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
>>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com
>>>>> <http://nabble.com>.
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Hi Mateusz,

I assigned both tickets that you have problems with on myself. They will 
be fixed as a part of the next release.
https://issues.apache.org/jira/browse/IGNITE-2339
https://issues.apache.org/jira/browse/IGNITE-1823

There is one more issue that was reproduced locally and refers to 
unexpected cache undeployment when the binary marshaller is used.
https://issues.apache.org/jira/browse/IGNITE-2647

Thanks for your patience and still showing the interest in Ignite.

Regards,
Denis

On 2/12/2016 4:41 PM, mp wrote:
> Hi Denis,
>
> But my test still fails in version 1.5 with default (ie, binary) 
> marshaller. See my message from January 7, and your reply in which you 
> mentioned a new Jira ticked for a bug concerning the new binary 
> marshaller: https://issues.apache.org/jira/browse/IGNITE-2339
>
> Basically, my test case (see 
> https://issues.apache.org/jira/browse/IGNITE-1823 ) fails in all of 
> the scenarios I tried:
>
> 1. Binary marshaller + default deployment mode
> 2. Binary marshaller + shared deployment mode
> 3. Binary marshaller + private deployment mode
> 4. Optimized marshaller + default deployment mode
> 5. Optimized marshaller + shared deployment mode
> 6. Optimized marshaller + private deployment mode
>
> Would you have any hint/advice on how I could proceed? Is there any 
> chance of fixing the issues related to my test case?
>
> Thanks for your help,
> -Mateusz
>
>
> On Wed, Feb 10, 2016 at 4:46 PM, Denis Magda <dmagda@gridgain.com 
> <ma...@gridgain.com>> wrote:
>
>     Hi Mateusz,
>
>     In version 1.5 we released the binary objects [1] format that
>     allows to store cache in class version independent form. Thus you
>     don't need to have any classes on server side.
>     This ability allows dynamic change to an objects structure, and
>     even allows multiple clients with different versions of class
>     definitions to co-exist.
>
>     In my understanding if you switch to this format you will be able
>     to support your use case.
>
>     If something is unclear don't hesitate to ask.
>
>     [1] https://apacheignite.readme.io/docs/binary-marshaller
>
>     --
>     Denis
>
>
>     On 2/10/2016 4:06 PM, mp wrote:
>>     Hi Denis,
>>
>>     Thanks for your reply.
>>     So, summing up, it seems that in the context of my use case,
>>     version 1.5 does not differ from 1.4? Which means that I still
>>     cannot achieve my goal: different versions of the same class
>>     (from different clients) running on the cluster at the same time?
>>
>>     As far as I understand this involves:
>>     1. https://issues.apache.org/jira/browse/IGNITE-1823
>>     2. https://issues.apache.org/jira/browse/IGNITE-2339
>>     3. Removing the requirement for caches to work only with SHARED
>>     and CONTINUOUS deployment modes (this was announced by Dmitriy in
>>     http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1829.html
>>     )
>>
>>     Is there any chance the above use case will be possible in near
>>     future (any upcoming version)?
>>
>>     I really like the API and concept of Ignite. If only I could
>>     achieve the above scenario...
>>
>>     Cheers,
>>     -Mateusz
>>
>>
>>
>>     On Thu, Jan 7, 2016 at 5:25 PM, Denis Magda <dmagda@gridgain.com
>>     <ma...@gridgain.com>> wrote:
>>
>>         Mateusz,
>>
>>         It doesn’t work for now because peerClassLoading doesn’t work
>>         for objects that are stored in the binary format in a cache.
>>         Since starting from 1.5 BinaryMarshaller is a default one all
>>         the objects are stored in a such format in caches by default.
>>
>>         If you prefer to turn off such a behavior you can set
>>         IgniteConfiguration.setMarshaller(new OptimizedMarshaller())
>>         for every node and your test should work as before.
>>
>>         —
>>         Denis
>>
>>>         On 7 янв. 2016 г., at 17:09, mp <mjjp00@gmail.com
>>>         <ma...@gmail.com>> wrote:
>>>
>>>         Hello Denis,
>>>
>>>         Thanks a lot for your reply!
>>>         Concerning point 2: does it mean that "peerClassLoading"
>>>         simply does not work in 1.5?
>>>         It used to work (partially) in 1.4 (details described
>>>         earlier in the message thread).
>>>
>>>         Cheers,
>>>         -Mateusz
>>>
>>>
>>>
>>>         On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda
>>>         <dmagda@gridgain.com <ma...@gridgain.com>> wrote:
>>>
>>>             Hi Mateusz,
>>>
>>>             1. It seems that distributed cache is still *not*
>>>             available in
>>>             PRIVATE/ISOLATED modes. Is this correct?
>>>
>>>             Right, it hasn't been fixed yet. I've just followed up
>>>             the related discussion on the dev list. Please follow it
>>>             to see the most up-to-date information
>>>             http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
>>>
>>>             2. When I run my simple test code in the default SHARED
>>>             mode (the same as
>>>             specified in
>>>             https://issues.apache.org/jira/browse/IGNITE-1823 jira
>>>             issue),
>>>             I still get an error. However the cause exception seems
>>>             to be different.
>>>             Please see attached server log.
>>>
>>>             The reason is that there is an attempt to deserialize a
>>>             binary object stored on a server node and the server
>>>             node doesn't have object's class definition in its class
>>>             path.
>>>             I've opened a ticket
>>>             https://issues.apache.org/jira/browse/IGNITE-2339
>>>
>>>             As a workaround you can put a class definition on
>>>             server's class path and the problem will disappear.
>>>
>>>             Regards,
>>>             Denis
>>>
>>>             On 1/7/2016 1:30 PM, mjjp wrote:
>>>
>>>                 Hello,
>>>
>>>                 I have just downloaded 1.5.0-final to check if my
>>>                 problem has been resolved.
>>>                 Either I'm doing something wrong, or version 1.5 has
>>>                 the same behavior in
>>>                 this context:
>>>
>>>                 1. It seems that distributed cache is still *not*
>>>                 available in
>>>                 PRIVATE/ISOLATED modes. Is this correct?
>>>
>>>                 2. When I run my simple test code in the default
>>>                 SHARED mode (the same as
>>>                 specified in
>>>                 https://issues.apache.org/jira/browse/IGNITE-1823
>>>                 jira issue),
>>>                 I still get an error. However the cause exception
>>>                 seems to be different.
>>>                 Please see attached server log.
>>>
>>>                 Would you be able to check the attached log to
>>>                 verify if this is an expected
>>>                 behavior in 1.5?
>>>
>>>                 Cheers,
>>>                 -Mateusz
>>>
>>>                 ignite-fd14d572.log
>>>                 <http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log>
>>>
>>>
>>>
>>>
>>>                 --
>>>                 View this message in context:
>>>                 http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
>>>                 Sent from the Apache Ignite Users mailing list
>>>                 archive at Nabble.com <http://nabble.com>.
>>>
>>>
>>>
>>
>>
>
>


Re: Distributed queue problem with peerClassLoading enabled

Posted by mp <mj...@gmail.com>.
Hi Denis,

But my test still fails in version 1.5 with default (ie, binary)
marshaller. See my message from January 7, and your reply in which you
mentioned a new Jira ticked for a bug concerning the new binary marshaller:
https://issues.apache.org/jira/browse/IGNITE-2339

Basically, my test case (see
https://issues.apache.org/jira/browse/IGNITE-1823 ) fails in all of the
scenarios I tried:

1. Binary marshaller + default deployment mode
2. Binary marshaller + shared deployment mode
3. Binary marshaller + private deployment mode
4. Optimized marshaller + default deployment mode
5. Optimized marshaller + shared deployment mode
6. Optimized marshaller + private deployment mode

Would you have any hint/advice on how I could proceed? Is there any chance
of fixing the issues related to my test case?

Thanks for your help,
-Mateusz


On Wed, Feb 10, 2016 at 4:46 PM, Denis Magda <dm...@gridgain.com> wrote:

> Hi Mateusz,
>
> In version 1.5 we released the binary objects [1] format that allows to
> store cache in class version independent form. Thus you don't need to have
> any classes on server side.
> This ability allows dynamic change to an objects structure, and even
> allows multiple clients with different versions of class definitions to
> co-exist.
>
> In my understanding if you switch to this format you will be able to
> support your use case.
>
> If something is unclear don't hesitate to ask.
>
> [1] https://apacheignite.readme.io/docs/binary-marshaller
>
> --
> Denis
>
>
> On 2/10/2016 4:06 PM, mp wrote:
>
> Hi Denis,
>
> Thanks for your reply.
> So, summing up, it seems that in the context of my use case, version 1.5
> does not differ from 1.4? Which means that I still cannot achieve my goal:
> different versions of the same class (from different clients) running on
> the cluster at the same time?
>
> As far as I understand this involves:
> 1. https://issues.apache.org/jira/browse/IGNITE-1823
> 2. https://issues.apache.org/jira/browse/IGNITE-2339
> 3. Removing the requirement for caches to work only with SHARED and
> CONTINUOUS deployment modes (this was announced by Dmitriy in
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1829.html
> )
>
> Is there any chance the above use case will be possible in near future
> (any upcoming version)?
>
> I really like the API and concept of Ignite. If only I could achieve the
> above scenario...
>
> Cheers,
> -Mateusz
>
>
>
> On Thu, Jan 7, 2016 at 5:25 PM, Denis Magda <dm...@gridgain.com> wrote:
>
>> Mateusz,
>>
>> It doesn’t work for now because peerClassLoading doesn’t work for objects
>> that are stored in the binary format in a cache.
>> Since starting from 1.5 BinaryMarshaller is a default one all the objects
>> are stored in a such format in caches by default.
>>
>> If you prefer to turn off such a behavior you can set
>> IgniteConfiguration.setMarshaller(new OptimizedMarshaller()) for every node
>> and your test should work as before.
>>
>> —
>> Denis
>>
>> On 7 янв. 2016 г., at 17:09, mp < <mj...@gmail.com>
>> wrote:
>>
>> Hello Denis,
>>
>> Thanks a lot for your reply!
>> Concerning point 2: does it mean that "peerClassLoading" simply does not
>> work in 1.5?
>> It used to work (partially) in 1.4 (details described earlier in the
>> message thread).
>>
>> Cheers,
>> -Mateusz
>>
>>
>>
>> On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda < <dm...@gridgain.com>
>> dmagda@gridgain.com> wrote:
>>
>>> Hi Mateusz,
>>>
>>> 1. It seems that distributed cache is still *not* available in
>>> PRIVATE/ISOLATED modes. Is this correct?
>>>
>>> Right, it hasn't been fixed yet. I've just followed up the related
>>> discussion on the dev list. Please follow it to see the most up-to-date
>>> information
>>>
>>> http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
>>>
>>> 2. When I run my simple test code in the default SHARED mode (the same as
>>> specified in <https://issues.apache.org/jira/browse/IGNITE-1823>
>>> https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
>>> I still get an error. However the cause exception seems to be different.
>>> Please see attached server log.
>>>
>>> The reason is that there is an attempt to deserialize a binary object
>>> stored on a server node and the server node doesn't have object's class
>>> definition in its class path.
>>> I've opened a ticket
>>> https://issues.apache.org/jira/browse/IGNITE-2339
>>>
>>> As a workaround you can put a class definition on server's class path
>>> and the problem will disappear.
>>>
>>> Regards,
>>> Denis
>>>
>>> On 1/7/2016 1:30 PM, mjjp wrote:
>>>
>>>> Hello,
>>>>
>>>> I have just downloaded 1.5.0-final to check if my problem has been
>>>> resolved.
>>>> Either I'm doing something wrong, or version 1.5 has the same behavior
>>>> in
>>>> this context:
>>>>
>>>> 1. It seems that distributed cache is still *not* available in
>>>> PRIVATE/ISOLATED modes. Is this correct?
>>>>
>>>> 2. When I run my simple test code in the default SHARED mode (the same
>>>> as
>>>> specified in <https://issues.apache.org/jira/browse/IGNITE-1823>
>>>> https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
>>>> I still get an error. However the cause exception seems to be different.
>>>> Please see attached server log.
>>>>
>>>> Would you be able to check the attached log to verify if this is an
>>>> expected
>>>> behavior in 1.5?
>>>>
>>>> Cheers,
>>>> -Mateusz
>>>>
>>>> ignite-fd14d572.log
>>>> <
>>>> http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log
>>>> >
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> <http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html>
>>>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com
>>>> <http://nabble.com>.
>>>>
>>>
>>>
>>
>>
>
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Hi Mateusz,

In version 1.5 we released the binary objects [1] format that allows to 
store cache in class version independent form. Thus you don't need to 
have any classes on server side.
This ability allows dynamic change to an objects structure, and even 
allows multiple clients with different versions of class definitions to 
co-exist.

In my understanding if you switch to this format you will be able to 
support your use case.

If something is unclear don't hesitate to ask.

[1] https://apacheignite.readme.io/docs/binary-marshaller

--
Denis

On 2/10/2016 4:06 PM, mp wrote:
> Hi Denis,
>
> Thanks for your reply.
> So, summing up, it seems that in the context of my use case, version 
> 1.5 does not differ from 1.4? Which means that I still cannot achieve 
> my goal: different versions of the same class (from different clients) 
> running on the cluster at the same time?
>
> As far as I understand this involves:
> 1. https://issues.apache.org/jira/browse/IGNITE-1823
> 2. https://issues.apache.org/jira/browse/IGNITE-2339
> 3. Removing the requirement for caches to work only with SHARED and 
> CONTINUOUS deployment modes (this was announced by Dmitriy in 
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1829.html 
> )
>
> Is there any chance the above use case will be possible in near future 
> (any upcoming version)?
>
> I really like the API and concept of Ignite. If only I could achieve 
> the above scenario...
>
> Cheers,
> -Mateusz
>
>
>
> On Thu, Jan 7, 2016 at 5:25 PM, Denis Magda <dmagda@gridgain.com 
> <ma...@gridgain.com>> wrote:
>
>     Mateusz,
>
>     It doesn’t work for now because peerClassLoading doesn’t work for
>     objects that are stored in the binary format in a cache.
>     Since starting from 1.5 BinaryMarshaller is a default one all the
>     objects are stored in a such format in caches by default.
>
>     If you prefer to turn off such a behavior you can set
>     IgniteConfiguration.setMarshaller(new OptimizedMarshaller()) for
>     every node and your test should work as before.
>
>     —
>     Denis
>
>>     On 7 янв. 2016 г., at 17:09, mp <mjjp00@gmail.com
>>     <ma...@gmail.com>> wrote:
>>
>>     Hello Denis,
>>
>>     Thanks a lot for your reply!
>>     Concerning point 2: does it mean that "peerClassLoading" simply
>>     does not work in 1.5?
>>     It used to work (partially) in 1.4 (details described earlier in
>>     the message thread).
>>
>>     Cheers,
>>     -Mateusz
>>
>>
>>
>>     On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda <dmagda@gridgain.com
>>     <ma...@gridgain.com>> wrote:
>>
>>         Hi Mateusz,
>>
>>         1. It seems that distributed cache is still *not* available in
>>         PRIVATE/ISOLATED modes. Is this correct?
>>
>>         Right, it hasn't been fixed yet. I've just followed up the
>>         related discussion on the dev list. Please follow it to see
>>         the most up-to-date information
>>         http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
>>
>>         2. When I run my simple test code in the default SHARED mode
>>         (the same as
>>         specified in
>>         https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
>>         I still get an error. However the cause exception seems to be
>>         different.
>>         Please see attached server log.
>>
>>         The reason is that there is an attempt to deserialize a
>>         binary object stored on a server node and the server node
>>         doesn't have object's class definition in its class path.
>>         I've opened a ticket
>>         https://issues.apache.org/jira/browse/IGNITE-2339
>>
>>         As a workaround you can put a class definition on server's
>>         class path and the problem will disappear.
>>
>>         Regards,
>>         Denis
>>
>>         On 1/7/2016 1:30 PM, mjjp wrote:
>>
>>             Hello,
>>
>>             I have just downloaded 1.5.0-final to check if my problem
>>             has been resolved.
>>             Either I'm doing something wrong, or version 1.5 has the
>>             same behavior in
>>             this context:
>>
>>             1. It seems that distributed cache is still *not*
>>             available in
>>             PRIVATE/ISOLATED modes. Is this correct?
>>
>>             2. When I run my simple test code in the default SHARED
>>             mode (the same as
>>             specified in
>>             https://issues.apache.org/jira/browse/IGNITE-1823 jira
>>             issue),
>>             I still get an error. However the cause exception seems
>>             to be different.
>>             Please see attached server log.
>>
>>             Would you be able to check the attached log to verify if
>>             this is an expected
>>             behavior in 1.5?
>>
>>             Cheers,
>>             -Mateusz
>>
>>             ignite-fd14d572.log
>>             <http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log>
>>
>>
>>
>>
>>             --
>>             View this message in context:
>>             http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
>>             Sent from the Apache Ignite Users mailing list archive at
>>             Nabble.com <http://nabble.com>.
>>
>>
>>
>
>


Re: Distributed queue problem with peerClassLoading enabled

Posted by mp <mj...@gmail.com>.
Hi Denis,

Thanks for your reply.
So, summing up, it seems that in the context of my use case, version 1.5
does not differ from 1.4? Which means that I still cannot achieve my goal:
different versions of the same class (from different clients) running on
the cluster at the same time?

As far as I understand this involves:
1. https://issues.apache.org/jira/browse/IGNITE-1823
2. https://issues.apache.org/jira/browse/IGNITE-2339
3. Removing the requirement for caches to work only with SHARED and
CONTINUOUS deployment modes (this was announced by Dmitriy in
http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1829.html
)

Is there any chance the above use case will be possible in near future (any
upcoming version)?

I really like the API and concept of Ignite. If only I could achieve the
above scenario...

Cheers,
-Mateusz



On Thu, Jan 7, 2016 at 5:25 PM, Denis Magda <dm...@gridgain.com> wrote:

> Mateusz,
>
> It doesn’t work for now because peerClassLoading doesn’t work for objects
> that are stored in the binary format in a cache.
> Since starting from 1.5 BinaryMarshaller is a default one all the objects
> are stored in a such format in caches by default.
>
> If you prefer to turn off such a behavior you can set
> IgniteConfiguration.setMarshaller(new OptimizedMarshaller()) for every node
> and your test should work as before.
>
> —
> Denis
>
> On 7 янв. 2016 г., at 17:09, mp <mj...@gmail.com> wrote:
>
> Hello Denis,
>
> Thanks a lot for your reply!
> Concerning point 2: does it mean that "peerClassLoading" simply does not
> work in 1.5?
> It used to work (partially) in 1.4 (details described earlier in the
> message thread).
>
> Cheers,
> -Mateusz
>
>
>
> On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda <dm...@gridgain.com> wrote:
>
>> Hi Mateusz,
>>
>> 1. It seems that distributed cache is still *not* available in
>> PRIVATE/ISOLATED modes. Is this correct?
>>
>> Right, it hasn't been fixed yet. I've just followed up the related
>> discussion on the dev list. Please follow it to see the most up-to-date
>> information
>>
>> http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
>>
>> 2. When I run my simple test code in the default SHARED mode (the same as
>> specified in https://issues.apache.org/jira/browse/IGNITE-1823 jira
>> issue),
>> I still get an error. However the cause exception seems to be different.
>> Please see attached server log.
>>
>> The reason is that there is an attempt to deserialize a binary object
>> stored on a server node and the server node doesn't have object's class
>> definition in its class path.
>> I've opened a ticket
>> https://issues.apache.org/jira/browse/IGNITE-2339
>>
>> As a workaround you can put a class definition on server's class path and
>> the problem will disappear.
>>
>> Regards,
>> Denis
>>
>> On 1/7/2016 1:30 PM, mjjp wrote:
>>
>>> Hello,
>>>
>>> I have just downloaded 1.5.0-final to check if my problem has been
>>> resolved.
>>> Either I'm doing something wrong, or version 1.5 has the same behavior in
>>> this context:
>>>
>>> 1. It seems that distributed cache is still *not* available in
>>> PRIVATE/ISOLATED modes. Is this correct?
>>>
>>> 2. When I run my simple test code in the default SHARED mode (the same as
>>> specified in https://issues.apache.org/jira/browse/IGNITE-1823 jira
>>> issue),
>>> I still get an error. However the cause exception seems to be different.
>>> Please see attached server log.
>>>
>>> Would you be able to check the attached log to verify if this is an
>>> expected
>>> behavior in 1.5?
>>>
>>> Cheers,
>>> -Mateusz
>>>
>>> ignite-fd14d572.log
>>> <
>>> http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log
>>> >
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com
>>> <http://nabble.com>.
>>>
>>
>>
>
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Mateusz,

It doesn’t work for now because peerClassLoading doesn’t work for objects that are stored in the binary format in a cache.
Since starting from 1.5 BinaryMarshaller is a default one all the objects are stored in a such format in caches by default.

If you prefer to turn off such a behavior you can set IgniteConfiguration.setMarshaller(new OptimizedMarshaller()) for every node and your test should work as before.

—
Denis

> On 7 янв. 2016 г., at 17:09, mp <mj...@gmail.com> wrote:
> 
> Hello Denis,
> 
> Thanks a lot for your reply!
> Concerning point 2: does it mean that "peerClassLoading" simply does not work in 1.5?
> It used to work (partially) in 1.4 (details described earlier in the message thread).
> 
> Cheers,
> -Mateusz
> 
> 
> 
> On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda <dmagda@gridgain.com <ma...@gridgain.com>> wrote:
> Hi Mateusz,
> 
> 1. It seems that distributed cache is still *not* available in
> PRIVATE/ISOLATED modes. Is this correct?
> 
> Right, it hasn't been fixed yet. I've just followed up the related discussion on the dev list. Please follow it to see the most up-to-date information
> http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html <http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html>
> 
> 2. When I run my simple test code in the default SHARED mode (the same as
> specified in https://issues.apache.org/jira/browse/IGNITE-1823 <https://issues.apache.org/jira/browse/IGNITE-1823> jira issue),
> I still get an error. However the cause exception seems to be different.
> Please see attached server log.
> 
> The reason is that there is an attempt to deserialize a binary object stored on a server node and the server node doesn't have object's class definition in its class path.
> I've opened a ticket
> https://issues.apache.org/jira/browse/IGNITE-2339 <https://issues.apache.org/jira/browse/IGNITE-2339>
> 
> As a workaround you can put a class definition on server's class path and the problem will disappear.
> 
> Regards,
> Denis
> 
> On 1/7/2016 1:30 PM, mjjp wrote:
> Hello,
> 
> I have just downloaded 1.5.0-final to check if my problem has been resolved.
> Either I'm doing something wrong, or version 1.5 has the same behavior in
> this context:
> 
> 1. It seems that distributed cache is still *not* available in
> PRIVATE/ISOLATED modes. Is this correct?
> 
> 2. When I run my simple test code in the default SHARED mode (the same as
> specified in https://issues.apache.org/jira/browse/IGNITE-1823 <https://issues.apache.org/jira/browse/IGNITE-1823> jira issue),
> I still get an error. However the cause exception seems to be different.
> Please see attached server log.
> 
> Would you be able to check the attached log to verify if this is an expected
> behavior in 1.5?
> 
> Cheers,
> -Mateusz
> 
> ignite-fd14d572.log
> <http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log <http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log>>
> 
> 
> 
> 
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html <http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html>
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> 
> 


Re: Distributed queue problem with peerClassLoading enabled

Posted by mp <mj...@gmail.com>.
Hello Denis,

Thanks a lot for your reply!
Concerning point 2: does it mean that "peerClassLoading" simply does not
work in 1.5?
It used to work (partially) in 1.4 (details described earlier in the
message thread).

Cheers,
-Mateusz



On Thu, Jan 7, 2016 at 1:38 PM, Denis Magda <dm...@gridgain.com> wrote:

> Hi Mateusz,
>
> 1. It seems that distributed cache is still *not* available in
> PRIVATE/ISOLATED modes. Is this correct?
>
> Right, it hasn't been fixed yet. I've just followed up the related
> discussion on the dev list. Please follow it to see the most up-to-date
> information
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
>
> 2. When I run my simple test code in the default SHARED mode (the same as
> specified in https://issues.apache.org/jira/browse/IGNITE-1823 jira
> issue),
> I still get an error. However the cause exception seems to be different.
> Please see attached server log.
>
> The reason is that there is an attempt to deserialize a binary object
> stored on a server node and the server node doesn't have object's class
> definition in its class path.
> I've opened a ticket
> https://issues.apache.org/jira/browse/IGNITE-2339
>
> As a workaround you can put a class definition on server's class path and
> the problem will disappear.
>
> Regards,
> Denis
>
> On 1/7/2016 1:30 PM, mjjp wrote:
>
>> Hello,
>>
>> I have just downloaded 1.5.0-final to check if my problem has been
>> resolved.
>> Either I'm doing something wrong, or version 1.5 has the same behavior in
>> this context:
>>
>> 1. It seems that distributed cache is still *not* available in
>> PRIVATE/ISOLATED modes. Is this correct?
>>
>> 2. When I run my simple test code in the default SHARED mode (the same as
>> specified in https://issues.apache.org/jira/browse/IGNITE-1823 jira
>> issue),
>> I still get an error. However the cause exception seems to be different.
>> Please see attached server log.
>>
>> Would you be able to check the attached log to verify if this is an
>> expected
>> behavior in 1.5?
>>
>> Cheers,
>> -Mateusz
>>
>> ignite-fd14d572.log
>> <
>> http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log
>> >
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Hi Mateusz,

1. It seems that distributed cache is still *not* available in
PRIVATE/ISOLATED modes. Is this correct?

Right, it hasn't been fixed yet. I've just followed up the related 
discussion on the dev list. Please follow it to see the most up-to-date 
information
http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html

2. When I run my simple test code in the default SHARED mode (the same as
specified in https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
I still get an error. However the cause exception seems to be different.
Please see attached server log.

The reason is that there is an attempt to deserialize a binary object 
stored on a server node and the server node doesn't have object's class 
definition in its class path.
I've opened a ticket
https://issues.apache.org/jira/browse/IGNITE-2339

As a workaround you can put a class definition on server's class path 
and the problem will disappear.

Regards,
Denis

On 1/7/2016 1:30 PM, mjjp wrote:
> Hello,
>
> I have just downloaded 1.5.0-final to check if my problem has been resolved.
> Either I'm doing something wrong, or version 1.5 has the same behavior in
> this context:
>
> 1. It seems that distributed cache is still *not* available in
> PRIVATE/ISOLATED modes. Is this correct?
>
> 2. When I run my simple test code in the default SHARED mode (the same as
> specified in https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
> I still get an error. However the cause exception seems to be different.
> Please see attached server log.
>
> Would you be able to check the attached log to verify if this is an expected
> behavior in 1.5?
>
> Cheers,
> -Mateusz
>
> ignite-fd14d572.log
> <http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.


Re: Distributed queue problem with peerClassLoading enabled

Posted by mjjp <mj...@gmail.com>.
Hello,

I have just downloaded 1.5.0-final to check if my problem has been resolved.
Either I'm doing something wrong, or version 1.5 has the same behavior in
this context:

1. It seems that distributed cache is still *not* available in
PRIVATE/ISOLATED modes. Is this correct?

2. When I run my simple test code in the default SHARED mode (the same as
specified in https://issues.apache.org/jira/browse/IGNITE-1823 jira issue),
I still get an error. However the cause exception seems to be different.
Please see attached server log.

Would you be able to check the attached log to verify if this is an expected
behavior in 1.5?

Cheers,
-Mateusz

ignite-fd14d572.log
<http://apache-ignite-users.70518.x6.nabble.com/file/n2416/ignite-fd14d572.log>  




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p2416.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Ticket is created
https://issues.apache.org/jira/browse/IGNITE-2667

If anyone is ready to pick it up and fix then go ahead ;)

--
Denis



--
View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p7397.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Alexey, Dmitriy,

So the restriction (see discussion above) is still there in 1.5 release.

If we still want to remove it then it makes sense to create a JIRA ticket.
Thoughts?

In addition if the restriction is removed then we have to take care of this
bug that is related to unintentional deserialization of binary objects
stored in a distributed queue
https://issues.apache.org/jira/browse/IGNITE-2339

--
Denis



--
View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-tp4521p6440.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: Distributed queue problem with peerClassLoading enabled

Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Wed, Nov 4, 2015 at 4:39 AM, Alexey Goncharuk <alexey.goncharuk@gmail.com
> wrote:

> I am not sure what behavior we want to achieve here. Do we want to
> disable cache peer class loading for binary marshaling altogether?
> This restriction is safe to remove only if user chooses to work exclusively
> with Binary object representation.
>

Alexey, yes, you are right. Currently we have a restriction for PRIVATE or
ISOLATED deployment modes in caches. I am suggesting we should remove this
restriction and add a warning in the log about using only Binary
representation.


> If user still wants to work with class object representation, SHARED
> or CONTINUOUS deployment mode is still a requirement for the same reasons
> it was a requirement before.
>

Agree.


>
> 2015-11-04 11:48 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
>
> > Igniters,
> >
> > I want to bring your attention to my email response on the user list. I
> > just want to confirm that we are all on the same page and that my
> > understanding is correct.
> >
> > I think that once we merge the BinaryObject to the release branch, we
> > should remove the restriction on SHARED or CONTINUOUS deployment modes in
> > cache, assuming that our own Binary marshaling is used.
> >
> > Alexey G, given that you are in charge of this merge, can you please make
> > this change as well?
> >
> > Thanks,
> > D.
> >
> > ---------- Forwarded message ----------
> > From: Dmitriy Setrakyan <ds...@apache.org>
> > Date: Tue, Nov 3, 2015 at 11:13 AM
> > Subject: Re: Distributed queue problem with peerClassLoading enabled
> > To: user <us...@ignite.apache.org>
> >
> >
> > Hi Mateusz,
> >
> > In the upcoming Ignite 1.5 release, we will not be storing user-classes
> on
> > the server side at all, so the need for deployment of enqueued objects
> goes
> > away.
> >
> > With that in mind, we will be removing the requirement for caches to work
> > only with SHARED and CONTINUOUS deployment modes, so you will be able to
> > use PRIVATE or ISOLATED deployment modes to deploy your computations.
> >
> > We are currently actively working on finalizing the 1.5 release, so it
> > should be coming real soon.
> >
> > D.
> >
> > On Tue, Nov 3, 2015 at 3:05 AM, mp <mj...@gmail.com> wrote:
> >
> > > Hi Denis,
> > >
> > > Thanks a lot for the clarifications.
> > >
> > > So, is there any way in Ignite to achieve my goal: different versions
> of
> > > the same class running on the cluster (they can be deployed at the same
> > > time from different client nodes)?
> > > If yes, I can think of a way in which I can achieve my goals without
> > using
> > > the distributed queue.
> > >
> > > Cheers,
> > > -Mateusz
> > >
> > >
> > > On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <dm...@gridgain.com>
> > wrote:
> > >
> > >> Hi Mateusz,
> > >>
> > >> Unfortunately I don't see any way to specify userVersion
> > programmatically.
> > >> In any case CONTINIOUS and SHARED mode is not suitable for your task
> > >> because
> > >> the developers may want to have different version of the same class.
> > >> Please
> > >> refer to DeploymentMode documentation for more info on different mode
> > >> types.
> > >>
> > >> Do you really need to use IgniteQueue in your tests because, yes,
> > neither
> > >> PRIVATE nor ISOLATED modes are supported for user caches.
> > >>
> > >> --
> > >> Denis
> > >>
> > >>
> > >>
> > >> --
> > >> View this message in context:
> > >>
> >
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
> > >> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> > >>
> > >
> > >
> >
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Alexey Goncharuk <al...@gmail.com>.
I am not sure what behavior we want to achieve here. Do we want to disable
cache peer class loading for binary marshalling altogether? This
restriction is safe to remove only if user chooses to work exclusively with
Binary object representation.

If user still wants to work with class object representation, SHARED or
CONTINUOUS deployment mode is still a requirement for the same reasons it
was a requirement before.

2015-11-04 11:48 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:

> Igniters,
>
> I want to bring your attention to my email response on the user list. I
> just want to confirm that we are all on the same page and that my
> understanding is correct.
>
> I think that once we merge the BinaryObject to the release branch, we
> should remove the restriction on SHARED or CONTINUOUS deployment modes in
> cache, assuming that our own Binary marshaling is used.
>
> Alexey G, given that you are in charge of this merge, can you please make
> this change as well?
>
> Thanks,
> D.
>
> ---------- Forwarded message ----------
> From: Dmitriy Setrakyan <ds...@apache.org>
> Date: Tue, Nov 3, 2015 at 11:13 AM
> Subject: Re: Distributed queue problem with peerClassLoading enabled
> To: user <us...@ignite.apache.org>
>
>
> Hi Mateusz,
>
> In the upcoming Ignite 1.5 release, we will not be storing user-classes on
> the server side at all, so the need for deployment of enqueued objects goes
> away.
>
> With that in mind, we will be removing the requirement for caches to work
> only with SHARED and CONTINUOUS deployment modes, so you will be able to
> use PRIVATE or ISOLATED deployment modes to deploy your computations.
>
> We are currently actively working on finalizing the 1.5 release, so it
> should be coming real soon.
>
> D.
>
> On Tue, Nov 3, 2015 at 3:05 AM, mp <mj...@gmail.com> wrote:
>
> > Hi Denis,
> >
> > Thanks a lot for the clarifications.
> >
> > So, is there any way in Ignite to achieve my goal: different versions of
> > the same class running on the cluster (they can be deployed at the same
> > time from different client nodes)?
> > If yes, I can think of a way in which I can achieve my goals without
> using
> > the distributed queue.
> >
> > Cheers,
> > -Mateusz
> >
> >
> > On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <dm...@gridgain.com>
> wrote:
> >
> >> Hi Mateusz,
> >>
> >> Unfortunately I don't see any way to specify userVersion
> programmatically.
> >> In any case CONTINIOUS and SHARED mode is not suitable for your task
> >> because
> >> the developers may want to have different version of the same class.
> >> Please
> >> refer to DeploymentMode documentation for more info on different mode
> >> types.
> >>
> >> Do you really need to use IgniteQueue in your tests because, yes,
> neither
> >> PRIVATE nor ISOLATED modes are supported for user caches.
> >>
> >> --
> >> Denis
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
> >> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
> >>
> >
> >
>

Fwd: Distributed queue problem with peerClassLoading enabled

Posted by Dmitriy Setrakyan <ds...@apache.org>.
Igniters,

I want to bring your attention to my email response on the user list. I
just want to confirm that we are all on the same page and that my
understanding is correct.

I think that once we merge the BinaryObject to the release branch, we
should remove the restriction on SHARED or CONTINUOUS deployment modes in
cache, assuming that our own Binary marshaling is used.

Alexey G, given that you are in charge of this merge, can you please make
this change as well?

Thanks,
D.

---------- Forwarded message ----------
From: Dmitriy Setrakyan <ds...@apache.org>
Date: Tue, Nov 3, 2015 at 11:13 AM
Subject: Re: Distributed queue problem with peerClassLoading enabled
To: user <us...@ignite.apache.org>


Hi Mateusz,

In the upcoming Ignite 1.5 release, we will not be storing user-classes on
the server side at all, so the need for deployment of enqueued objects goes
away.

With that in mind, we will be removing the requirement for caches to work
only with SHARED and CONTINUOUS deployment modes, so you will be able to
use PRIVATE or ISOLATED deployment modes to deploy your computations.

We are currently actively working on finalizing the 1.5 release, so it
should be coming real soon.

D.

On Tue, Nov 3, 2015 at 3:05 AM, mp <mj...@gmail.com> wrote:

> Hi Denis,
>
> Thanks a lot for the clarifications.
>
> So, is there any way in Ignite to achieve my goal: different versions of
> the same class running on the cluster (they can be deployed at the same
> time from different client nodes)?
> If yes, I can think of a way in which I can achieve my goals without using
> the distributed queue.
>
> Cheers,
> -Mateusz
>
>
> On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <dm...@gridgain.com> wrote:
>
>> Hi Mateusz,
>>
>> Unfortunately I don't see any way to specify userVersion programmatically.
>> In any case CONTINIOUS and SHARED mode is not suitable for your task
>> because
>> the developers may want to have different version of the same class.
>> Please
>> refer to DeploymentMode documentation for more info on different mode
>> types.
>>
>> Do you really need to use IgniteQueue in your tests because, yes, neither
>> PRIVATE nor ISOLATED modes are supported for user caches.
>>
>> --
>> Denis
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Wed, Nov 4, 2015 at 5:50 AM, mp <mj...@gmail.com> wrote:

>
> But as I understand, in the previous message in this thread Dmitriy stated
> that in Ignite 1.5 it *will* be possible to use distributed cache in both
> PRIVATE and ISOLATED modes. Am I right?
> If this is correct, I assume that I will be able to use PRIVATE or
> ISLOLATED mode (where distributed cache will work, making IgniteQueue
> usable), and in such modes "true" peerClassLoading will also work (ie,
> different versions of the same class from different client nodes at the
> same time). If that is correct, I couldn't dream of a better scenario :)
>

Mateusz, yes you are right. I have also sent another email to the dev list
to confirm this. You can track the discussion here:

http://apache-ignite-developers.2346864.n4.nabble.com/Fwd-Distributed-queue-problem-with-peerClassLoading-enabled-td4521.html

D.

Re: Distributed queue problem with peerClassLoading enabled

Posted by mp <mj...@gmail.com>.
Denis,

Please see below.

On Wed, Nov 4, 2015 at 1:51 PM, Denis Magda <dm...@gridgain.com> wrote:

> Hi Mateusz,
>
> Please see below
>
> On 11/4/2015 11:13 AM, mp wrote:
>
> Hi Dmitriy,
>
> That is great news! I will definitely try out the new version as soon as
> it's released.
>
> Is there any chance to include a fix for
> <https://issues.apache.org/jira/browse/IGNITE-1823>
> https://issues.apache.org/jira/browse/IGNITE-1823 in the upcoming version
> (see the message from Artem earlier in this thread)?
>
> It depends on whether anyone in the community will pick it up or not.
>
> In any case I still wouldn't recommend you to use the SHARED deployment
> mode if multiple developers actively testing something in a cluster. One
> day you may face with the issue when one developer's class is overriden by
> a new version of the same class from the other developer.
>

OK, got it.


>
>
>
> Would it be possible to achieve the following in the new version (this is
> a typical scenario in my setup): a cluster of server nodes on powerful
> machines, two analysts/developers are actively testing some prototype
> algorithm and are using the same class *with different code* and running
> their computations on the cluster *at the same time*. Each computation uses
> different code for the same class (the code exists only on the developer's
> client node)?
>
> Are you going to work with any cache in a compute task body?
> - If you aren't going then you can freely enable PRIVATE or ISOLATE mode
> (to exclude a possible case when one developer's class is overridden by a
> new class's version from the other developer) and start using Compute
> Engine;
>
- If you are going than try to give a shot to SHARED mode but use Compute
> Engine instead of IgniteQueue. IGNITE-1823 is reproduced with IgniteQueue
> but probably this will work with Compute Engine.
>

But as I understand, in the previous message in this thread Dmitriy stated
that in Ignite 1.5 it *will* be possible to use distributed cache in both
PRIVATE and ISOLATED modes. Am I right?
If this is correct, I assume that I will be able to use PRIVATE or
ISLOLATED mode (where distributed cache will work, making IgniteQueue
usable), and in such modes "true" peerClassLoading will also work (ie,
different versions of the same class from different client nodes at the
same time). If that is correct, I couldn't dream of a better scenario :)

Thanks to all of you for such prompt support!

Cheers,
-Mateusz

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Hi Mateusz,

Please see below

On 11/4/2015 11:13 AM, mp wrote:
> Hi Dmitriy,
>
> That is great news! I will definitely try out the new version as soon 
> as it's released.
>
> Is there any chance to include a fix for 
> https://issues.apache.org/jira/browse/IGNITE-1823 in the upcoming 
> version (see the message from Artem earlier in this thread)?
It depends on whether anyone in the community will pick it up or not.

In any case I still wouldn't recommend you to use the SHARED deployment 
mode if multiple developers actively testing something in a cluster. One 
day you may face with the issue when one developer's class is overriden 
by a new version of the same class from the other developer.

> Would it be possible to achieve the following in the new version (this 
> is a typical scenario in my setup): a cluster of server nodes on 
> powerful machines, two analysts/developers are actively testing some 
> prototype algorithm and are using the same class *with different code* 
> and running their computations on the cluster *at the same time*. Each 
> computation uses different code for the same class (the code exists 
> only on the developer's client node)?
>
Are you going to work with any cache in a compute task body?
- If you aren't going then you can freely enable PRIVATE or ISOLATE mode 
(to exclude a possible case when one developer's class is overridden by 
a new class's version from the other developer) and start using Compute 
Engine;
- If you are going than try to give a shot to SHARED mode but use 
Compute Engine instead of IgniteQueue. IGNITE-1823 is reproduced with 
IgniteQueue but probably this will work with Compute Engine.

Dmitriy is clarifying regarding the way the new version, he mentioned 
below, on the dev list. Should reply you soon.

--
Denis

> Thanks a lot for sharing the outlook for the upcoming release!
>
> Cheers,
> -Mateusz
>
> On Tue, Nov 3, 2015 at 8:13 PM, Dmitriy Setrakyan 
> <dsetrakyan@apache.org <ma...@apache.org>> wrote:
>
>     Hi Mateusz,
>
>     In the upcoming Ignite 1.5 release, we will not be storing
>     user-classes on the server side at all, so the need for deployment
>     of enqueued objects goes away.
>
>     With that in mind, we will be removing the requirement for caches
>     to work only with SHARED and CONTINUOUS deployment modes, so you
>     will be able to use PRIVATE or ISOLATED deployment modes to deploy
>     your computations.
>
>     We are currently actively working on finalizing the 1.5 release,
>     so it should be coming real soon.
>
>     D.
>
>     On Tue, Nov 3, 2015 at 3:05 AM, mp <mjjp00@gmail.com
>     <ma...@gmail.com>> wrote:
>
>         Hi Denis,
>
>         Thanks a lot for the clarifications.
>
>         So, is there any way in Ignite to achieve my goal: different
>         versions of the same class running on the cluster (they can be
>         deployed at the same time from different client nodes)?
>         If yes, I can think of a way in which I can achieve my goals
>         without using the distributed queue.
>
>         Cheers,
>         -Mateusz
>
>
>         On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda
>         <dmagda@gridgain.com <ma...@gridgain.com>> wrote:
>
>             Hi Mateusz,
>
>             Unfortunately I don't see any way to specify userVersion
>             programmatically.
>             In any case CONTINIOUS and SHARED mode is not suitable for
>             your task because
>             the developers may want to have different version of the
>             same class. Please
>             refer to DeploymentMode documentation for more info on
>             different mode types.
>
>             Do you really need to use IgniteQueue in your tests
>             because, yes, neither
>             PRIVATE nor ISOLATED modes are supported for user caches.
>
>             --
>             Denis
>
>
>
>             --
>             View this message in context:
>             http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
>             Sent from the Apache Ignite Users mailing list archive at
>             Nabble.com.
>
>
>
>


Re: Distributed queue problem with peerClassLoading enabled

Posted by mp <mj...@gmail.com>.
Hi Dmitriy,

That is great news! I will definitely try out the new version as soon as
it's released.

Is there any chance to include a fix for
https://issues.apache.org/jira/browse/IGNITE-1823 in the upcoming version
(see the message from Artem earlier in this thread)?
Would it be possible to achieve the following in the new version (this is a
typical scenario in my setup): a cluster of server nodes on powerful
machines, two analysts/developers are actively testing some prototype
algorithm and are using the same class *with different code* and running
their computations on the cluster *at the same time*. Each computation uses
different code for the same class (the code exists only on the developer's
client node)?

Thanks a lot for sharing the outlook for the upcoming release!

Cheers,
-Mateusz

On Tue, Nov 3, 2015 at 8:13 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> Hi Mateusz,
>
> In the upcoming Ignite 1.5 release, we will not be storing user-classes on
> the server side at all, so the need for deployment of enqueued objects goes
> away.
>
> With that in mind, we will be removing the requirement for caches to work
> only with SHARED and CONTINUOUS deployment modes, so you will be able to
> use PRIVATE or ISOLATED deployment modes to deploy your computations.
>
> We are currently actively working on finalizing the 1.5 release, so it
> should be coming real soon.
>
> D.
>
> On Tue, Nov 3, 2015 at 3:05 AM, mp <mj...@gmail.com> wrote:
>
>> Hi Denis,
>>
>> Thanks a lot for the clarifications.
>>
>> So, is there any way in Ignite to achieve my goal: different versions of
>> the same class running on the cluster (they can be deployed at the same
>> time from different client nodes)?
>> If yes, I can think of a way in which I can achieve my goals without
>> using the distributed queue.
>>
>> Cheers,
>> -Mateusz
>>
>>
>> On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <dm...@gridgain.com> wrote:
>>
>>> Hi Mateusz,
>>>
>>> Unfortunately I don't see any way to specify userVersion
>>> programmatically.
>>> In any case CONTINIOUS and SHARED mode is not suitable for your task
>>> because
>>> the developers may want to have different version of the same class.
>>> Please
>>> refer to DeploymentMode documentation for more info on different mode
>>> types.
>>>
>>> Do you really need to use IgniteQueue in your tests because, yes, neither
>>> PRIVATE nor ISOLATED modes are supported for user caches.
>>>
>>> --
>>> Denis
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>
>>
>>
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Dmitriy Setrakyan <ds...@apache.org>.
Hi Mateusz,

In the upcoming Ignite 1.5 release, we will not be storing user-classes on
the server side at all, so the need for deployment of enqueued objects goes
away.

With that in mind, we will be removing the requirement for caches to work
only with SHARED and CONTINUOUS deployment modes, so you will be able to
use PRIVATE or ISOLATED deployment modes to deploy your computations.

We are currently actively working on finalizing the 1.5 release, so it
should be coming real soon.

D.

On Tue, Nov 3, 2015 at 3:05 AM, mp <mj...@gmail.com> wrote:

> Hi Denis,
>
> Thanks a lot for the clarifications.
>
> So, is there any way in Ignite to achieve my goal: different versions of
> the same class running on the cluster (they can be deployed at the same
> time from different client nodes)?
> If yes, I can think of a way in which I can achieve my goals without using
> the distributed queue.
>
> Cheers,
> -Mateusz
>
>
> On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <dm...@gridgain.com> wrote:
>
>> Hi Mateusz,
>>
>> Unfortunately I don't see any way to specify userVersion programmatically.
>> In any case CONTINIOUS and SHARED mode is not suitable for your task
>> because
>> the developers may want to have different version of the same class.
>> Please
>> refer to DeploymentMode documentation for more info on different mode
>> types.
>>
>> Do you really need to use IgniteQueue in your tests because, yes, neither
>> PRIVATE nor ISOLATED modes are supported for user caches.
>>
>> --
>> Denis
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by mp <mj...@gmail.com>.
Hi Denis,

Thanks a lot for the clarifications.

So, is there any way in Ignite to achieve my goal: different versions of
the same class running on the cluster (they can be deployed at the same
time from different client nodes)?
If yes, I can think of a way in which I can achieve my goals without using
the distributed queue.

Cheers,
-Mateusz


On Tue, Nov 3, 2015 at 11:53 AM, Denis Magda <dm...@gridgain.com> wrote:

> Hi Mateusz,
>
> Unfortunately I don't see any way to specify userVersion programmatically.
> In any case CONTINIOUS and SHARED mode is not suitable for your task
> because
> the developers may want to have different version of the same class. Please
> refer to DeploymentMode documentation for more info on different mode
> types.
>
> Do you really need to use IgniteQueue in your tests because, yes, neither
> PRIVATE nor ISOLATED modes are supported for user caches.
>
> --
> Denis
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Hi Mateusz,

Unfortunately I don't see any way to specify userVersion programmatically.
In any case CONTINIOUS and SHARED mode is not suitable for your task because
the developers may want to have different version of the same class. Please
refer to DeploymentMode documentation for more info on different mode types.

Do you really need to use IgniteQueue in your tests because, yes, neither
PRIVATE nor ISOLATED modes are supported for user caches.

--
Denis



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1820.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Distributed queue problem with peerClassLoading enabled

Posted by mp <mj...@gmail.com>.
Hi Denis,

I tried PRIVATE deployment mode but either I'm doing something wrong or
distributed cache is not supported for this (and ISOLATED) mode. Is that so?
When queue creation method is called, ie, the line:

      IgniteQueue<Task> queue = ignite.queue("myTaskQueue", 0, new
CollectionConfiguration());

I get the error message: Cannot start cache in PRIVATE or ISOLATED
deployment mode: PRIVATE
(please see bottom of message for full stack trace)

How can I set the userVersion in client code that just starts an ad-hoc
node (configuration in code, node started with
Ignition.start(IgniteConfiguration cfg) method), and then shuts it down
when the client test is done?
Sorry if this can be found in the documentation - I tried to find it but
failed.
Anyway, changing the userVersion in client code (or its configuration)
every time a class is changed will probably not be a valid option for me.
People will certainly sometimes forget to do it. What's dangerous here it
that in such cases (forgetting to change the userVersion) the test will run
on the remote nodes with obsolete code and provide obsolete results, which
I imagine, could lead to confusion.

Cheers,
-Mateusz


stack trace follows:


[14:33:41,907][SEVERE][exchange-worker-#79%null%][GridDhtPartitionsExchangeFuture]
Failed to reinitialize local partitions (preloading will be stopped):
GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4,
minorTopVer=1], nodeId=97b2c06e, evt=DISCOVERY_CUSTOM_EVT]
class org.apache.ignite.IgniteCheckedException: Cannot start cache in
PRIVATE or ISOLATED deployment mode: PRIVATE
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:408)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1268)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1642)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCachesStart(GridCacheProcessor.java:1567)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.startCaches(GridDhtPartitionsExchangeFuture.java:930)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:506)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1182)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)
[14:33:41,907][SEVERE][exchange-worker-#79%null%][GridCachePartitionExchangeManager]
Failed to wait for completion of partition map exchange (preloading will
not start): GridDhtPartitionsExchangeFuture [dummy=false,
forcePreload=false, reassign=false, discoEvt=DiscoveryCustomEvent
[customMsg=DynamicCacheChangeBatch [reqs=[DynamicCacheChangeRequest
[deploymentId=dab0768c051-934738fd-3379-458b-b21c-db9061a455c3,
startCfg=CacheConfiguration [name=datastructures_0, rebalancePoolSize=2,
rebalanceTimeout=10000, evictPlc=null, evictSync=false,
evictKeyBufSize=1024, evictSyncConcurrencyLvl=4, evictSyncTimeout=10000,
evictFilter=null, evictMaxOverflowRatio=10.0, eagerTtl=true,
dfltLockTimeout=0, startSize=1500000, nearCfg=null, writeSync=FULL_SYNC,
storeFactory=null, loadPrevVal=false,
aff=o.a.i.cache.affinity.rendezvous.RendezvousAffinityFunction@940d130,
cacheMode=PARTITIONED, atomicityMode=ATOMIC, atomicWriteOrderMode=PRIMARY,
backups=0, invalidate=false, tmLookupClsName=null, rebalanceMode=SYNC,
rebalanceOrder=0, rebalanceBatchSize=524288, offHeapMaxMem=-1,
swapEnabled=false, maxConcurrentAsyncOps=500, writeBehindEnabled=false,
writeBehindFlushSize=10240, writeBehindFlushFreq=5000,
writeBehindFlushThreadCnt=1, writeBehindBatchSize=512,
memMode=ONHEAP_TIERED,
affMapper=o.a.i.i.processors.cache.GridCacheDefaultAffinityKeyMapper@693c7236,
rebalanceDelay=0, rebalanceThrottle=0, interceptor=null,
longQryWarnTimeout=3000, readFromBackup=true,
nodeFilter=o.a.i.configuration.CacheConfiguration$IgniteAllNodesPredicate@655b7adf,
sqlEscapeAll=false, sqlOnheapRowCacheSize=10240, cpOnRead=true,
topValidator=null], cacheType=INTERNAL,
initiatingNodeId=97b2c06e-ea4f-4e6a-a4dd-9e7081056d30, nearCacheCfg=null,
clientStartOnly=false, stop=false, close=false, failIfExists=false,
template=false, exchangeNeeded=true, cacheName=datastructures_0]],
clientNodes=null, id=eab0768c051-934738fd-3379-458b-b21c-db9061a455c3,
clientReconnect=false], affTopVer=AffinityTopologyVersion [topVer=4,
minorTopVer=1], super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=97b2c06e-ea4f-4e6a-a4dd-9e7081056d30, addrs=[0:0:0:0:0:0:0:1,
10.48.1.58, 127.0.0.1], sockAddrs=[
DEVSRV4807.deltavista.com/10.48.1.58:47501, /0:0:0:0:0:0:0:1:47501, /
10.48.1.58:47501, /127.0.0.1:47501], discPort=47501, order=4, intOrder=3,
lastExchangeTime=1446471221907, loc=true, ver=1.4.0#20150924-sha1:c2def5f6,
isClient=false], topVer=4, nodeId8=97b2c06e, msg=null,
type=DISCOVERY_CUSTOM_EVT, tstamp=1446471221907]],
rcvdIds=GridConcurrentHashSet [elements=[]], rmtIds=null,
exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion
[topVer=4, minorTopVer=1], nodeId=97b2c06e, evt=DISCOVERY_CUSTOM_EVT],
init=true, ready=false, replied=false, added=true,
initFut=GridFutureAdapter [resFlag=2, res=false, startTime=1446471221907,
endTime=1446471221907, ignoreInterrupts=false, lsnr=null, state=DONE],
topSnapshot=null, lastVer=null, partReleaseFut=null, skipPreload=false,
clientOnlyExchange=false, oldest=80a54708-220e-45d9-bca4-2269ba1d3c15,
oldestOrder=1, evtLatch=0, remaining=[], super=GridFutureAdapter
[resFlag=1, res=class o.a.i.IgniteCheckedException: Cannot start cache in
PRIVATE or ISOLATED deployment mode: PRIVATE, startTime=1446471221907,
endTime=1446471221907, ignoreInterrupts=false, lsnr=null, state=DONE]]
class org.apache.ignite.IgniteCheckedException: Cannot start cache in
PRIVATE or ISOLATED deployment mode: PRIVATE
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.validate(GridCacheProcessor.java:408)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1268)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1642)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCachesStart(GridCacheProcessor.java:1567)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.startCaches(GridDhtPartitionsExchangeFuture.java:930)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:506)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1182)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:745)

Re: Distributed queue problem with peerClassLoading enabled

Posted by Denis Magda <dm...@gridgain.com>.
Hi Mateusz,

Seems that either PRIVATE or ISOLATED deployment mode perfectly fits for
your task. Please try to switch to one of them.

CONTINUOUS mode reloads previously deployed classes only when 'userInfo'
changes. Refer to the documentation of this node.
In short to fix this you should increase the userVersion on the node that
changed Task class implementation

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
    <description>Ignite Spring configuration file in gar-file.</description>

    <bean id="userVersion" class="java.lang.String">
        <constructor-arg value="2"/>
    </bean>
</beans>


> Then I tried to change the serialVersionUID of the class to somehow
> "force" the redeployment on the > class but this caused a problem on the
> client side, when attempting to fill the queue (please see below > for
> stack trace). This is especially strange because the queue was empty, so
> how come the old serialVersionUID prevents filling the queue?

This is an expected exception. If you want to change serialVersionUID and
you use CONTINIOUS mode then you have to restart the whole cluster. 

Regards,
Denis



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-queue-problem-with-peerClassLoading-enabled-tp1762p1809.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.