You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Gert Dubois <ge...@trendminer.com> on 2019/01/04 15:48:34 UTC
Re: Dealing with changing class definitions in Ignite
Denis, All
First of all best wishes for the New Year.
We understand the limitations concerning data that is stored in caches.
But I feel it's quite limiting to apply these restrictions as well to
runnables, it really makes dealing with changing code in runnables very
annoying. And it prevents 2 clients with conflicting versions of runnables
to execute code on the same cluster.
Gert
On Mon, Dec 24, 2018 at 7:56 PM Denis Magda <dm...@apache.org> wrote:
> Gert, All,
>
> That's not a peer-class-loading issue but rather the binary objects
> limitation. It's impossible to change the *type* of an object field. You
> can add new fields or remove existing ones. The same limitation applies to
> SQL - now way to change the type in runtime. That latter should be
> addressed in 2019.
>
> Vladimir, shouldn't it be easier to support fields type change for compute
> runnables which are not stored in caches/tables?
>
> --
> Denis
>
> On Thu, Dec 20, 2018 at 4:24 AM Gert Dubois <ge...@trendminer.com>
> wrote:
>
>> For now we can work around the issue by overriding the
>> BinaryNameMarshaller and ensuring our clients append a unique identifier
>> behind the class name. This ensures that every client gets their own Binary
>> Metadata records in the Marshaller for every class they serialize in the
>> cluster.
>>
>> .setNameMapper(new BinaryNameMapper() {
>> @Override
>> public String typeName(String clsName) {
>> if(shouldAddSuffix) {
>> return clsName + "_clientVersionId";
>> }
>> return clsName;
>> }
>>
>> @Override
>> public String fieldName(String fieldName) {
>> return fieldName;
>> }
>> });
>>
>> But we expected the ISOLATED deployment mode would have solved this issue
>> for us. It also forces us to maintain some kind of client versioning system
>> so when the classes in our jobs change they get a fresh record in the
>> Marshaller.
>>
>> On Wed, Dec 19, 2018 at 6:30 PM Ilya Kasnacheev <
>> ilya.kasnacheev@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> I have found that you can avoid compute job type incompatibility problem
>>> (please see history below) by setting
>>>
>>> .setMarshaller(new OptimizedMarshaller())
>>>
>>> in your Ignite configuration on all nodes.
>>>
>>> However, it is not clear at all why this is needed. Anybody can help?
>>> Why compute jobs are processed by capricious BinaryMarshaller?
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> ср, 19 дек. 2018 г. в 02:29, Gert Dubois <ge...@trendminer.com>:
>>>
>>>> Hey Ilya,
>>>>
>>>> I opened a ticket on the ignite Jira about it.
>>>> https://issues.apache.org/jira/browse/IGNITE-10717
>>>>
>>>> I attached a zip file containing a maven project with sample code that
>>>> reproduces our issue. Reproducing the issue is rather easy though
>>>>
>>>> 1. Have a client + server, with peer class loading enabled
>>>> 2. Create a simple class that implements IgniteRunnable with some class
>>>> field
>>>> 3. Execute an instance of this class on the ignite cluster
>>>> 4. Change the type of the field
>>>> 5. Recompile and execute again. Now it breaks because the class can't
>>>> be serialized using the binary marshaller.
>>>>
>>>> Code to execute these steps is included in the maven project.
>>>>
>>>> On Tue, Dec 18, 2018 at 12:04 PM Ilya Kasnacheev <
>>>> ilya.kasnacheev@gmail.com> wrote:
>>>>
>>>>> Hello!
>>>>>
>>>>> Can you show an example of "Ignite Runnables conflicting on the Binary
>>>>> Marshaller"? As a small code snippet perhaps?
>>>>>
>>>>> Maybe I could recommend something but I lack understanding of your use
>>>>> case.
>>>>>
>>>>> Regards,
>>>>> --
>>>>> Ilya Kasnacheev
>>>>>
>>>>>
>>>>> пн, 17 дек. 2018 г. в 18:12, Gert Dubois <ge...@trendminer.com>:
>>>>>
>>>>>> Thanks for the reply.
>>>>>>
>>>>>> Everything related to the cache we understand the current
>>>>>> architecture, in our code base we'll probably treat everything cache
>>>>>> related the same as schema migrations in a DB (migration scripts, etc.).
>>>>>> Our real issue is with Ignite Runnables conflicting on the Binary
>>>>>> Marshaller. Every class that gets executed on Ignite as a job gets a stored
>>>>>> definition, this means we can't refactor classes that get used internally
>>>>>> by our Ignite Runnables, because doing so would prevent the updated code
>>>>>> from running. Even worse, if we update our libraries and they changed class
>>>>>> definitions we might run into the same issue, without changing a letter in
>>>>>> our own code. From the documentation it looked like Deployment Mode could
>>>>>> provide a solution for this issue but the Binary Marshaller seems to run
>>>>>> completely separate from the Deployment Mode.
>>>>>>
>>>>>> On Mon, Dec 17, 2018 at 3:18 PM Ilya Kasnacheev <
>>>>>> ilya.kasnacheev@gmail.com> wrote:
>>>>>>
>>>>>>> Hello!
>>>>>>>
>>>>>>> As far as my understanding goes:
>>>>>>> You can't peer class load your Key/Value types.
>>>>>>> You also can't redeploy your Key/Value types.
>>>>>>> They even survive node restart via WORKDIR/marshaller directory, and
>>>>>>> come back to haunt you.
>>>>>>>
>>>>>>> There are plans to maybe ease some of those limitations in 3.0, but
>>>>>>> nothing concrete yet. It is not a bug rather a pillar of current Ignite
>>>>>>> architecture. You will have to route around it, such as introducing new
>>>>>>> fields instead of changing types. And maybe avoid having those types on
>>>>>>> server nodes at all, and relying on BinaryObject.
>>>>>>>
>>>>>>> Regards,
>>>>>>> --
>>>>>>> Ilya Kasnacheev
>>>>>>>
>>>>>>>
>>>>>>> пн, 17 дек. 2018 г. в 15:38, Gert Dubois <gert.dubois@trendminer.com
>>>>>>> >:
>>>>>>>
>>>>>>>> The issue is still present in 2.7.
>>>>>>>> I added a ticket on Jira with sample code that reproduces the issue.
>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-10717
>>>>>>>>
>>>>>>>> For now I think we can work around the issue by overriding the
>>>>>>>> default BinaryNameMapper, but this feels rather hacky to me.
>>>>>>>>
>>>>>>>> On Mon, Dec 17, 2018 at 11:54 AM Denis Mekhanikov <
>>>>>>>> dmekhanikov@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Gert,
>>>>>>>>>
>>>>>>>>> Could you check if the problem with a deployment mode reproduces
>>>>>>>>> on Ignite 2.7?
>>>>>>>>> If it does, please file a ticket with an explanation and a
>>>>>>>>> reproducer to https://issues.apache.org/jira/
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> Denis
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> пн, 17 дек. 2018 г. в 12:12, Gert Dubois <
>>>>>>>>> gert.dubois@trendminer.com>:
>>>>>>>>>
>>>>>>>>>> I investigated the issue further and narrowed the issue down to
>>>>>>>>>> the Binary Marshaller not working as expected given the configured
>>>>>>>>>> Deployment Mode. When forcing my clients to use unique class names in the
>>>>>>>>>> metadata of the Binary Marshaller (I forced this by overriding the global
>>>>>>>>>> BinaryNameMapper and appending a per-client unique suffix to every class
>>>>>>>>>> name) the Deployment Mode behaves as expected. I assume the Binary
>>>>>>>>>> Marshaller keeps a cluster wide state of the Metadata of classes and it
>>>>>>>>>> merges it whenever we serialize a class on a node (regardless of the
>>>>>>>>>> configured Deployment Mode).
>>>>>>>>>> Why is the behaviour of the Binary Marshaller not consistent with
>>>>>>>>>> the way the Deployment Mode works? Is there a cleaner way to solve this,
>>>>>>>>>> besides overriding the BinaryNameMapper?
>>>>>>>>>>
>>>>>>>>>> On Fri, Dec 14, 2018 at 1:07 PM Gert Dubois <
>>>>>>>>>> gert.dubois@trendminer.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> On my current project we are confronted with an issue we are
>>>>>>>>>>> struggling to figure out.
>>>>>>>>>>> We have a simple topology where we have a client node running an
>>>>>>>>>>> IgniteRunnable to a server node. Both nodes have peerClassLoading=true and
>>>>>>>>>>> the default binaryConfiguration with compactFooters=false. The client is
>>>>>>>>>>> started with clientMode=true
>>>>>>>>>>> After execution we shut the client down and refactor the field
>>>>>>>>>>> type of one of the fields in our IgniteRunnable. When we now restart the
>>>>>>>>>>> client we get the following exception:
>>>>>>>>>>>
>>>>>>>>>>> Caused by: org.apache.ignite.binary.BinaryObjectException:
>>>>>>>>>>> Binary type has different field types
>>>>>>>>>>> [typeName=com.trendminer.compute.ClientJob, fieldName=param2,
>>>>>>>>>>> fieldTypeName1=double, fieldTypeName2=int]
>>>>>>>>>>>
>>>>>>>>>>> We tried different Deployment modes, ISOLATED, SHARED/CONTINUOUS
>>>>>>>>>>> with different userVersion. But nothing we change enables us to execute the
>>>>>>>>>>> new class definition on the server node.
>>>>>>>>>>>
>>>>>>>>>>> Ignite Version is 2.4.0
>>>>>>>>>>>
>>>>>>>>>>> Since we are using persistence in our production environment and
>>>>>>>>>>> this causes class definitions for the binary marshaller to be remembered
>>>>>>>>>>> even after cluster restart this is blocking for us since this either forces
>>>>>>>>>>> us to manually clear out the binary data and restart our cluster or it
>>>>>>>>>>> prevents us from refactoring code we we send into Ignite, which is not
>>>>>>>>>>> always possible.
>>>>>>>>>>> So we would like to know what the intended way is to deal with
>>>>>>>>>>> changing class definitions of tasks sent into Ignite?
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Gert
>>>>>>>>>>>
>>>>>>>>>>