You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by 姜 为 <yd...@gmail.com> on 2015/11/08 13:27:41 UTC
OptimizedClassDescriptor verifyChecksum object set as optional
Hi guys:
I’m using ignite 1.4.
In IgniteCompute.call will transfer of an object to the cluster.
The object should implement Serializable or Externalizable interface.
OptimizedClassDescriptor.read method will check whether the object is in the same class.
In my use case,I have some type of servers in cluster.
The server type A will check the business,and the server type B will persistent data.
There is a entity interface Entity extends Externalizable have different implementations on different servers.
Such like this:
interface Entity extends Externalizable {
method a();
method b();
method c();
}
class ServiceEntity implements Entity {
method a(){
// do something...
}
method b(){
// do something...
}
method c(){
throw new UnsupportedException...
}
Externalizable.read...
Externalizable.write...
}
class DataEntity implements Entity {
method a(){
// do something...
}
method b(){
throw new UnsupportedException...
}
method c(){
// do something...
}
Externalizable.read...
Externalizable.write...
}
And IgniteCompute.call(new IgniteCallable(
public Object call(){
Entity.a() or b and c;..
}
));
Different implementations of the same class are to achieve read and write methods.
But OptimizedClassDescriptor.read will check the class sum and throw ClassNotFoundException.
I recommend verifyChecksum object set as optional,and I really need is change.
Here is my pr:
https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>
https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by 姜 为 <yd...@gmail.com>.
Hi Denis,
This is not what I want ,but it works for my service.
Thank you!
Jiang
> 在 2015年11月11日，下午9:00，Denis Magda <dm...@gridgain.com> 写道：
>
> Hi Jiang,
>
> I'm a bit confused why you can't give different names to the services located in different modules. If each service has a different name then my approach will work for you, doesn't it?
>
> But if my service is by ModuleA.ExampleServiceImpl call ModuleB.ExampleServiceImpl,
> and ModuleB.ExampleServiceImpl call other complicated service.
>
>
> Could you provide me with the code showing what you're trying to achieve.
>
> BTW, you use service notion in the thread. Probably you can implement everything using Ignite Services module:
> https://apacheignite.readme.io/docs/service-grid <https://apacheignite.readme.io/docs/service-grid>
> What do you think?
>
>
> Regards,
> Denis
>
> On 11/11/2015 5:54 AM, 姜 为 wrote:
>> Hi Denis:
>>
>> I tried compute.call(new ExampleServiceImplA());
>>
>> But if my service is by ModuleA.ExampleServiceImpl call ModuleB.ExampleServiceImpl,
>> and ModuleB.ExampleServiceImpl call other complicated service.
>>
>> The ModuleA only has ExampleServiceImpl there is none ModuleB.other service.
>> It does not work.
>>
>>
>>> 在 2015年11月10日，下午9:55，Denis Magda <dmagda@gridgain.com <ma...@gridgain.com> <mailto:dmagda@gridgain.com <ma...@gridgain.com>>> 写道：
>>>
>>> It should fail because you send your service impl inside of Callable
>>> compute.call(new Callable(new ExampleServiceImplA()));
>>>
>>> Callable is loaded by application class loader and presents on both the sender and receiver. This leads to the situation when the receiver tries to loads ExampleServiceImplA using the app class loader as well.
>>>
>>> To avoid this try to implement IgniteCallable by ExampleServiceImplA and send the task this way:
>>>
>>> compute.call(new ExampleServiceImplA());
>>>
>>>
>>>> I think. If I have lots of type servers. It will be ExampleServiceImpl1 2 3 4 5….,will look very bloated.
>>> You can use java 8 closures sending a particular one depending on a server type. Will it work for you?
>>> https://apacheignite.readme.io/docs/distributed-closures <https://apacheignite.readme.io/docs/distributed-closures> <https://apacheignite.readme.io/docs/distributed-closures <https://apacheignite.readme.io/docs/distributed-closures>>
>>>
>>>
>>> Regards,
>>> Denis
>>> On 11/10/2015 4:31 PM, 姜 为 wrote:
>>>> Hi Denis,
>>>>
>>>> I tried your suggested, here is the code : <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497 <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497>>https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497 <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497> <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497 <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497>>
>>>>
>>>> But I still got ClassNotFoundException.
>>>>
>>>> <邮件附件.png>
>>>>
>>>>
>>>> I think. If I have lots of type servers. It will be ExampleServiceImpl1 2 3 4 5….,will look very bloated.
>>>>
>>>>
>>>>> 在 2015年11月10日，下午8:02，Denis Magda < <mailto:dmagda@gridgain.com <ma...@gridgain.com>>dmagda@gridgain.com <ma...@gridgain.com><mailto:dmagda@gridgain.com <ma...@gridgain.com>>> 写道：
>>>>>
>>>>> Jiang,
>>>>>
>>>>> This exception happens exactly because you have two implementations with the same name. If the names were different you wouldn't get to the point of checksum validation.
>>>>> Here I fully share Val's opinion that checksum's verification shouldn't be optional and it's a responsibility of an application to take care of such situations.
>>>>>
>>>>> I see the following approaches you can use in your code to get required behavior:
>>>>>
>>>>> 1) As suggested below use the approach with Serializable + serialVersionUID. It will work;
>>>>>
>>>>> 2) More preferable is to have different implementations with different names (ExampleServiceImpl1, ExampleServiceImpl2, etc...). Is there any reasons you can't use this straightforward approach?
>>>>> When the implementations are ready you can put their classes onto every machine in order to have them in the classpath or you can leverage IgniteConfiguration.peerClassLoadingEnabled feature.
>>>>> The feature allows to load missing classes from a machine that sends compute based tasks onto a machine that will execute a task and you don't need to copy implementations' classes manually at all.
>>>>> You can read more on this here: https://apacheignite.readme.io/docs/zero-deployment <https://apacheignite.readme.io/docs/zero-deployment> <https://apacheignite.readme.io/docs/zero-deployment <https://apacheignite.readme.io/docs/zero-deployment>>
>>>>>
>>>>> Regards,
>>>>> Denis
>>>>>
>>>>> On 11/10/2015 10:39 AM, 姜 为 wrote:
>>>>>> Hi:
>>>>>>
>>>>>> Here is example: https://github.com/wmz7year/ignite-test <https://github.com/wmz7year/ignite-test> <https://github.com/wmz7year/ignite-test <https://github.com/wmz7year/ignite-test>>
>>>>>>
>>>>>> First start ModuleB, then start ModuleA.
>>>>>>
>>>>>> ModuleB ExampleServiceImpl has three field.
>>>>>>
>>>>>> ModuleA ExampleServiceImpl has only one field.
>>>>>>
>>>>>> Then throw Exception :
>>>>>>
>>>>>> <邮件 附件.png>
>>>>>>> 在 2015年11月10日，下午2:48，Denis Magda <dmagda@gridgain.com <ma...@gridgain.com> <mailto:dmagda@gridgain.com <ma...@gridgain.com>>> 写道：
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> +1 to Val.
>>>>>>>
>>>>>>> Actually if you have different class names on different machines (DataEntity, ServiceEntity) I don't realize how you can get to the checksum validation stage.
>>>>>>> In my understanding you should have caught ClassNotFoundException.
>>>>>>>
>>>>>>> Provide me with a full runnable example, I'll run and see what you're trying to implement and at which point you fail.
>>>>>>>
>>>>>>> --
>>>>>>> Denis
>>>>>>>
>>>>>>> On 11/10/2015 9:30 AM, Valentin Kulichenko wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I'm against optional checksum verification. It's not safe, adds one more
>>>>>>>> configuration property and I don't see any use case that can require this.
>>>>>>>>
>>>>>>>> I also don't completely understand what you're trying to achieve. Can you
>>>>>>>> please describe the sequence of serialization/deserialization events that
>>>>>>>> you expect in your application?
>>>>>>>>
>>>>>>>> -Val
>>>>>>>>
>>>>>>>> On Mon, Nov 9, 2015 at 5:13 PM, 姜 为 < <mailto:ydswcy513@gmail.com <ma...@gmail.com>>ydswcy513@gmail.com <ma...@gmail.com><mailto:ydswcy513@gmail.com <ma...@gmail.com>>> wrote:
>>>>>>>>
>>>>>>>>> Hi:
>>>>>>>>>
>>>>>>>>> The implementations name maybe not same.
>>>>>>>>>
>>>>>>>>> Like my example:
>>>>>>>>>
>>>>>>>>> class DataEntity {
>>>>>>>>> Integer id;
>>>>>>>>> string name;
>>>>>>>>> Integer age;
>>>>>>>>>
>>>>>>>>> ….. and other
>>>>>>>>>
>>>>>>>>> Externalizable.read(in){
>>>>>>>>> if(in.readBoolean()) {
>>>>>>>>> id = in.readInt();
>>>>>>>>> }
>>>>>>>>> same as name,age...
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Externalizable.write(out){
>>>>>>>>> out.writeBoolean(id != null);
>>>>>>>>> if(id != null) {
>>>>>>>>> out.writeInt(id);
>>>>>>>>> }
>>>>>>>>> same as name,age...
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> class ServiceEntity {
>>>>>>>>> Integer id;
>>>>>>>>> Integer age;
>>>>>>>>>
>>>>>>>>> …. and other
>>>>>>>>>
>>>>>>>>> Externalizable.read(in){
>>>>>>>>> if(in.readBoolean()) {
>>>>>>>>> id = in.readInt();
>>>>>>>>> }
>>>>>>>>> if(in.readBoolean()) {
>>>>>>>>> in.readString(); // ignore name
>>>>>>>>> }
>>>>>>>>> if(in.readBoolean()) {
>>>>>>>>> age = in.readInt();
>>>>>>>>> }
>>>>>>>>> … and other
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Externalizable.write(out){
>>>>>>>>> out.writeBoolean(id != null);
>>>>>>>>> if(id != null) {
>>>>>>>>> out.writeInt(id);
>>>>>>>>> }
>>>>>>>>> out.writeBoolean(false); // null for name
>>>>>>>>> out.writeBoolean(age != null);
>>>>>>>>> if(age != null) {
>>>>>>>>> out.writeInt(age);
>>>>>>>>> }
>>>>>>>>> … and other
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> The other implementation class can be serialized and deserialized
>>>>>>>>> custom rules by Externalizable.read and Externalizable.write.
>>>>>>>>> Each of the different types of servers property needs are
>>>>>>>>> different,it does not require a complete serialization.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> 在 2015年11月9日，下午10:22，Denis Magda < <mailto:dmagda@gridgain.com <ma...@gridgain.com>>dmagda@gridgain.com <ma...@gridgain.com><mailto:dmagda@gridgain.com <ma...@gridgain.com>>> 写道：
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> As I understand both servers have different implementations but the
>>>>>>>>> names of those implementations are the same, correct?
>>>>>>>>>> Because otherwise I don't see how your code could get to the point of
>>>>>>>>> checksum validation if one implementation's name is ServiceEntity while the
>>>>>>>>> other's is DataEntity.
>>>>>>>>>> If my assumptions above are correct then I would recommend to do the
>>>>>>>>> following:
>>>>>>>>>> 1) Extend Serializable instead of Externalizable
>>>>>>>>>>
>>>>>>>>>> interface Entity extends Serializable {
>>>>>>>>>> .....
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> 2) Add custom serialVersionUID to each implementation. This will help
>>>>>>>>> you get rid off checksum related exception
>>>>>>>>>> class EntityImpl implements Entity {
>>>>>>>>>> private static final long serialVersionUID = 0L;
>>>>>>>>>> ......
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>> On 11/8/2015 3:27 PM, 姜 为 wrote:
>>>>>>>>>>> Hi guys:
>>>>>>>>>>>
>>>>>>>>>>> I’m using ignite 1.4.
>>>>>>>>>>> In IgniteCompute.call will transfer of an object to the cluster.
>>>>>>>>>>> The object should implement Serializable or Externalizable
>>>>>>>>> interface.
>>>>>>>>>>> OptimizedClassDescriptor.read method will check whether the object
>>>>>>>>> is in the same class.
>>>>>>>>>>> In my use case,I have some type of servers in cluster.
>>>>>>>>>>> The server type A will check the business,and the server type B
>>>>>>>>> will persistent data.
>>>>>>>>>>> There is a entity interface Entity extends Externalizable have
>>>>>>>>> different implementations on different servers.
>>>>>>>>>>> Such like this:
>>>>>>>>>>>
>>>>>>>>>>> interface Entity extends Externalizable {
>>>>>>>>>>> method a();
>>>>>>>>>>> method b();
>>>>>>>>>>> method c();
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> class ServiceEntity implements Entity {
>>>>>>>>>>> method a(){
>>>>>>>>>>> // do something...
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> method b(){
>>>>>>>>>>> // do something...
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> method c(){
>>>>>>>>>>> throw new UnsupportedException...
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> Externalizable.read...
>>>>>>>>>>> Externalizable.write...
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> class DataEntity implements Entity {
>>>>>>>>>>> method a(){
>>>>>>>>>>> // do something...
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> method b(){
>>>>>>>>>>> throw new UnsupportedException...
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> method c(){
>>>>>>>>>>> // do something...
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> Externalizable.read...
>>>>>>>>>>> Externalizable.write...
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> And IgniteCompute.call(new IgniteCallable(
>>>>>>>>>>> public Object call(){
>>>>>>>>>>> Entity.a() or b and c;..
>>>>>>>>>>> }
>>>>>>>>>>> ));
>>>>>>>>>>>
>>>>>>>>>>> Different implementations of the same class are to achieve read and
>>>>>>>>> write methods.
>>>>>>>>>>> But OptimizedClassDescriptor.read will check the class sum and throw
>>>>>>>>> ClassNotFoundException.
>>>>>>>>>>> I recommend verifyChecksum object set as optional,and I really need
>>>>>>>>> is change.
>>>>>>>>>>> Here is my pr:
>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854> <https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>> <
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854> <https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>>> <
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854> <https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>>> <
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854> <https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>>>
>>>>>>>>>>> <https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>>https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/><https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>> <
>>>>>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/> <https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>>> <
>>>>>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/> <https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>>> <
>>>>>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/> <https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>>>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by Denis Magda <dm...@gridgain.com>.
Hi Jiang,
I'm a bit confused why you can't give different names to the services
located in different modules. If each service has a different name then
my approach will work for you, doesn't it?
But if my service is by ModuleA.ExampleServiceImpl call ModuleB.ExampleServiceImpl,
and ModuleB.ExampleServiceImpl call other complicated service.
Could you provide me with the code showing what you're trying to achieve.
BTW, you use service notion in the thread. Probably you can implement
everything using Ignite Services module:
https://apacheignite.readme.io/docs/service-grid
What do you think?
Regards,
Denis
On 11/11/2015 5:54 AM, 姜 为 wrote:
> Hi Denis:
>
> I tried compute.call(new ExampleServiceImplA());
>
> But if my service is by ModuleA.ExampleServiceImpl call ModuleB.ExampleServiceImpl,
> and ModuleB.ExampleServiceImpl call other complicated service.
>
> The ModuleA only has ExampleServiceImpl there is none ModuleB.other service.
> It does not work.
>
>
>> 在 2015年11月10日，下午9:55，Denis Magda <dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>>
>> It should fail because you send your service impl inside of Callable
>> compute.call(new Callable(new ExampleServiceImplA()));
>>
>> Callable is loaded by application class loader and presents on both the sender and receiver. This leads to the situation when the receiver tries to loads ExampleServiceImplA using the app class loader as well.
>>
>> To avoid this try to implement IgniteCallable by ExampleServiceImplA and send the task this way:
>>
>> compute.call(new ExampleServiceImplA());
>>
>>
>>> I think. If I have lots of type servers. It will be ExampleServiceImpl1 2 3 4 5….,will look very bloated.
>> You can use java 8 closures sending a particular one depending on a server type. Will it work for you?
>> https://apacheignite.readme.io/docs/distributed-closures <https://apacheignite.readme.io/docs/distributed-closures>
>>
>>
>> Regards,
>> Denis
>> On 11/10/2015 4:31 PM, 姜 为 wrote:
>>> Hi Denis,
>>>
>>> I tried your suggested, here is the code : <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497>https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497 <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497>
>>>
>>> But I still got ClassNotFoundException.
>>>
>>> <邮件附件.png>
>>>
>>>
>>> I think. If I have lots of type servers. It will be ExampleServiceImpl1 2 3 4 5….,will look very bloated.
>>>
>>>
>>>> 在 2015年11月10日，下午8:02，Denis Magda < <ma...@gridgain.com>dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>>>>
>>>> Jiang,
>>>>
>>>> This exception happens exactly because you have two implementations with the same name. If the names were different you wouldn't get to the point of checksum validation.
>>>> Here I fully share Val's opinion that checksum's verification shouldn't be optional and it's a responsibility of an application to take care of such situations.
>>>>
>>>> I see the following approaches you can use in your code to get required behavior:
>>>>
>>>> 1) As suggested below use the approach with Serializable + serialVersionUID. It will work;
>>>>
>>>> 2) More preferable is to have different implementations with different names (ExampleServiceImpl1, ExampleServiceImpl2, etc...). Is there any reasons you can't use this straightforward approach?
>>>> When the implementations are ready you can put their classes onto every machine in order to have them in the classpath or you can leverage IgniteConfiguration.peerClassLoadingEnabled feature.
>>>> The feature allows to load missing classes from a machine that sends compute based tasks onto a machine that will execute a task and you don't need to copy implementations' classes manually at all.
>>>> You can read more on this here: https://apacheignite.readme.io/docs/zero-deployment <https://apacheignite.readme.io/docs/zero-deployment>
>>>>
>>>> Regards,
>>>> Denis
>>>>
>>>> On 11/10/2015 10:39 AM, 姜 为 wrote:
>>>>> Hi:
>>>>>
>>>>> Here is example: https://github.com/wmz7year/ignite-test <https://github.com/wmz7year/ignite-test>
>>>>>
>>>>> First start ModuleB, then start ModuleA.
>>>>>
>>>>> ModuleB ExampleServiceImpl has three field.
>>>>>
>>>>> ModuleA ExampleServiceImpl has only one field.
>>>>>
>>>>> Then throw Exception :
>>>>>
>>>>> <邮件 附件.png>
>>>>>> 在 2015年11月10日，下午2:48，Denis Magda <dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> +1 to Val.
>>>>>>
>>>>>> Actually if you have different class names on different machines (DataEntity, ServiceEntity) I don't realize how you can get to the checksum validation stage.
>>>>>> In my understanding you should have caught ClassNotFoundException.
>>>>>>
>>>>>> Provide me with a full runnable example, I'll run and see what you're trying to implement and at which point you fail.
>>>>>>
>>>>>> --
>>>>>> Denis
>>>>>>
>>>>>> On 11/10/2015 9:30 AM, Valentin Kulichenko wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm against optional checksum verification. It's not safe, adds one more
>>>>>>> configuration property and I don't see any use case that can require this.
>>>>>>>
>>>>>>> I also don't completely understand what you're trying to achieve. Can you
>>>>>>> please describe the sequence of serialization/deserialization events that
>>>>>>> you expect in your application?
>>>>>>>
>>>>>>> -Val
>>>>>>>
>>>>>>> On Mon, Nov 9, 2015 at 5:13 PM, 姜 为 < <ma...@gmail.com>ydswcy513@gmail.com <ma...@gmail.com>> wrote:
>>>>>>>
>>>>>>>> Hi:
>>>>>>>>
>>>>>>>> The implementations name maybe not same.
>>>>>>>>
>>>>>>>> Like my example:
>>>>>>>>
>>>>>>>> class DataEntity {
>>>>>>>> Integer id;
>>>>>>>> string name;
>>>>>>>> Integer age;
>>>>>>>>
>>>>>>>> ….. and other
>>>>>>>>
>>>>>>>> Externalizable.read(in){
>>>>>>>> if(in.readBoolean()) {
>>>>>>>> id = in.readInt();
>>>>>>>> }
>>>>>>>> same as name,age...
>>>>>>>> }
>>>>>>>>
>>>>>>>> Externalizable.write(out){
>>>>>>>> out.writeBoolean(id != null);
>>>>>>>> if(id != null) {
>>>>>>>> out.writeInt(id);
>>>>>>>> }
>>>>>>>> same as name,age...
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> class ServiceEntity {
>>>>>>>> Integer id;
>>>>>>>> Integer age;
>>>>>>>>
>>>>>>>> …. and other
>>>>>>>>
>>>>>>>> Externalizable.read(in){
>>>>>>>> if(in.readBoolean()) {
>>>>>>>> id = in.readInt();
>>>>>>>> }
>>>>>>>> if(in.readBoolean()) {
>>>>>>>> in.readString(); // ignore name
>>>>>>>> }
>>>>>>>> if(in.readBoolean()) {
>>>>>>>> age = in.readInt();
>>>>>>>> }
>>>>>>>> … and other
>>>>>>>> }
>>>>>>>>
>>>>>>>> Externalizable.write(out){
>>>>>>>> out.writeBoolean(id != null);
>>>>>>>> if(id != null) {
>>>>>>>> out.writeInt(id);
>>>>>>>> }
>>>>>>>> out.writeBoolean(false); // null for name
>>>>>>>> out.writeBoolean(age != null);
>>>>>>>> if(age != null) {
>>>>>>>> out.writeInt(age);
>>>>>>>> }
>>>>>>>> … and other
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> The other implementation class can be serialized and deserialized
>>>>>>>> custom rules by Externalizable.read and Externalizable.write.
>>>>>>>> Each of the different types of servers property needs are
>>>>>>>> different,it does not require a complete serialization.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> 在 2015年11月9日，下午10:22，Denis Magda < <ma...@gridgain.com>dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> As I understand both servers have different implementations but the
>>>>>>>> names of those implementations are the same, correct?
>>>>>>>>> Because otherwise I don't see how your code could get to the point of
>>>>>>>> checksum validation if one implementation's name is ServiceEntity while the
>>>>>>>> other's is DataEntity.
>>>>>>>>> If my assumptions above are correct then I would recommend to do the
>>>>>>>> following:
>>>>>>>>> 1) Extend Serializable instead of Externalizable
>>>>>>>>>
>>>>>>>>> interface Entity extends Serializable {
>>>>>>>>> .....
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> 2) Add custom serialVersionUID to each implementation. This will help
>>>>>>>> you get rid off checksum related exception
>>>>>>>>> class EntityImpl implements Entity {
>>>>>>>>> private static final long serialVersionUID = 0L;
>>>>>>>>> ......
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Denis
>>>>>>>>>
>>>>>>>>> On 11/8/2015 3:27 PM, 姜 为 wrote:
>>>>>>>>>> Hi guys:
>>>>>>>>>>
>>>>>>>>>> I’m using ignite 1.4.
>>>>>>>>>> In IgniteCompute.call will transfer of an object to the cluster.
>>>>>>>>>> The object should implement Serializable or Externalizable
>>>>>>>> interface.
>>>>>>>>>> OptimizedClassDescriptor.read method will check whether the object
>>>>>>>> is in the same class.
>>>>>>>>>> In my use case,I have some type of servers in cluster.
>>>>>>>>>> The server type A will check the business,and the server type B
>>>>>>>> will persistent data.
>>>>>>>>>> There is a entity interface Entity extends Externalizable have
>>>>>>>> different implementations on different servers.
>>>>>>>>>> Such like this:
>>>>>>>>>>
>>>>>>>>>> interface Entity extends Externalizable {
>>>>>>>>>> method a();
>>>>>>>>>> method b();
>>>>>>>>>> method c();
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> class ServiceEntity implements Entity {
>>>>>>>>>> method a(){
>>>>>>>>>> // do something...
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> method b(){
>>>>>>>>>> // do something...
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> method c(){
>>>>>>>>>> throw new UnsupportedException...
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Externalizable.read...
>>>>>>>>>> Externalizable.write...
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> class DataEntity implements Entity {
>>>>>>>>>> method a(){
>>>>>>>>>> // do something...
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> method b(){
>>>>>>>>>> throw new UnsupportedException...
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> method c(){
>>>>>>>>>> // do something...
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Externalizable.read...
>>>>>>>>>> Externalizable.write...
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> And IgniteCompute.call(new IgniteCallable(
>>>>>>>>>> public Object call(){
>>>>>>>>>> Entity.a() or b and c;..
>>>>>>>>>> }
>>>>>>>>>> ));
>>>>>>>>>>
>>>>>>>>>> Different implementations of the same class are to achieve read and
>>>>>>>> write methods.
>>>>>>>>>> But OptimizedClassDescriptor.read will check the class sum and throw
>>>>>>>> ClassNotFoundException.
>>>>>>>>>> I recommend verifyChecksum object set as optional,and I really need
>>>>>>>> is change.
>>>>>>>>>> Here is my pr:
>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854> <
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>> <
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>> <
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>>
>>>>>>>>>> <https://github.com/apache/ignite/pull/200/>https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/> <
>>>>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>> <
>>>>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>> <
>>>>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>>
>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by 姜 为 <yd...@gmail.com>.
Hi Denis:
I tried compute.call(new ExampleServiceImplA());
But if my service is by ModuleA.ExampleServiceImpl call ModuleB.ExampleServiceImpl,
and ModuleB.ExampleServiceImpl call other complicated service.
The ModuleA only has ExampleServiceImpl there is none ModuleB.other service.
It does not work.
> 在 2015年11月10日，下午9:55，Denis Magda <dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>
> It should fail because you send your service impl inside of Callable
> compute.call(new Callable(new ExampleServiceImplA()));
>
> Callable is loaded by application class loader and presents on both the sender and receiver. This leads to the situation when the receiver tries to loads ExampleServiceImplA using the app class loader as well.
>
> To avoid this try to implement IgniteCallable by ExampleServiceImplA and send the task this way:
>
> compute.call(new ExampleServiceImplA());
>
>
> > I think. If I have lots of type servers. It will be ExampleServiceImpl1 2 3 4 5….,will look very bloated.
>
> You can use java 8 closures sending a particular one depending on a server type. Will it work for you?
> https://apacheignite.readme.io/docs/distributed-closures <https://apacheignite.readme.io/docs/distributed-closures>
>
>
> Regards,
> Denis
> On 11/10/2015 4:31 PM, 姜 为 wrote:
>> Hi Denis,
>>
>> I tried your suggested, here is the code : <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497>https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497 <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497>
>>
>> But I still got ClassNotFoundException.
>>
>> <邮件附件.png>
>>
>>
>> I think. If I have lots of type servers. It will be ExampleServiceImpl1 2 3 4 5….,will look very bloated.
>>
>>
>>> 在 2015年11月10日，下午8:02，Denis Magda < <ma...@gridgain.com>dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>>>
>>> Jiang,
>>>
>>> This exception happens exactly because you have two implementations with the same name. If the names were different you wouldn't get to the point of checksum validation.
>>> Here I fully share Val's opinion that checksum's verification shouldn't be optional and it's a responsibility of an application to take care of such situations.
>>>
>>> I see the following approaches you can use in your code to get required behavior:
>>>
>>> 1) As suggested below use the approach with Serializable + serialVersionUID. It will work;
>>>
>>> 2) More preferable is to have different implementations with different names (ExampleServiceImpl1, ExampleServiceImpl2, etc...). Is there any reasons you can't use this straightforward approach?
>>> When the implementations are ready you can put their classes onto every machine in order to have them in the classpath or you can leverage IgniteConfiguration.peerClassLoadingEnabled feature.
>>> The feature allows to load missing classes from a machine that sends compute based tasks onto a machine that will execute a task and you don't need to copy implementations' classes manually at all.
>>> You can read more on this here: https://apacheignite.readme.io/docs/zero-deployment <https://apacheignite.readme.io/docs/zero-deployment>
>>>
>>> Regards,
>>> Denis
>>>
>>> On 11/10/2015 10:39 AM, 姜 为 wrote:
>>>> Hi:
>>>>
>>>> Here is example: https://github.com/wmz7year/ignite-test <https://github.com/wmz7year/ignite-test>
>>>>
>>>> First start ModuleB, then start ModuleA.
>>>>
>>>> ModuleB ExampleServiceImpl has three field.
>>>>
>>>> ModuleA ExampleServiceImpl has only one field.
>>>>
>>>> Then throw Exception :
>>>>
>>>> <邮件 附件.png>
>>>>> 在 2015年11月10日，下午2:48，Denis Magda <dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>>>>>
>>>>> Hi,
>>>>>
>>>>> +1 to Val.
>>>>>
>>>>> Actually if you have different class names on different machines (DataEntity, ServiceEntity) I don't realize how you can get to the checksum validation stage.
>>>>> In my understanding you should have caught ClassNotFoundException.
>>>>>
>>>>> Provide me with a full runnable example, I'll run and see what you're trying to implement and at which point you fail.
>>>>>
>>>>> --
>>>>> Denis
>>>>>
>>>>> On 11/10/2015 9:30 AM, Valentin Kulichenko wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I'm against optional checksum verification. It's not safe, adds one more
>>>>>> configuration property and I don't see any use case that can require this.
>>>>>>
>>>>>> I also don't completely understand what you're trying to achieve. Can you
>>>>>> please describe the sequence of serialization/deserialization events that
>>>>>> you expect in your application?
>>>>>>
>>>>>> -Val
>>>>>>
>>>>>> On Mon, Nov 9, 2015 at 5:13 PM, 姜 为 < <ma...@gmail.com>ydswcy513@gmail.com <ma...@gmail.com>> wrote:
>>>>>>
>>>>>>> Hi:
>>>>>>>
>>>>>>> The implementations name maybe not same.
>>>>>>>
>>>>>>> Like my example:
>>>>>>>
>>>>>>> class DataEntity {
>>>>>>> Integer id;
>>>>>>> string name;
>>>>>>> Integer age;
>>>>>>>
>>>>>>> ….. and other
>>>>>>>
>>>>>>> Externalizable.read(in){
>>>>>>> if(in.readBoolean()) {
>>>>>>> id = in.readInt();
>>>>>>> }
>>>>>>> same as name,age...
>>>>>>> }
>>>>>>>
>>>>>>> Externalizable.write(out){
>>>>>>> out.writeBoolean(id != null);
>>>>>>> if(id != null) {
>>>>>>> out.writeInt(id);
>>>>>>> }
>>>>>>> same as name,age...
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> class ServiceEntity {
>>>>>>> Integer id;
>>>>>>> Integer age;
>>>>>>>
>>>>>>> …. and other
>>>>>>>
>>>>>>> Externalizable.read(in){
>>>>>>> if(in.readBoolean()) {
>>>>>>> id = in.readInt();
>>>>>>> }
>>>>>>> if(in.readBoolean()) {
>>>>>>> in.readString(); // ignore name
>>>>>>> }
>>>>>>> if(in.readBoolean()) {
>>>>>>> age = in.readInt();
>>>>>>> }
>>>>>>> … and other
>>>>>>> }
>>>>>>>
>>>>>>> Externalizable.write(out){
>>>>>>> out.writeBoolean(id != null);
>>>>>>> if(id != null) {
>>>>>>> out.writeInt(id);
>>>>>>> }
>>>>>>> out.writeBoolean(false); // null for name
>>>>>>> out.writeBoolean(age != null);
>>>>>>> if(age != null) {
>>>>>>> out.writeInt(age);
>>>>>>> }
>>>>>>> … and other
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> The other implementation class can be serialized and deserialized
>>>>>>> custom rules by Externalizable.read and Externalizable.write.
>>>>>>> Each of the different types of servers property needs are
>>>>>>> different,it does not require a complete serialization.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> 在 2015年11月9日，下午10:22，Denis Magda < <ma...@gridgain.com>dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> As I understand both servers have different implementations but the
>>>>>>> names of those implementations are the same, correct?
>>>>>>>> Because otherwise I don't see how your code could get to the point of
>>>>>>> checksum validation if one implementation's name is ServiceEntity while the
>>>>>>> other's is DataEntity.
>>>>>>>> If my assumptions above are correct then I would recommend to do the
>>>>>>> following:
>>>>>>>> 1) Extend Serializable instead of Externalizable
>>>>>>>>
>>>>>>>> interface Entity extends Serializable {
>>>>>>>> .....
>>>>>>>> }
>>>>>>>>
>>>>>>>> 2) Add custom serialVersionUID to each implementation. This will help
>>>>>>> you get rid off checksum related exception
>>>>>>>> class EntityImpl implements Entity {
>>>>>>>> private static final long serialVersionUID = 0L;
>>>>>>>> ......
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On 11/8/2015 3:27 PM, 姜 为 wrote:
>>>>>>>>> Hi guys:
>>>>>>>>>
>>>>>>>>> I’m using ignite 1.4.
>>>>>>>>> In IgniteCompute.call will transfer of an object to the cluster.
>>>>>>>>> The object should implement Serializable or Externalizable
>>>>>>> interface.
>>>>>>>>> OptimizedClassDescriptor.read method will check whether the object
>>>>>>> is in the same class.
>>>>>>>>> In my use case,I have some type of servers in cluster.
>>>>>>>>> The server type A will check the business,and the server type B
>>>>>>> will persistent data.
>>>>>>>>> There is a entity interface Entity extends Externalizable have
>>>>>>> different implementations on different servers.
>>>>>>>>> Such like this:
>>>>>>>>>
>>>>>>>>> interface Entity extends Externalizable {
>>>>>>>>> method a();
>>>>>>>>> method b();
>>>>>>>>> method c();
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> class ServiceEntity implements Entity {
>>>>>>>>> method a(){
>>>>>>>>> // do something...
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> method b(){
>>>>>>>>> // do something...
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> method c(){
>>>>>>>>> throw new UnsupportedException...
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Externalizable.read...
>>>>>>>>> Externalizable.write...
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> class DataEntity implements Entity {
>>>>>>>>> method a(){
>>>>>>>>> // do something...
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> method b(){
>>>>>>>>> throw new UnsupportedException...
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> method c(){
>>>>>>>>> // do something...
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Externalizable.read...
>>>>>>>>> Externalizable.write...
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> And IgniteCompute.call(new IgniteCallable(
>>>>>>>>> public Object call(){
>>>>>>>>> Entity.a() or b and c;..
>>>>>>>>> }
>>>>>>>>> ));
>>>>>>>>>
>>>>>>>>> Different implementations of the same class are to achieve read and
>>>>>>> write methods.
>>>>>>>>> But OptimizedClassDescriptor.read will check the class sum and throw
>>>>>>> ClassNotFoundException.
>>>>>>>>> I recommend verifyChecksum object set as optional,and I really need
>>>>>>> is change.
>>>>>>>>> Here is my pr:
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854> <
>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>> <
>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>> <
>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>>
>>>>>>>>> <https://github.com/apache/ignite/pull/200/>https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/> <
>>>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>> <
>>>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>> <
>>>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by Denis Magda <dm...@gridgain.com>.
It should fail because you send your service impl inside of Callable
compute.call(new Callable(new ExampleServiceImplA()));
Callable is loaded by application class loader and presents on both the
sender and receiver. This leads to the situation when the receiver tries
to loads ExampleServiceImplA using the app class loader as well.
To avoid this try to implement IgniteCallable by ExampleServiceImplA and
send the task this way:
compute.call(new ExampleServiceImplA());
> I think. If I have lots of type servers. It will be
ExampleServiceImpl1 2 3 4 5….,will look very bloated.
You can use java 8 closures sending a particular one depending on a
server type. Will it work for you?
https://apacheignite.readme.io/docs/distributed-closures
Regards,
Denis
On 11/10/2015 4:31 PM, 姜 为 wrote:
> Hi Denis,
>
> I tried your suggested, here is the code
> :https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497
> But I still got ClassNotFoundException.
>
>
>
> I think. If I have lots of type servers. It will be
> ExampleServiceImpl1 2 3 4 5….,will look very bloated.
>
>> 在 2015年11月10日，下午8:02，Denis Magda <dmagda@gridgain.com
>> <ma...@gridgain.com>> 写道：
>>
>> Jiang,
>>
>> This exception happens exactly because you have two implementations
>> with the same name. If the names were different you wouldn't get to
>> the point of checksum validation.
>> Here I fully share Val's opinion that checksum's verification
>> shouldn't be optional and it's a responsibility of an application to
>> take care of such situations.
>>
>> I see the following approaches you can use in your code to get
>> required behavior:
>>
>> 1) As suggested below use the approach with Serializable +
>> serialVersionUID. It will work;
>>
>> 2) More preferable is to have different implementations with
>> different names (ExampleServiceImpl1, ExampleServiceImpl2, etc...).
>> Is there any reasons you can't use this straightforward approach?
>> When the implementations are ready you can put their classes onto
>> every machine in order to have them in the classpath or you can
>> leverage IgniteConfiguration.peerClassLoadingEnabled feature.
>> The feature allows to load missing classes from a machine that sends
>> compute based tasks onto a machine that will execute a task and you
>> don't need to copy implementations' classes manually at all.
>> You can read more on this here:
>> https://apacheignite.readme.io/docs/zero-deployment
>>
>> Regards,
>> Denis
>>
>> On 11/10/2015 10:39 AM, 姜 为 wrote:
>>> Hi:
>>>
>>> Here is example: https://github.com/wmz7year/ignite-test
>>>
>>> First start ModuleB, then start ModuleA.
>>>
>>> ModuleB ExampleServiceImpl has three field.
>>>
>>> ModuleA ExampleServiceImpl has only one field.
>>>
>>> Then throw Exception :
>>>
>>> <邮件 附件.png>
>>>> 在 2015年11月10日，下午2:48，Denis Magda <dm...@gridgain.com> 写道：
>>>>
>>>> Hi,
>>>>
>>>> +1 to Val.
>>>>
>>>> Actually if you have different class names on different machines
>>>> (DataEntity, ServiceEntity) I don't realize how you can get to the
>>>> checksum validation stage.
>>>> In my understanding you should have caught ClassNotFoundException.
>>>>
>>>> Provide me with a full runnable example, I'll run and see what
>>>> you're trying to implement and at which point you fail.
>>>>
>>>> --
>>>> Denis
>>>>
>>>> On 11/10/2015 9:30 AM, Valentin Kulichenko wrote:
>>>>> Hi,
>>>>>
>>>>> I'm against optional checksum verification. It's not safe, adds
>>>>> one more
>>>>> configuration property and I don't see any use case that can
>>>>> require this.
>>>>>
>>>>> I also don't completely understand what you're trying to achieve.
>>>>> Can you
>>>>> please describe the sequence of serialization/deserialization
>>>>> events that
>>>>> you expect in your application?
>>>>>
>>>>> -Val
>>>>>
>>>>> On Mon, Nov 9, 2015 at 5:13 PM, 姜 为 <yd...@gmail.com> wrote:
>>>>>
>>>>>> Hi:
>>>>>>
>>>>>> The implementations name maybe not same.
>>>>>>
>>>>>> Like my example:
>>>>>>
>>>>>> class DataEntity {
>>>>>> Integer id;
>>>>>> string name;
>>>>>> Integer age;
>>>>>>
>>>>>> ….. and other
>>>>>>
>>>>>> Externalizable.read(in){
>>>>>> if(in.readBoolean()) {
>>>>>> id = in.readInt();
>>>>>> }
>>>>>> same as name,age...
>>>>>> }
>>>>>>
>>>>>> Externalizable.write(out){
>>>>>> out.writeBoolean(id != null);
>>>>>> if(id != null) {
>>>>>> out.writeInt(id);
>>>>>> }
>>>>>> same as name,age...
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> class ServiceEntity {
>>>>>> Integer id;
>>>>>> Integer age;
>>>>>>
>>>>>> …. and other
>>>>>>
>>>>>> Externalizable.read(in){
>>>>>> if(in.readBoolean()) {
>>>>>> id = in.readInt();
>>>>>> }
>>>>>> if(in.readBoolean()) {
>>>>>> in.readString(); // ignore name
>>>>>> }
>>>>>> if(in.readBoolean()) {
>>>>>> age = in.readInt();
>>>>>> }
>>>>>> … and other
>>>>>> }
>>>>>>
>>>>>> Externalizable.write(out){
>>>>>> out.writeBoolean(id != null);
>>>>>> if(id != null) {
>>>>>> out.writeInt(id);
>>>>>> }
>>>>>> out.writeBoolean(false); // null for name
>>>>>> out.writeBoolean(age != null);
>>>>>> if(age != null) {
>>>>>> out.writeInt(age);
>>>>>> }
>>>>>> … and other
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> The other implementation class can be serialized and
>>>>>> deserialized
>>>>>> custom rules by Externalizable.read and Externalizable.write.
>>>>>> Each of the different types of servers property needs are
>>>>>> different,it does not require a complete serialization.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> 在 2015年11月9日，下午10:22，Denis Magda <dm...@gridgain.com>
>>>>>>> 写道：
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> As I understand both servers have different implementations but the
>>>>>> names of those implementations are the same, correct?
>>>>>>> Because otherwise I don't see how your code could get to the
>>>>>>> point of
>>>>>> checksum validation if one implementation's name is ServiceEntity
>>>>>> while the
>>>>>> other's is DataEntity.
>>>>>>> If my assumptions above are correct then I would recommend to do the
>>>>>> following:
>>>>>>> 1) Extend Serializable instead of Externalizable
>>>>>>>
>>>>>>> interface Entity extends Serializable {
>>>>>>> .....
>>>>>>> }
>>>>>>>
>>>>>>> 2) Add custom serialVersionUID to each implementation. This will
>>>>>>> help
>>>>>> you get rid off checksum related exception
>>>>>>> class EntityImpl implements Entity {
>>>>>>> private static final long serialVersionUID = 0L;
>>>>>>> ......
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Denis
>>>>>>>
>>>>>>> On 11/8/2015 3:27 PM, 姜 为 wrote:
>>>>>>>> Hi guys:
>>>>>>>>
>>>>>>>> I’m using ignite 1.4.
>>>>>>>> In IgniteCompute.call will transfer of an object to the
>>>>>>>> cluster.
>>>>>>>> The object should implement Serializable or Externalizable
>>>>>> interface.
>>>>>>>> OptimizedClassDescriptor.read method will check whether the
>>>>>>>> object
>>>>>> is in the same class.
>>>>>>>> In my use case,I have some type of servers in cluster.
>>>>>>>> The server type A will check the business,and the server type B
>>>>>> will persistent data.
>>>>>>>> There is a entity interface Entity extends Externalizable have
>>>>>> different implementations on different servers.
>>>>>>>> Such like this:
>>>>>>>>
>>>>>>>> interface Entity extends Externalizable {
>>>>>>>> method a();
>>>>>>>> method b();
>>>>>>>> method c();
>>>>>>>> }
>>>>>>>>
>>>>>>>> class ServiceEntity implements Entity {
>>>>>>>> method a(){
>>>>>>>> // do something...
>>>>>>>> }
>>>>>>>>
>>>>>>>> method b(){
>>>>>>>> // do something...
>>>>>>>> }
>>>>>>>>
>>>>>>>> method c(){
>>>>>>>> throw new UnsupportedException...
>>>>>>>> }
>>>>>>>>
>>>>>>>> Externalizable.read...
>>>>>>>> Externalizable.write...
>>>>>>>> }
>>>>>>>>
>>>>>>>> class DataEntity implements Entity {
>>>>>>>> method a(){
>>>>>>>> // do something...
>>>>>>>> }
>>>>>>>>
>>>>>>>> method b(){
>>>>>>>> throw new UnsupportedException...
>>>>>>>> }
>>>>>>>>
>>>>>>>> method c(){
>>>>>>>> // do something...
>>>>>>>> }
>>>>>>>>
>>>>>>>> Externalizable.read...
>>>>>>>> Externalizable.write...
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> And IgniteCompute.call(new IgniteCallable(
>>>>>>>> public Object call(){
>>>>>>>> Entity.a() or b and c;..
>>>>>>>> }
>>>>>>>> ));
>>>>>>>>
>>>>>>>> Different implementations of the same class are to achieve
>>>>>>>> read and
>>>>>> write methods.
>>>>>>>> But OptimizedClassDescriptor.read will check the class sum
>>>>>>>> and throw
>>>>>> ClassNotFoundException.
>>>>>>>> I recommend verifyChecksum object set as optional,and I
>>>>>>>> really need
>>>>>> is change.
>>>>>>>> Here is my pr:
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <
>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854> <
>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854> <
>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854>
>>>>>>>> https://github.com/apache/ignite/pull/200/ <
>>>>>> https://github.com/apache/ignite/pull/200/> <
>>>>>> https://github.com/apache/ignite/pull/200/> <
>>>>>> https://github.com/apache/ignite/pull/200/>
>>>>>>>>
>>>>>>
>>>>
>>>
>>
>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by 姜 为 <yd...@gmail.com>.
Hi Denis,
I tried your suggested, here is the code :https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497 <https://github.com/wmz7year/ignite-test/commit/4eaa4231e165eaa4de00dab94cd394f629e68497>
But I still got ClassNotFoundException.
I think. If I have lots of type servers. It will be ExampleServiceImpl1 2 3 4 5….,will look very bloated.
> 在 2015年11月10日，下午8:02，Denis Magda <dm...@gridgain.com> 写道：
>
> Jiang,
>
> This exception happens exactly because you have two implementations with the same name. If the names were different you wouldn't get to the point of checksum validation.
> Here I fully share Val's opinion that checksum's verification shouldn't be optional and it's a responsibility of an application to take care of such situations.
>
> I see the following approaches you can use in your code to get required behavior:
>
> 1) As suggested below use the approach with Serializable + serialVersionUID. It will work;
>
> 2) More preferable is to have different implementations with different names (ExampleServiceImpl1, ExampleServiceImpl2, etc...). Is there any reasons you can't use this straightforward approach?
> When the implementations are ready you can put their classes onto every machine in order to have them in the classpath or you can leverage IgniteConfiguration.peerClassLoadingEnabled feature.
> The feature allows to load missing classes from a machine that sends compute based tasks onto a machine that will execute a task and you don't need to copy implementations' classes manually at all.
> You can read more on this here: https://apacheignite.readme.io/docs/zero-deployment <https://apacheignite.readme.io/docs/zero-deployment>
>
> Regards,
> Denis
>
> On 11/10/2015 10:39 AM, 姜 为 wrote:
>> Hi:
>>
>> Here is example: https://github.com/wmz7year/ignite-test <https://github.com/wmz7year/ignite-test>
>>
>> First start ModuleB, then start ModuleA.
>>
>> ModuleB ExampleServiceImpl has three field.
>>
>> ModuleA ExampleServiceImpl has only one field.
>>
>> Then throw Exception :
>>
>> <邮件附件.png>
>>> 在 2015年11月10日，下午2:48，Denis Magda < <ma...@gridgain.com>dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>>>
>>> Hi,
>>>
>>> +1 to Val.
>>>
>>> Actually if you have different class names on different machines (DataEntity, ServiceEntity) I don't realize how you can get to the checksum validation stage.
>>> In my understanding you should have caught ClassNotFoundException.
>>>
>>> Provide me with a full runnable example, I'll run and see what you're trying to implement and at which point you fail.
>>>
>>> --
>>> Denis
>>>
>>> On 11/10/2015 9:30 AM, Valentin Kulichenko wrote:
>>>> Hi,
>>>>
>>>> I'm against optional checksum verification. It's not safe, adds one more
>>>> configuration property and I don't see any use case that can require this.
>>>>
>>>> I also don't completely understand what you're trying to achieve. Can you
>>>> please describe the sequence of serialization/deserialization events that
>>>> you expect in your application?
>>>>
>>>> -Val
>>>>
>>>> On Mon, Nov 9, 2015 at 5:13 PM, 姜 为 < <ma...@gmail.com>ydswcy513@gmail.com <ma...@gmail.com>> wrote:
>>>>
>>>>> Hi:
>>>>>
>>>>> The implementations name maybe not same.
>>>>>
>>>>> Like my example:
>>>>>
>>>>> class DataEntity {
>>>>> Integer id;
>>>>> string name;
>>>>> Integer age;
>>>>>
>>>>> ….. and other
>>>>>
>>>>> Externalizable.read(in){
>>>>> if(in.readBoolean()) {
>>>>> id = in.readInt();
>>>>> }
>>>>> same as name,age...
>>>>> }
>>>>>
>>>>> Externalizable.write(out){
>>>>> out.writeBoolean(id != null);
>>>>> if(id != null) {
>>>>> out.writeInt(id);
>>>>> }
>>>>> same as name,age...
>>>>> }
>>>>> }
>>>>>
>>>>> class ServiceEntity {
>>>>> Integer id;
>>>>> Integer age;
>>>>>
>>>>> …. and other
>>>>>
>>>>> Externalizable.read(in){
>>>>> if(in.readBoolean()) {
>>>>> id = in.readInt();
>>>>> }
>>>>> if(in.readBoolean()) {
>>>>> in.readString(); // ignore name
>>>>> }
>>>>> if(in.readBoolean()) {
>>>>> age = in.readInt();
>>>>> }
>>>>> … and other
>>>>> }
>>>>>
>>>>> Externalizable.write(out){
>>>>> out.writeBoolean(id != null);
>>>>> if(id != null) {
>>>>> out.writeInt(id);
>>>>> }
>>>>> out.writeBoolean(false); // null for name
>>>>> out.writeBoolean(age != null);
>>>>> if(age != null) {
>>>>> out.writeInt(age);
>>>>> }
>>>>> … and other
>>>>> }
>>>>> }
>>>>>
>>>>> The other implementation class can be serialized and deserialized
>>>>> custom rules by Externalizable.read and Externalizable.write.
>>>>> Each of the different types of servers property needs are
>>>>> different,it does not require a complete serialization.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> 在 2015年11月9日，下午10:22，Denis Magda < <ma...@gridgain.com>dmagda@gridgain.com <ma...@gridgain.com>> 写道：
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> As I understand both servers have different implementations but the
>>>>> names of those implementations are the same, correct?
>>>>>> Because otherwise I don't see how your code could get to the point of
>>>>> checksum validation if one implementation's name is ServiceEntity while the
>>>>> other's is DataEntity.
>>>>>> If my assumptions above are correct then I would recommend to do the
>>>>> following:
>>>>>> 1) Extend Serializable instead of Externalizable
>>>>>>
>>>>>> interface Entity extends Serializable {
>>>>>> .....
>>>>>> }
>>>>>>
>>>>>> 2) Add custom serialVersionUID to each implementation. This will help
>>>>> you get rid off checksum related exception
>>>>>> class EntityImpl implements Entity {
>>>>>> private static final long serialVersionUID = 0L;
>>>>>> ......
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Denis
>>>>>>
>>>>>> On 11/8/2015 3:27 PM, 姜 为 wrote:
>>>>>>> Hi guys:
>>>>>>>
>>>>>>> I’m using ignite 1.4.
>>>>>>> In IgniteCompute.call will transfer of an object to the cluster.
>>>>>>> The object should implement Serializable or Externalizable
>>>>> interface.
>>>>>>> OptimizedClassDescriptor.read method will check whether the object
>>>>> is in the same class.
>>>>>>> In my use case,I have some type of servers in cluster.
>>>>>>> The server type A will check the business,and the server type B
>>>>> will persistent data.
>>>>>>> There is a entity interface Entity extends Externalizable have
>>>>> different implementations on different servers.
>>>>>>> Such like this:
>>>>>>>
>>>>>>> interface Entity extends Externalizable {
>>>>>>> method a();
>>>>>>> method b();
>>>>>>> method c();
>>>>>>> }
>>>>>>>
>>>>>>> class ServiceEntity implements Entity {
>>>>>>> method a(){
>>>>>>> // do something...
>>>>>>> }
>>>>>>>
>>>>>>> method b(){
>>>>>>> // do something...
>>>>>>> }
>>>>>>>
>>>>>>> method c(){
>>>>>>> throw new UnsupportedException...
>>>>>>> }
>>>>>>>
>>>>>>> Externalizable.read...
>>>>>>> Externalizable.write...
>>>>>>> }
>>>>>>>
>>>>>>> class DataEntity implements Entity {
>>>>>>> method a(){
>>>>>>> // do something...
>>>>>>> }
>>>>>>>
>>>>>>> method b(){
>>>>>>> throw new UnsupportedException...
>>>>>>> }
>>>>>>>
>>>>>>> method c(){
>>>>>>> // do something...
>>>>>>> }
>>>>>>>
>>>>>>> Externalizable.read...
>>>>>>> Externalizable.write...
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> And IgniteCompute.call(new IgniteCallable(
>>>>>>> public Object call(){
>>>>>>> Entity.a() or b and c;..
>>>>>>> }
>>>>>>> ));
>>>>>>>
>>>>>>> Different implementations of the same class are to achieve read and
>>>>> write methods.
>>>>>>> But OptimizedClassDescriptor.read will check the class sum and throw
>>>>> ClassNotFoundException.
>>>>>>> I recommend verifyChecksum object set as optional,and I really need
>>>>> is change.
>>>>>>> Here is my pr:
>>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854> <
>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>> <
>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>> <
>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>>
>>>>>>> <https://github.com/apache/ignite/pull/200/>https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/> <
>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>> <
>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>> <
>>>>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>>
>>>>>>>
>>>>>
>>>
>>
>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by Denis Magda <dm...@gridgain.com>.
Jiang,
This exception happens exactly because you have two implementations with
the same name. If the names were different you wouldn't get to the point
of checksum validation.
Here I fully share Val's opinion that checksum's verification shouldn't
be optional and it's a responsibility of an application to take care of
such situations.
I see the following approaches you can use in your code to get required
behavior:
1) As suggested below use the approach with Serializable +
serialVersionUID. It will work;
2) More preferable is to have different implementations with different
names (ExampleServiceImpl1, ExampleServiceImpl2, etc...). Is there any
reasons you can't use this straightforward approach?
When the implementations are ready you can put their classes onto every
machine in order to have them in the classpath or you can leverage
IgniteConfiguration.peerClassLoadingEnabled feature.
The feature allows to load missing classes from a machine that sends
compute based tasks onto a machine that will execute a task and you
don't need to copy implementations' classes manually at all.
You can read more on this here:
https://apacheignite.readme.io/docs/zero-deployment
Regards,
Denis
On 11/10/2015 10:39 AM, 姜 为 wrote:
> Hi:
>
> Here is example: https://github.com/wmz7year/ignite-test
>
> First start ModuleB, then start ModuleA.
>
> ModuleB ExampleServiceImpl has three field.
>
> ModuleA ExampleServiceImpl has only one field.
>
> Then throw Exception :
>
>> 在 2015年11月10日，下午2:48，Denis Magda <dmagda@gridgain.com
>> <ma...@gridgain.com>> 写道：
>>
>> Hi,
>>
>> +1 to Val.
>>
>> Actually if you have different class names on different machines
>> (DataEntity, ServiceEntity) I don't realize how you can get to the
>> checksum validation stage.
>> In my understanding you should have caught ClassNotFoundException.
>>
>> Provide me with a full runnable example, I'll run and see what you're
>> trying to implement and at which point you fail.
>>
>> --
>> Denis
>>
>> On 11/10/2015 9:30 AM, Valentin Kulichenko wrote:
>>> Hi,
>>>
>>> I'm against optional checksum verification. It's not safe, adds one more
>>> configuration property and I don't see any use case that can require
>>> this.
>>>
>>> I also don't completely understand what you're trying to achieve.
>>> Can you
>>> please describe the sequence of serialization/deserialization events
>>> that
>>> you expect in your application?
>>>
>>> -Val
>>>
>>> On Mon, Nov 9, 2015 at 5:13 PM, 姜 为 <ydswcy513@gmail.com
>>> <ma...@gmail.com>> wrote:
>>>
>>>> Hi:
>>>>
>>>> The implementations name maybe not same.
>>>>
>>>> Like my example:
>>>>
>>>> class DataEntity {
>>>> Integer id;
>>>> string name;
>>>> Integer age;
>>>>
>>>> ….. and other
>>>>
>>>> Externalizable.read(in){
>>>> if(in.readBoolean()) {
>>>> id = in.readInt();
>>>> }
>>>> same as name,age...
>>>> }
>>>>
>>>> Externalizable.write(out){
>>>> out.writeBoolean(id != null);
>>>> if(id != null) {
>>>> out.writeInt(id);
>>>> }
>>>> same as name,age...
>>>> }
>>>> }
>>>>
>>>> class ServiceEntity {
>>>> Integer id;
>>>> Integer age;
>>>>
>>>> …. and other
>>>>
>>>> Externalizable.read(in){
>>>> if(in.readBoolean()) {
>>>> id = in.readInt();
>>>> }
>>>> if(in.readBoolean()) {
>>>> in.readString(); // ignore name
>>>> }
>>>> if(in.readBoolean()) {
>>>> age = in.readInt();
>>>> }
>>>> … and other
>>>> }
>>>>
>>>> Externalizable.write(out){
>>>> out.writeBoolean(id != null);
>>>> if(id != null) {
>>>> out.writeInt(id);
>>>> }
>>>> out.writeBoolean(false); // null for name
>>>> out.writeBoolean(age != null);
>>>> if(age != null) {
>>>> out.writeInt(age);
>>>> }
>>>> … and other
>>>> }
>>>> }
>>>>
>>>> The other implementation class can be serialized and
>>>> deserialized
>>>> custom rules by Externalizable.read and Externalizable.write.
>>>> Each of the different types of servers property needs are
>>>> different,it does not require a complete serialization.
>>>>
>>>>
>>>>
>>>>
>>>>> 在 2015年11月9日，下午10:22，Denis Magda <dmagda@gridgain.com
>>>>> <ma...@gridgain.com>> 写道：
>>>>>
>>>>> Hi,
>>>>>
>>>>> As I understand both servers have different implementations but the
>>>> names of those implementations are the same, correct?
>>>>> Because otherwise I don't see how your code could get to the point of
>>>> checksum validation if one implementation's name is ServiceEntity
>>>> while the
>>>> other's is DataEntity.
>>>>> If my assumptions above are correct then I would recommend to do the
>>>> following:
>>>>> 1) Extend Serializable instead of Externalizable
>>>>>
>>>>> interface Entity extends Serializable {
>>>>> .....
>>>>> }
>>>>>
>>>>> 2) Add custom serialVersionUID to each implementation. This will help
>>>> you get rid off checksum related exception
>>>>> class EntityImpl implements Entity {
>>>>> private static final long serialVersionUID = 0L;
>>>>> ......
>>>>> }
>>>>>
>>>>>
>>>>> Regards,
>>>>> Denis
>>>>>
>>>>> On 11/8/2015 3:27 PM, 姜 为 wrote:
>>>>>> Hi guys:
>>>>>>
>>>>>> I’m using ignite 1.4.
>>>>>> In IgniteCompute.call will transfer of an object to the cluster.
>>>>>> The object should implement Serializable or Externalizable
>>>> interface.
>>>>>> OptimizedClassDescriptor.read method will check whether the
>>>>>> object
>>>> is in the same class.
>>>>>> In my use case,I have some type of servers in cluster.
>>>>>> The server type A will check the business,and the server type B
>>>> will persistent data.
>>>>>> There is a entity interface Entity extends Externalizable have
>>>> different implementations on different servers.
>>>>>> Such like this:
>>>>>>
>>>>>> interface Entity extends Externalizable {
>>>>>> method a();
>>>>>> method b();
>>>>>> method c();
>>>>>> }
>>>>>>
>>>>>> class ServiceEntity implements Entity {
>>>>>> method a(){
>>>>>> // do something...
>>>>>> }
>>>>>>
>>>>>> method b(){
>>>>>> // do something...
>>>>>> }
>>>>>>
>>>>>> method c(){
>>>>>> throw new UnsupportedException...
>>>>>> }
>>>>>>
>>>>>> Externalizable.read...
>>>>>> Externalizable.write...
>>>>>> }
>>>>>>
>>>>>> class DataEntity implements Entity {
>>>>>> method a(){
>>>>>> // do something...
>>>>>> }
>>>>>>
>>>>>> method b(){
>>>>>> throw new UnsupportedException...
>>>>>> }
>>>>>>
>>>>>> method c(){
>>>>>> // do something...
>>>>>> }
>>>>>>
>>>>>> Externalizable.read...
>>>>>> Externalizable.write...
>>>>>> }
>>>>>>
>>>>>>
>>>>>> And IgniteCompute.call(new IgniteCallable(
>>>>>> public Object call(){
>>>>>> Entity.a() or b and c;..
>>>>>> }
>>>>>> ));
>>>>>>
>>>>>> Different implementations of the same class are to achieve
>>>>>> read and
>>>> write methods.
>>>>>> But OptimizedClassDescriptor.read will check the class sum and
>>>>>> throw
>>>> ClassNotFoundException.
>>>>>> I recommend verifyChecksum object set as optional,and I really
>>>>>> need
>>>> is change.
>>>>>> Here is my pr:
>>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <
>>>> https://issues.apache.org/jira/browse/IGNITE-1854> <
>>>> https://issues.apache.org/jira/browse/IGNITE-1854> <
>>>> https://issues.apache.org/jira/browse/IGNITE-1854>
>>>>>> https://github.com/apache/ignite/pull/200/ <
>>>> https://github.com/apache/ignite/pull/200/> <
>>>> https://github.com/apache/ignite/pull/200/> <
>>>> https://github.com/apache/ignite/pull/200/>
>>>>>>
>>>>
>>
>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by 姜 为 <yd...@gmail.com>.
Hi:
Here is example: https://github.com/wmz7year/ignite-test <https://github.com/wmz7year/ignite-test>
First start ModuleB, then start ModuleA.
ModuleB ExampleServiceImpl has three field.
ModuleA ExampleServiceImpl has only one field.
Then throw Exception :
> 在 2015年11月10日，下午2:48，Denis Magda <dm...@gridgain.com> 写道：
>
> Hi,
>
> +1 to Val.
>
> Actually if you have different class names on different machines (DataEntity, ServiceEntity) I don't realize how you can get to the checksum validation stage.
> In my understanding you should have caught ClassNotFoundException.
>
> Provide me with a full runnable example, I'll run and see what you're trying to implement and at which point you fail.
>
> --
> Denis
>
> On 11/10/2015 9:30 AM, Valentin Kulichenko wrote:
>> Hi,
>>
>> I'm against optional checksum verification. It's not safe, adds one more
>> configuration property and I don't see any use case that can require this.
>>
>> I also don't completely understand what you're trying to achieve. Can you
>> please describe the sequence of serialization/deserialization events that
>> you expect in your application?
>>
>> -Val
>>
>> On Mon, Nov 9, 2015 at 5:13 PM, 姜 为 <yd...@gmail.com> wrote:
>>
>>> Hi:
>>>
>>> The implementations name maybe not same.
>>>
>>> Like my example:
>>>
>>> class DataEntity {
>>> Integer id;
>>> string name;
>>> Integer age;
>>>
>>> ….. and other
>>>
>>> Externalizable.read(in){
>>> if(in.readBoolean()) {
>>> id = in.readInt();
>>> }
>>> same as name,age...
>>> }
>>>
>>> Externalizable.write(out){
>>> out.writeBoolean(id != null);
>>> if(id != null) {
>>> out.writeInt(id);
>>> }
>>> same as name,age...
>>> }
>>> }
>>>
>>> class ServiceEntity {
>>> Integer id;
>>> Integer age;
>>>
>>> …. and other
>>>
>>> Externalizable.read(in){
>>> if(in.readBoolean()) {
>>> id = in.readInt();
>>> }
>>> if(in.readBoolean()) {
>>> in.readString(); // ignore name
>>> }
>>> if(in.readBoolean()) {
>>> age = in.readInt();
>>> }
>>> … and other
>>> }
>>>
>>> Externalizable.write(out){
>>> out.writeBoolean(id != null);
>>> if(id != null) {
>>> out.writeInt(id);
>>> }
>>> out.writeBoolean(false); // null for name
>>> out.writeBoolean(age != null);
>>> if(age != null) {
>>> out.writeInt(age);
>>> }
>>> … and other
>>> }
>>> }
>>>
>>> The other implementation class can be serialized and deserialized
>>> custom rules by Externalizable.read and Externalizable.write.
>>> Each of the different types of servers property needs are
>>> different,it does not require a complete serialization.
>>>
>>>
>>>
>>>
>>>> 在 2015年11月9日，下午10:22，Denis Magda <dm...@gridgain.com> 写道：
>>>>
>>>> Hi,
>>>>
>>>> As I understand both servers have different implementations but the
>>> names of those implementations are the same, correct?
>>>> Because otherwise I don't see how your code could get to the point of
>>> checksum validation if one implementation's name is ServiceEntity while the
>>> other's is DataEntity.
>>>> If my assumptions above are correct then I would recommend to do the
>>> following:
>>>> 1) Extend Serializable instead of Externalizable
>>>>
>>>> interface Entity extends Serializable {
>>>> .....
>>>> }
>>>>
>>>> 2) Add custom serialVersionUID to each implementation. This will help
>>> you get rid off checksum related exception
>>>> class EntityImpl implements Entity {
>>>> private static final long serialVersionUID = 0L;
>>>> ......
>>>> }
>>>>
>>>>
>>>> Regards,
>>>> Denis
>>>>
>>>> On 11/8/2015 3:27 PM, 姜 为 wrote:
>>>>> Hi guys:
>>>>>
>>>>> I’m using ignite 1.4.
>>>>> In IgniteCompute.call will transfer of an object to the cluster.
>>>>> The object should implement Serializable or Externalizable
>>> interface.
>>>>> OptimizedClassDescriptor.read method will check whether the object
>>> is in the same class.
>>>>> In my use case,I have some type of servers in cluster.
>>>>> The server type A will check the business,and the server type B
>>> will persistent data.
>>>>> There is a entity interface Entity extends Externalizable have
>>> different implementations on different servers.
>>>>> Such like this:
>>>>>
>>>>> interface Entity extends Externalizable {
>>>>> method a();
>>>>> method b();
>>>>> method c();
>>>>> }
>>>>>
>>>>> class ServiceEntity implements Entity {
>>>>> method a(){
>>>>> // do something...
>>>>> }
>>>>>
>>>>> method b(){
>>>>> // do something...
>>>>> }
>>>>>
>>>>> method c(){
>>>>> throw new UnsupportedException...
>>>>> }
>>>>>
>>>>> Externalizable.read...
>>>>> Externalizable.write...
>>>>> }
>>>>>
>>>>> class DataEntity implements Entity {
>>>>> method a(){
>>>>> // do something...
>>>>> }
>>>>>
>>>>> method b(){
>>>>> throw new UnsupportedException...
>>>>> }
>>>>>
>>>>> method c(){
>>>>> // do something...
>>>>> }
>>>>>
>>>>> Externalizable.read...
>>>>> Externalizable.write...
>>>>> }
>>>>>
>>>>>
>>>>> And IgniteCompute.call(new IgniteCallable(
>>>>> public Object call(){
>>>>> Entity.a() or b and c;..
>>>>> }
>>>>> ));
>>>>>
>>>>> Different implementations of the same class are to achieve read and
>>> write methods.
>>>>> But OptimizedClassDescriptor.read will check the class sum and throw
>>> ClassNotFoundException.
>>>>> I recommend verifyChecksum object set as optional,and I really need
>>> is change.
>>>>> Here is my pr:
>>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <
>>> https://issues.apache.org/jira/browse/IGNITE-1854> <
>>> https://issues.apache.org/jira/browse/IGNITE-1854> <
>>> https://issues.apache.org/jira/browse/IGNITE-1854>
>>>>> https://github.com/apache/ignite/pull/200/ <
>>> https://github.com/apache/ignite/pull/200/> <
>>> https://github.com/apache/ignite/pull/200/> <
>>> https://github.com/apache/ignite/pull/200/>
>>>>>
>>>
>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by Denis Magda <dm...@gridgain.com>.
Hi,
+1 to Val.
Actually if you have different class names on different machines
(DataEntity, ServiceEntity) I don't realize how you can get to the
checksum validation stage.
In my understanding you should have caught ClassNotFoundException.
Provide me with a full runnable example, I'll run and see what you're
trying to implement and at which point you fail.
--
Denis
On 11/10/2015 9:30 AM, Valentin Kulichenko wrote:
> Hi,
>
> I'm against optional checksum verification. It's not safe, adds one more
> configuration property and I don't see any use case that can require this.
>
> I also don't completely understand what you're trying to achieve. Can you
> please describe the sequence of serialization/deserialization events that
> you expect in your application?
>
> -Val
>
> On Mon, Nov 9, 2015 at 5:13 PM, 姜 为 <yd...@gmail.com> wrote:
>
>> Hi:
>>
>> The implementations name maybe not same.
>>
>> Like my example:
>>
>> class DataEntity {
>> Integer id;
>> string name;
>> Integer age;
>>
>> ….. and other
>>
>> Externalizable.read(in){
>> if(in.readBoolean()) {
>> id = in.readInt();
>> }
>> same as name,age...
>> }
>>
>> Externalizable.write(out){
>> out.writeBoolean(id != null);
>> if(id != null) {
>> out.writeInt(id);
>> }
>> same as name,age...
>> }
>> }
>>
>> class ServiceEntity {
>> Integer id;
>> Integer age;
>>
>> …. and other
>>
>> Externalizable.read(in){
>> if(in.readBoolean()) {
>> id = in.readInt();
>> }
>> if(in.readBoolean()) {
>> in.readString(); // ignore name
>> }
>> if(in.readBoolean()) {
>> age = in.readInt();
>> }
>> … and other
>> }
>>
>> Externalizable.write(out){
>> out.writeBoolean(id != null);
>> if(id != null) {
>> out.writeInt(id);
>> }
>> out.writeBoolean(false); // null for name
>> out.writeBoolean(age != null);
>> if(age != null) {
>> out.writeInt(age);
>> }
>> … and other
>> }
>> }
>>
>> The other implementation class can be serialized and deserialized
>> custom rules by Externalizable.read and Externalizable.write.
>> Each of the different types of servers property needs are
>> different,it does not require a complete serialization.
>>
>>
>>
>>
>>> 在 2015年11月9日，下午10:22，Denis Magda <dm...@gridgain.com> 写道：
>>>
>>> Hi,
>>>
>>> As I understand both servers have different implementations but the
>> names of those implementations are the same, correct?
>>> Because otherwise I don't see how your code could get to the point of
>> checksum validation if one implementation's name is ServiceEntity while the
>> other's is DataEntity.
>>> If my assumptions above are correct then I would recommend to do the
>> following:
>>> 1) Extend Serializable instead of Externalizable
>>>
>>> interface Entity extends Serializable {
>>> .....
>>> }
>>>
>>> 2) Add custom serialVersionUID to each implementation. This will help
>> you get rid off checksum related exception
>>> class EntityImpl implements Entity {
>>> private static final long serialVersionUID = 0L;
>>> ......
>>> }
>>>
>>>
>>> Regards,
>>> Denis
>>>
>>> On 11/8/2015 3:27 PM, 姜 为 wrote:
>>>> Hi guys:
>>>>
>>>> I’m using ignite 1.4.
>>>> In IgniteCompute.call will transfer of an object to the cluster.
>>>> The object should implement Serializable or Externalizable
>> interface.
>>>> OptimizedClassDescriptor.read method will check whether the object
>> is in the same class.
>>>> In my use case,I have some type of servers in cluster.
>>>> The server type A will check the business,and the server type B
>> will persistent data.
>>>> There is a entity interface Entity extends Externalizable have
>> different implementations on different servers.
>>>> Such like this:
>>>>
>>>> interface Entity extends Externalizable {
>>>> method a();
>>>> method b();
>>>> method c();
>>>> }
>>>>
>>>> class ServiceEntity implements Entity {
>>>> method a(){
>>>> // do something...
>>>> }
>>>>
>>>> method b(){
>>>> // do something...
>>>> }
>>>>
>>>> method c(){
>>>> throw new UnsupportedException...
>>>> }
>>>>
>>>> Externalizable.read...
>>>> Externalizable.write...
>>>> }
>>>>
>>>> class DataEntity implements Entity {
>>>> method a(){
>>>> // do something...
>>>> }
>>>>
>>>> method b(){
>>>> throw new UnsupportedException...
>>>> }
>>>>
>>>> method c(){
>>>> // do something...
>>>> }
>>>>
>>>> Externalizable.read...
>>>> Externalizable.write...
>>>> }
>>>>
>>>>
>>>> And IgniteCompute.call(new IgniteCallable(
>>>> public Object call(){
>>>> Entity.a() or b and c;..
>>>> }
>>>> ));
>>>>
>>>> Different implementations of the same class are to achieve read and
>> write methods.
>>>> But OptimizedClassDescriptor.read will check the class sum and throw
>> ClassNotFoundException.
>>>> I recommend verifyChecksum object set as optional,and I really need
>> is change.
>>>> Here is my pr:
>>>> https://issues.apache.org/jira/browse/IGNITE-1854 <
>> https://issues.apache.org/jira/browse/IGNITE-1854> <
>> https://issues.apache.org/jira/browse/IGNITE-1854> <
>> https://issues.apache.org/jira/browse/IGNITE-1854>
>>>> https://github.com/apache/ignite/pull/200/ <
>> https://github.com/apache/ignite/pull/200/> <
>> https://github.com/apache/ignite/pull/200/> <
>> https://github.com/apache/ignite/pull/200/>
>>>>
>>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by Valentin Kulichenko <va...@gmail.com>.
Hi,
I'm against optional checksum verification. It's not safe, adds one more
configuration property and I don't see any use case that can require this.
I also don't completely understand what you're trying to achieve. Can you
please describe the sequence of serialization/deserialization events that
you expect in your application?
-Val
On Mon, Nov 9, 2015 at 5:13 PM, 姜 为 <yd...@gmail.com> wrote:
> Hi:
>
> The implementations name maybe not same.
>
> Like my example:
>
> class DataEntity {
> Integer id;
> string name;
> Integer age;
>
> ….. and other
>
> Externalizable.read(in){
> if(in.readBoolean()) {
> id = in.readInt();
> }
> same as name,age...
> }
>
> Externalizable.write(out){
> out.writeBoolean(id != null);
> if(id != null) {
> out.writeInt(id);
> }
> same as name,age...
> }
> }
>
> class ServiceEntity {
> Integer id;
> Integer age;
>
> …. and other
>
> Externalizable.read(in){
> if(in.readBoolean()) {
> id = in.readInt();
> }
> if(in.readBoolean()) {
> in.readString(); // ignore name
> }
> if(in.readBoolean()) {
> age = in.readInt();
> }
> … and other
> }
>
> Externalizable.write(out){
> out.writeBoolean(id != null);
> if(id != null) {
> out.writeInt(id);
> }
> out.writeBoolean(false); // null for name
> out.writeBoolean(age != null);
> if(age != null) {
> out.writeInt(age);
> }
> … and other
> }
> }
>
> The other implementation class can be serialized and deserialized
> custom rules by Externalizable.read and Externalizable.write.
> Each of the different types of servers property needs are
> different,it does not require a complete serialization.
>
>
>
>
> > 在 2015年11月9日，下午10:22，Denis Magda <dm...@gridgain.com> 写道：
> >
> > Hi,
> >
> > As I understand both servers have different implementations but the
> names of those implementations are the same, correct?
> > Because otherwise I don't see how your code could get to the point of
> checksum validation if one implementation's name is ServiceEntity while the
> other's is DataEntity.
> >
> > If my assumptions above are correct then I would recommend to do the
> following:
> >
> > 1) Extend Serializable instead of Externalizable
> >
> > interface Entity extends Serializable {
> > .....
> > }
> >
> > 2) Add custom serialVersionUID to each implementation. This will help
> you get rid off checksum related exception
> >
> > class EntityImpl implements Entity {
> > private static final long serialVersionUID = 0L;
> > ......
> > }
> >
> >
> > Regards,
> > Denis
> >
> > On 11/8/2015 3:27 PM, 姜 为 wrote:
> >> Hi guys:
> >>
> >> I’m using ignite 1.4.
> >> In IgniteCompute.call will transfer of an object to the cluster.
> >> The object should implement Serializable or Externalizable
> interface.
> >> OptimizedClassDescriptor.read method will check whether the object
> is in the same class.
> >>
> >> In my use case,I have some type of servers in cluster.
> >> The server type A will check the business,and the server type B
> will persistent data.
> >> There is a entity interface Entity extends Externalizable have
> different implementations on different servers.
> >> Such like this:
> >>
> >> interface Entity extends Externalizable {
> >> method a();
> >> method b();
> >> method c();
> >> }
> >>
> >> class ServiceEntity implements Entity {
> >> method a(){
> >> // do something...
> >> }
> >>
> >> method b(){
> >> // do something...
> >> }
> >>
> >> method c(){
> >> throw new UnsupportedException...
> >> }
> >>
> >> Externalizable.read...
> >> Externalizable.write...
> >> }
> >>
> >> class DataEntity implements Entity {
> >> method a(){
> >> // do something...
> >> }
> >>
> >> method b(){
> >> throw new UnsupportedException...
> >> }
> >>
> >> method c(){
> >> // do something...
> >> }
> >>
> >> Externalizable.read...
> >> Externalizable.write...
> >> }
> >>
> >>
> >> And IgniteCompute.call(new IgniteCallable(
> >> public Object call(){
> >> Entity.a() or b and c;..
> >> }
> >> ));
> >>
> >> Different implementations of the same class are to achieve read and
> write methods.
> >> But OptimizedClassDescriptor.read will check the class sum and throw
> ClassNotFoundException.
> >>
> >> I recommend verifyChecksum object set as optional,and I really need
> is change.
> >>
> >> Here is my pr:
> >> https://issues.apache.org/jira/browse/IGNITE-1854 <
> https://issues.apache.org/jira/browse/IGNITE-1854> <
> https://issues.apache.org/jira/browse/IGNITE-1854> <
> https://issues.apache.org/jira/browse/IGNITE-1854>
> >> https://github.com/apache/ignite/pull/200/ <
> https://github.com/apache/ignite/pull/200/> <
> https://github.com/apache/ignite/pull/200/> <
> https://github.com/apache/ignite/pull/200/>
> >>
> >>
> >
>
>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by 姜 为 <yd...@gmail.com>.
Hi:
The implementations name maybe not same.
Like my example:
class DataEntity {
Integer id;
string name;
Integer age;
….. and other
Externalizable.read(in){
if(in.readBoolean()) {
id = in.readInt();
}
same as name,age...
}
Externalizable.write(out){
out.writeBoolean(id != null);
if(id != null) {
out.writeInt(id);
}
same as name,age...
}
}
class ServiceEntity {
Integer id;
Integer age;
…. and other
Externalizable.read(in){
if(in.readBoolean()) {
id = in.readInt();
}
if(in.readBoolean()) {
in.readString(); // ignore name
}
if(in.readBoolean()) {
age = in.readInt();
}
… and other
}
Externalizable.write(out){
out.writeBoolean(id != null);
if(id != null) {
out.writeInt(id);
}
out.writeBoolean(false); // null for name
out.writeBoolean(age != null);
if(age != null) {
out.writeInt(age);
}
… and other
}
}
The other implementation class can be serialized and deserialized custom rules by Externalizable.read and Externalizable.write.
Each of the different types of servers property needs are different,it does not require a complete serialization.
> 在 2015年11月9日，下午10:22，Denis Magda <dm...@gridgain.com> 写道：
>
> Hi,
>
> As I understand both servers have different implementations but the names of those implementations are the same, correct?
> Because otherwise I don't see how your code could get to the point of checksum validation if one implementation's name is ServiceEntity while the other's is DataEntity.
>
> If my assumptions above are correct then I would recommend to do the following:
>
> 1) Extend Serializable instead of Externalizable
>
> interface Entity extends Serializable {
> .....
> }
>
> 2) Add custom serialVersionUID to each implementation. This will help you get rid off checksum related exception
>
> class EntityImpl implements Entity {
> private static final long serialVersionUID = 0L;
> ......
> }
>
>
> Regards,
> Denis
>
> On 11/8/2015 3:27 PM, 姜 为 wrote:
>> Hi guys:
>>
>> I’m using ignite 1.4.
>> In IgniteCompute.call will transfer of an object to the cluster.
>> The object should implement Serializable or Externalizable interface.
>> OptimizedClassDescriptor.read method will check whether the object is in the same class.
>>
>> In my use case,I have some type of servers in cluster.
>> The server type A will check the business,and the server type B will persistent data.
>> There is a entity interface Entity extends Externalizable have different implementations on different servers.
>> Such like this:
>>
>> interface Entity extends Externalizable {
>> method a();
>> method b();
>> method c();
>> }
>>
>> class ServiceEntity implements Entity {
>> method a(){
>> // do something...
>> }
>>
>> method b(){
>> // do something...
>> }
>>
>> method c(){
>> throw new UnsupportedException...
>> }
>>
>> Externalizable.read...
>> Externalizable.write...
>> }
>>
>> class DataEntity implements Entity {
>> method a(){
>> // do something...
>> }
>>
>> method b(){
>> throw new UnsupportedException...
>> }
>>
>> method c(){
>> // do something...
>> }
>>
>> Externalizable.read...
>> Externalizable.write...
>> }
>>
>>
>> And IgniteCompute.call(new IgniteCallable(
>> public Object call(){
>> Entity.a() or b and c;..
>> }
>> ));
>>
>> Different implementations of the same class are to achieve read and write methods.
>> But OptimizedClassDescriptor.read will check the class sum and throw ClassNotFoundException.
>>
>> I recommend verifyChecksum object set as optional,and I really need is change.
>>
>> Here is my pr:
>> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854> <https://issues.apache.org/jira/browse/IGNITE-1854> <https://issues.apache.org/jira/browse/IGNITE-1854>
>> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/> <https://github.com/apache/ignite/pull/200/> <https://github.com/apache/ignite/pull/200/>
>>
>>
>
Re: OptimizedClassDescriptor verifyChecksum object set as optional
Posted by Denis Magda <dm...@gridgain.com>.
Hi,
As I understand both servers have different implementations but the
names of those implementations are the same, correct?
Because otherwise I don't see how your code could get to the point of
checksum validation if one implementation's name is ServiceEntity while
the other's is DataEntity.
If my assumptions above are correct then I would recommend to do the
following:
1) Extend Serializable instead of Externalizable
interface Entity extends Serializable {
.....
}
2) Add custom serialVersionUID to each implementation. This will help
you get rid off checksum related exception
class EntityImpl implements Entity {
private static final long serialVersionUID =0L;
......
}
Regards,
Denis
On 11/8/2015 3:27 PM, 姜 为 wrote:
> Hi guys:
>
> I’m using ignite 1.4.
> In IgniteCompute.call will transfer of an object to the cluster.
> The object should implement Serializable or Externalizable interface.
> OptimizedClassDescriptor.read method will check whether the object is in the same class.
>
> In my use case,I have some type of servers in cluster.
> The server type A will check the business,and the server type B will persistent data.
> There is a entity interface Entity extends Externalizable have different implementations on different servers.
> Such like this:
>
> interface Entity extends Externalizable {
> method a();
> method b();
> method c();
> }
>
> class ServiceEntity implements Entity {
> method a(){
> // do something...
> }
>
> method b(){
> // do something...
> }
>
> method c(){
> throw new UnsupportedException...
> }
>
> Externalizable.read...
> Externalizable.write...
> }
>
> class DataEntity implements Entity {
> method a(){
> // do something...
> }
>
> method b(){
> throw new UnsupportedException...
> }
>
> method c(){
> // do something...
> }
>
> Externalizable.read...
> Externalizable.write...
> }
>
>
> And IgniteCompute.call(new IgniteCallable(
> public Object call(){
> Entity.a() or b and c;..
> }
> ));
>
> Different implementations of the same class are to achieve read and write methods.
> But OptimizedClassDescriptor.read will check the class sum and throw ClassNotFoundException.
>
> I recommend verifyChecksum object set as optional,and I really need is change.
>
> Here is my pr:
> https://issues.apache.org/jira/browse/IGNITE-1854 <https://issues.apache.org/jira/browse/IGNITE-1854>
> https://github.com/apache/ignite/pull/200/ <https://github.com/apache/ignite/pull/200/>
>
>