You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by mcherkasov <mc...@gridgain.com> on 2017/07/24 14:36:18 UTC
Re: Failure to deserialize simple model object
Hi.
Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscribe@ignite.apache.org and follow simple instructions in
the reply.
Could you please share with us a small pom based project that can reproduce
the problem?
Thanks,
Mikhail.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p15456.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by franck102 <fr...@yahoo.com>.
The issue is significant because it is preventing us from dynamically
destroying / adding caches: if the new cache(s) use new key or value types,
we need to update the binaryConfiguration on all nodes but that is
cluster-level configuration and requires a node restart :(
Franck
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p16245.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by franck102 <fr...@yahoo.com>.
Awesome, thanks!
Franck
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p16333.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by vkulichenko <va...@gmail.com>.
Thanks Alexey, that's great!
Looks like we can expect the fix in 2.2.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p16296.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by "kukushal ." <ku...@gmail.com>.
This is a suggested fix:
https://reviews.ignite.apache.org/ignite/review/IGNT-CR-269
On Fri, Aug 18, 2017 at 12:15 PM, kukushal . <ku...@gmail.com>
wrote:
> I am fixing this issue: https://issues.apache.org/jira/browse/IGNITE-5966?
> filter=-1
> Reviewing it with Vladimir Ozerov and Sergey Chugunov:
> ----------------------------------
> ПРОБЛЕМА
> “get” не работает для данных загруженных из cache store
> ROOT CAUSE
> There two separate type information stores: metadata store backed by the
> file system and binary marshaller’s “type ID -> type name” store. Binary
> marshaller use the latter one when deserialising objects.
> BinaryObjectBuilderImpl used by the CacheJdbcPojoStore updates only
> metadata store but not the marshaller mappings cache. Thus we get this
> issue described in the ticket.
> SOLUTION
> Update BinaryObjectBuilderImpl to add type mapping to the marshaller cache.
>
> On Fri, Aug 18, 2017 at 1:41 AM, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
>
>> Guys,
>>
>> Does anyone has ideas?
>>
>> -Val
>>
>> On Mon, Aug 14, 2017 at 4:33 PM, Valentin Kulichenko <
>> valentin.kulichenko@gmail.com> wrote:
>>
>>> Cross-posting to dev
>>>
>>> Folks,
>>>
>>> I'm confused by the issue discussed in this thread.
>>>
>>> Here is the scenario:
>>> - Start server node with a cache with POJO store configured. There is
>>> one type declared, read-through enabled.
>>> - Start client node and execute get() for a key that exists in
>>> underlying DB.
>>> - During deserialization on the client, 'Requesting mapping from grid
>>> failed for' exception is thrown.
>>>
>>> Specifying the type explicitly in BinaryConfiguration solves the issue,
>>> and I think I understand technical reasons for this. But is this really
>>> expected? Is it possible to fix the issue without requiring to provide this
>>> configuration?
>>>
>>> I thought we do not require to provide types in configuration as long as
>>> there is only one platform involved, am I wrong? If yes, we need to
>>> identify scenarios when this documentation is required and document them.
>>>
>>> -Val
>>>
>>> On Mon, Aug 14, 2017 at 4:23 AM, franck102 <fr...@yahoo.com> wrote:
>>>
>>>> My bad, here is the whole project.
>>>>
>>>> Franck ignite-binary-sample.zip
>>>> <http://apache-ignite-users.70518.x6.nabble.com/file/n16158/
>>>> ignite-binary-sample.zip>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: http://apache-ignite-users.705
>>>> 18.x6.nabble.com/Failure-to-deserialize-simple-model-object-
>>>> tp15440p16158.html
>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>>
>>>
>>>
>>
>
>
> --
> Best regards,
> Alexey
>
--
Best regards,
Alexey
Re: Failure to deserialize simple model object
Posted by "kukushal ." <ku...@gmail.com>.
This is a suggested fix:
https://reviews.ignite.apache.org/ignite/review/IGNT-CR-269
On Fri, Aug 18, 2017 at 12:15 PM, kukushal . <ku...@gmail.com>
wrote:
> I am fixing this issue: https://issues.apache.org/jira/browse/IGNITE-5966?
> filter=-1
> Reviewing it with Vladimir Ozerov and Sergey Chugunov:
> ----------------------------------
> ПРОБЛЕМА
> “get” не работает для данных загруженных из cache store
> ROOT CAUSE
> There two separate type information stores: metadata store backed by the
> file system and binary marshaller’s “type ID -> type name” store. Binary
> marshaller use the latter one when deserialising objects.
> BinaryObjectBuilderImpl used by the CacheJdbcPojoStore updates only
> metadata store but not the marshaller mappings cache. Thus we get this
> issue described in the ticket.
> SOLUTION
> Update BinaryObjectBuilderImpl to add type mapping to the marshaller cache.
>
> On Fri, Aug 18, 2017 at 1:41 AM, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
>
>> Guys,
>>
>> Does anyone has ideas?
>>
>> -Val
>>
>> On Mon, Aug 14, 2017 at 4:33 PM, Valentin Kulichenko <
>> valentin.kulichenko@gmail.com> wrote:
>>
>>> Cross-posting to dev
>>>
>>> Folks,
>>>
>>> I'm confused by the issue discussed in this thread.
>>>
>>> Here is the scenario:
>>> - Start server node with a cache with POJO store configured. There is
>>> one type declared, read-through enabled.
>>> - Start client node and execute get() for a key that exists in
>>> underlying DB.
>>> - During deserialization on the client, 'Requesting mapping from grid
>>> failed for' exception is thrown.
>>>
>>> Specifying the type explicitly in BinaryConfiguration solves the issue,
>>> and I think I understand technical reasons for this. But is this really
>>> expected? Is it possible to fix the issue without requiring to provide this
>>> configuration?
>>>
>>> I thought we do not require to provide types in configuration as long as
>>> there is only one platform involved, am I wrong? If yes, we need to
>>> identify scenarios when this documentation is required and document them.
>>>
>>> -Val
>>>
>>> On Mon, Aug 14, 2017 at 4:23 AM, franck102 <fr...@yahoo.com> wrote:
>>>
>>>> My bad, here is the whole project.
>>>>
>>>> Franck ignite-binary-sample.zip
>>>> <http://apache-ignite-users.70518.x6.nabble.com/file/n16158/
>>>> ignite-binary-sample.zip>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: http://apache-ignite-users.705
>>>> 18.x6.nabble.com/Failure-to-deserialize-simple-model-object-
>>>> tp15440p16158.html
>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>>
>>>
>>>
>>
>
>
> --
> Best regards,
> Alexey
>
--
Best regards,
Alexey
Re: Failure to deserialize simple model object
Posted by "kukushal ." <ku...@gmail.com>.
I am fixing this issue:
https://issues.apache.org/jira/browse/IGNITE-5966?filter=-1
Reviewing it with Vladimir Ozerov and Sergey Chugunov:
----------------------------------
ПРОБЛЕМА
“get” не работает для данных загруженных из cache store
ROOT CAUSE
There two separate type information stores: metadata store backed by the
file system and binary marshaller’s “type ID -> type name” store. Binary
marshaller use the latter one when deserialising objects.
BinaryObjectBuilderImpl used by the CacheJdbcPojoStore updates only
metadata store but not the marshaller mappings cache. Thus we get this
issue described in the ticket.
SOLUTION
Update BinaryObjectBuilderImpl to add type mapping to the marshaller cache.
On Fri, Aug 18, 2017 at 1:41 AM, Valentin Kulichenko <
valentin.kulichenko@gmail.com> wrote:
> Guys,
>
> Does anyone has ideas?
>
> -Val
>
> On Mon, Aug 14, 2017 at 4:33 PM, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
>
>> Cross-posting to dev
>>
>> Folks,
>>
>> I'm confused by the issue discussed in this thread.
>>
>> Here is the scenario:
>> - Start server node with a cache with POJO store configured. There is one
>> type declared, read-through enabled.
>> - Start client node and execute get() for a key that exists in underlying
>> DB.
>> - During deserialization on the client, 'Requesting mapping from grid
>> failed for' exception is thrown.
>>
>> Specifying the type explicitly in BinaryConfiguration solves the issue,
>> and I think I understand technical reasons for this. But is this really
>> expected? Is it possible to fix the issue without requiring to provide this
>> configuration?
>>
>> I thought we do not require to provide types in configuration as long as
>> there is only one platform involved, am I wrong? If yes, we need to
>> identify scenarios when this documentation is required and document them.
>>
>> -Val
>>
>> On Mon, Aug 14, 2017 at 4:23 AM, franck102 <fr...@yahoo.com> wrote:
>>
>>> My bad, here is the whole project.
>>>
>>> Franck ignite-binary-sample.zip
>>> <http://apache-ignite-users.70518.x6.nabble.com/file/n16158/
>>> ignite-binary-sample.zip>
>>>
>>>
>>>
>>> --
>>> View this message in context: http://apache-ignite-users.705
>>> 18.x6.nabble.com/Failure-to-deserialize-simple-model-object-
>>> tp15440p16158.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>
>>
>>
>
--
Best regards,
Alexey
Re: Failure to deserialize simple model object
Posted by "kukushal ." <ku...@gmail.com>.
I am fixing this issue:
https://issues.apache.org/jira/browse/IGNITE-5966?filter=-1
Reviewing it with Vladimir Ozerov and Sergey Chugunov:
----------------------------------
ПРОБЛЕМА
“get” не работает для данных загруженных из cache store
ROOT CAUSE
There two separate type information stores: metadata store backed by the
file system and binary marshaller’s “type ID -> type name” store. Binary
marshaller use the latter one when deserialising objects.
BinaryObjectBuilderImpl used by the CacheJdbcPojoStore updates only
metadata store but not the marshaller mappings cache. Thus we get this
issue described in the ticket.
SOLUTION
Update BinaryObjectBuilderImpl to add type mapping to the marshaller cache.
On Fri, Aug 18, 2017 at 1:41 AM, Valentin Kulichenko <
valentin.kulichenko@gmail.com> wrote:
> Guys,
>
> Does anyone has ideas?
>
> -Val
>
> On Mon, Aug 14, 2017 at 4:33 PM, Valentin Kulichenko <
> valentin.kulichenko@gmail.com> wrote:
>
>> Cross-posting to dev
>>
>> Folks,
>>
>> I'm confused by the issue discussed in this thread.
>>
>> Here is the scenario:
>> - Start server node with a cache with POJO store configured. There is one
>> type declared, read-through enabled.
>> - Start client node and execute get() for a key that exists in underlying
>> DB.
>> - During deserialization on the client, 'Requesting mapping from grid
>> failed for' exception is thrown.
>>
>> Specifying the type explicitly in BinaryConfiguration solves the issue,
>> and I think I understand technical reasons for this. But is this really
>> expected? Is it possible to fix the issue without requiring to provide this
>> configuration?
>>
>> I thought we do not require to provide types in configuration as long as
>> there is only one platform involved, am I wrong? If yes, we need to
>> identify scenarios when this documentation is required and document them.
>>
>> -Val
>>
>> On Mon, Aug 14, 2017 at 4:23 AM, franck102 <fr...@yahoo.com> wrote:
>>
>>> My bad, here is the whole project.
>>>
>>> Franck ignite-binary-sample.zip
>>> <http://apache-ignite-users.70518.x6.nabble.com/file/n16158/
>>> ignite-binary-sample.zip>
>>>
>>>
>>>
>>> --
>>> View this message in context: http://apache-ignite-users.705
>>> 18.x6.nabble.com/Failure-to-deserialize-simple-model-object-
>>> tp15440p16158.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>
>>
>>
>
--
Best regards,
Alexey
Re: Failure to deserialize simple model object
Posted by Valentin Kulichenko <va...@gmail.com>.
Guys,
Does anyone has ideas?
-Val
On Mon, Aug 14, 2017 at 4:33 PM, Valentin Kulichenko <
valentin.kulichenko@gmail.com> wrote:
> Cross-posting to dev
>
> Folks,
>
> I'm confused by the issue discussed in this thread.
>
> Here is the scenario:
> - Start server node with a cache with POJO store configured. There is one
> type declared, read-through enabled.
> - Start client node and execute get() for a key that exists in underlying
> DB.
> - During deserialization on the client, 'Requesting mapping from grid
> failed for' exception is thrown.
>
> Specifying the type explicitly in BinaryConfiguration solves the issue,
> and I think I understand technical reasons for this. But is this really
> expected? Is it possible to fix the issue without requiring to provide this
> configuration?
>
> I thought we do not require to provide types in configuration as long as
> there is only one platform involved, am I wrong? If yes, we need to
> identify scenarios when this documentation is required and document them.
>
> -Val
>
> On Mon, Aug 14, 2017 at 4:23 AM, franck102 <fr...@yahoo.com> wrote:
>
>> My bad, here is the whole project.
>>
>> Franck ignite-binary-sample.zip
>> <http://apache-ignite-users.70518.x6.nabble.com/file/n16158/
>> ignite-binary-sample.zip>
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/Failure-to-deserialize-simple-model-object-
>> tp15440p16158.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
Re: Failure to deserialize simple model object
Posted by Valentin Kulichenko <va...@gmail.com>.
Guys,
Does anyone has ideas?
-Val
On Mon, Aug 14, 2017 at 4:33 PM, Valentin Kulichenko <
valentin.kulichenko@gmail.com> wrote:
> Cross-posting to dev
>
> Folks,
>
> I'm confused by the issue discussed in this thread.
>
> Here is the scenario:
> - Start server node with a cache with POJO store configured. There is one
> type declared, read-through enabled.
> - Start client node and execute get() for a key that exists in underlying
> DB.
> - During deserialization on the client, 'Requesting mapping from grid
> failed for' exception is thrown.
>
> Specifying the type explicitly in BinaryConfiguration solves the issue,
> and I think I understand technical reasons for this. But is this really
> expected? Is it possible to fix the issue without requiring to provide this
> configuration?
>
> I thought we do not require to provide types in configuration as long as
> there is only one platform involved, am I wrong? If yes, we need to
> identify scenarios when this documentation is required and document them.
>
> -Val
>
> On Mon, Aug 14, 2017 at 4:23 AM, franck102 <fr...@yahoo.com> wrote:
>
>> My bad, here is the whole project.
>>
>> Franck ignite-binary-sample.zip
>> <http://apache-ignite-users.70518.x6.nabble.com/file/n16158/
>> ignite-binary-sample.zip>
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/Failure-to-deserialize-simple-model-object-
>> tp15440p16158.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
Re: Failure to deserialize simple model object
Posted by Valentin Kulichenko <va...@gmail.com>.
Cross-posting to dev
Folks,
I'm confused by the issue discussed in this thread.
Here is the scenario:
- Start server node with a cache with POJO store configured. There is one
type declared, read-through enabled.
- Start client node and execute get() for a key that exists in underlying
DB.
- During deserialization on the client, 'Requesting mapping from grid
failed for' exception is thrown.
Specifying the type explicitly in BinaryConfiguration solves the issue, and
I think I understand technical reasons for this. But is this really
expected? Is it possible to fix the issue without requiring to provide this
configuration?
I thought we do not require to provide types in configuration as long as
there is only one platform involved, am I wrong? If yes, we need to
identify scenarios when this documentation is required and document them.
-Val
On Mon, Aug 14, 2017 at 4:23 AM, franck102 <fr...@yahoo.com> wrote:
> My bad, here is the whole project.
>
> Franck ignite-binary-sample.zip
> <http://apache-ignite-users.70518.x6.nabble.com/file/
> n16158/ignite-binary-sample.zip>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Failure-to-deserialize-simple-model-
> object-tp15440p16158.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
Re: Failure to deserialize simple model object
Posted by Valentin Kulichenko <va...@gmail.com>.
Cross-posting to dev
Folks,
I'm confused by the issue discussed in this thread.
Here is the scenario:
- Start server node with a cache with POJO store configured. There is one
type declared, read-through enabled.
- Start client node and execute get() for a key that exists in underlying
DB.
- During deserialization on the client, 'Requesting mapping from grid
failed for' exception is thrown.
Specifying the type explicitly in BinaryConfiguration solves the issue, and
I think I understand technical reasons for this. But is this really
expected? Is it possible to fix the issue without requiring to provide this
configuration?
I thought we do not require to provide types in configuration as long as
there is only one platform involved, am I wrong? If yes, we need to
identify scenarios when this documentation is required and document them.
-Val
On Mon, Aug 14, 2017 at 4:23 AM, franck102 <fr...@yahoo.com> wrote:
> My bad, here is the whole project.
>
> Franck ignite-binary-sample.zip
> <http://apache-ignite-users.70518.x6.nabble.com/file/
> n16158/ignite-binary-sample.zip>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Failure-to-deserialize-simple-model-
> object-tp15440p16158.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
Re: Failure to deserialize simple model object
Posted by franck102 <fr...@yahoo.com>.
My bad, here is the whole project.
Franck ignite-binary-sample.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/n16158/ignite-binary-sample.zip>
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p16158.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by vkulichenko <va...@gmail.com>.
Franck,
I see only POM file attached. Can you attach the whole project you're
referring to?
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p15745.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by franck102 <fr...@yahoo.com>.
BTW if you experiment around you will want to clean up the work/marshallers
directories on server & client between tests; once the typeId is saved there
the problem goes away obviously...
Franck
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p15709.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by franck102 <fr...@yahoo.com>.
Hi Mikhail,
I have extracted the simplest possible configuration, I have attached a
self-contained project that uses an in memory H2 database and reproduces the
problem, here are the steps:
1. mvn install in the root
2. run the junit test org.apache.ignite.sample.test.ServerTests#runIgnite
(this starts the server node, the test method won't return)
3. run the junit test
org.apache.ignite.sample.test.client.ClientTests#loadItem: this starts a
client that tries to get a single entry from the cache.
I get this stack trace, and just as before adding a dummy binary
configuration that simply mentioned the Employees type fixes the problem:
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p15708.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by franck102 <fr...@yahoo.com>.
If I explicitly declare my model class in my binary configuration - just the
class name, no special settings - everything starts working. This causes
org.apache.ignite.internal.binary.BinaryContext#configure will register the
class; I assume something should cause that to happen on the fly, but that
is not happening for me...
<property name="binaryConfiguration">
<bean
class="org.apache.ignite.configuration.BinaryConfiguration">
<property name="typeConfigurations">
<list>
<bean
class="org.apache.ignite.binary.BinaryTypeConfiguration">
<property name="typeName"
value="com.fico.ignite.model.Employees"/>
</bean>
</list>
</property>
</bean>
</property>
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p15490.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by franck102 <fr...@yahoo.com>.
Some more findings after a debugging session:
- the client sends a MissingMappingRequest with the correct typeId; the
server doesn't have the typeId in its context
- I don't understand what/who is supposed to register the typeid with the
MissingMappingRequestListener's marshallerContext; one possible code path is
that a BinaryObjectBuilder would do it when an object instance is
constructed...
- however if I debug the response to the cache.get() client request, my
CacheJdbcPojoStore finds the entry then calls this constructor:
org.apache.ignite.internal.binary.builder.BinaryObjectBuilderImpl#BinaryObjectBuilderImpl(org.apache.ignite.internal.binary.BinaryContext,
int, java.lang.String)
... which doesn't register the typeId (the next constructor does by calling
ctx.descriptorForClass(...)).
I am probably doing something wrong, but I am out of ideas...
Thanks!
Franck
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p15489.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Failure to deserialize simple model object
Posted by franck102 <fr...@yahoo.com>.
Hi Mikhail,
I have subscribed to the mailing list - just didn't long enough before
posting :(
I have attached the POM. Since my post I tried a couple more things:
- I removed the CacheConfiguration elements from the client; I can still
find the server's partitioned caches in ignite.cacheNames(), so I guess
replicating cache configuration on the client is not needed. I wonder why
the webConsole generates it by default; and this is far from obvious for the
documentation...
- With that I still get the exact same problem trying to deserialize a
single cache entry. What I don't understand is that throughout the entire
documentation I see that BinaryMarshaller will be the default if I don't
specify a specific marshaller.
The source code however still creates JdkMarshallers in many places (4-5
when breaking in debug mode), and a JdkMarshaller is the one failing to read
the entry.
The marshaller is taken from
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi which seems unaffected
by any configuration I could create...
POM file for the project:
pom.xml <http://apache-ignite-users.70518.x6.nabble.com/file/n15458/pom.xml>
-
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Failure-to-deserialize-simple-model-object-tp15440p15458.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.