You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "Rotondi, Antonio " <an...@credit-suisse.com> on 2021/12/02 08:56:47 UTC
RE: Binary types configuration and cluster configuration consistency between server and client nodes
Hi Tupitsyn,
Thanks for your reply.
Can you please clarify your suggestion about removing types form the static binaryconfiguration?
To clarify, all our caches store objects in binary format. The reason why we need custom serializers is to allow conversion of basic java types like Instant and LocalDate (which I suggest you include in the Ignite core) to the supported Ignite binary ones..
I tried with a general serializer setup but that was trying to deserialize even lambdas sent to the server nodes. On the other side, type specific serialization configurations at nested level (e.g. an Instant inside a custom class) does not seem to work as the process manager is only checking for the top type when assigning serializers and when a not blanket serializer is configured (in other words it does not seem to recurse inside not configured types for configured types).
In this scenario we are forced to register all our business objects containing types we want to convert to supported binary types with our seriailzer and every time we add a new type to the cache, on top of dropping the new classes needed for the new cash configurations into the server nodes and changing the cluster configuration to add the new binary types, we also need to modify the cluster configuration for all clients, including the ones that are not concerned with the new business types.
Personally I would have made binary types a cache specific configuration instead of a cluster global one.
Thanks again for all the suggestions you may provide and, please, give me a pointer to that static type clean-up advise you sent me before.
Regards,
Antonio
From: Pavel Tupitsyn <pt...@apache.org>
Sent: 30 November 2021 13:49
To: user
Subject: Re: Binary types configuration and cluster configuration consistency between server and client nodes
Hello Antonio,
You can solve this by removing all types from the static BinaryConfiguration,
and rely on dynamic type registration instead.
In most cases, you don't need to do anything extra and types will be registered automatically on first use (e.g. cache.put call).
If you still encounter an edge case where a type can't be resolved, you can force the type registration like this:
ignite.binary().type(MyClass.class)
https://ignite.apache.org/docs/latest/data-modeling/data-modeling
On Tue, Nov 30, 2021 at 4:09 PM Rotondi, Antonio <an...@credit-suisse.com>> wrote:
Hello,
I have a question regarding an application we have already in production and for which we are trying to clean as much as possible the dependencies between server node and client node ones resources.
In particular we are seeing that it is not possible to set the binary types configuration in the server cluster configuration without adding that to all client nodes cluster configuration as well. This is a rather annoying dependency to manage as it will force the redeployment or patching of all the client nodes when a new type is added to the cache (think microservices) .
I could not find a way to circumvent that as the client needs to connect tot the gird before it can call a remote task to pull the binary types configuration and the client will fail to start ignite with an error stating a difference between the server and client binary type configuration.
This server resources dependencies in the client seems to be a general issue across the whole Ignite architecture, especially because configuration related ones are not included in class peer loading.
Many thanks for any advise anyone can give me.
Best regards,
Antonio
===============================================================================
Please access the attached hyperlink for an important data protection disclaimer:
https://www.credit-suisse.com/uk/en/legal/privacy-statement.html
If you have received any marketing or product promotion communications in this
email, you can unsubscribe from receiving such communications by contacting the
sender of this message or your regular Credit Suisse contact.
===============================================================================
==============================================================================
Where required by law, and except for FX instruments, pre-trade mid-market mark
is the arithmetic mean of bid and offer prices unless otherwise stated; other
regulatory disclosures are at https://plus.credit-suisse.com
==============================================================================
==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================
===============================================================================
Please access the attached hyperlink for an important data protection disclaimer:
https://www.credit-suisse.com/uk/en/legal/privacy-statement.html
If you have received any marketing or product promotion communications in this email, you can
unsubscribe from receiving such communications by contacting the sender of this message or
your regular Credit Suisse contact.
===============================================================================
==============================================================================
Where required by law, and except for FX instruments, pre-trade mid-market mark
is the arithmetic mean of bid and offer prices unless otherwise stated; other
regulatory disclosures are at https://plus.credit-suisse.com
==============================================================================
===============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
===============================================================================
Re: Binary types configuration and cluster configuration consistency between server and client nodes
Posted by Pavel Tupitsyn <pt...@apache.org>.
> we are forced to register all our business objects
> changing the cluster configuration to add the new binary types
I'm trying to say that you don't need to do that.
Instead of listing all binary types in configuration, you can register them
at runtime, after the cluster has started.
On Thu, Dec 2, 2021 at 11:57 AM Rotondi, Antonio <
antonio.rotondi@credit-suisse.com> wrote:
> Hi Tupitsyn,
>
> Thanks for your reply.
>
>
>
> Can you please clarify your suggestion about removing types form the
> static binaryconfiguration?
>
>
>
> To clarify, all our caches store objects in binary format. The reason why
> we need custom serializers is to allow conversion of basic java types like
> Instant and LocalDate (which I suggest you include in the Ignite core) to
> the supported Ignite binary ones..
>
> I tried with a general serializer setup but that was trying to deserialize
> even lambdas sent to the server nodes. On the other side, type specific
> serialization configurations at nested level (e.g. an Instant inside a
> custom class) does not seem to work as the process manager is only checking
> for the top type when assigning serializers and when a not blanket
> serializer is configured (in other words it does not seem to recurse inside
> not configured types for configured types).
>
> In this scenario we are forced to register all our business objects
> containing types we want to convert to supported binary types with our
> seriailzer and every time we add a new type to the cache, on top of
> dropping the new classes needed for the new cash configurations into the
> server nodes and changing the cluster configuration to add the new binary
> types, we also need to modify the cluster configuration for all clients,
> including the ones that are not concerned with the new business types.
>
> Personally I would have made binary types a cache specific configuration
> instead of a cluster global one.
>
>
>
> Thanks again for all the suggestions you may provide and, please, give me
> a pointer to that static type clean-up advise you sent me before.
>
>
>
> Regards,
>
> Antonio
>
>
>
> *From:* Pavel Tupitsyn <pt...@apache.org>
> *Sent:* 30 November 2021 13:49
> *To:* user
> *Subject:* Re: Binary types configuration and cluster configuration
> consistency between server and client nodes
>
>
>
> Hello Antonio,
>
>
>
> You can solve this by removing all types from the static
> BinaryConfiguration,
>
> and rely on dynamic type registration instead.
>
>
>
> In most cases, you don't need to do anything extra and types will be
> registered automatically on first use (e.g. cache.put call).
>
> If you still encounter an edge case where a type can't be resolved, you
> can force the type registration like this:
>
> ignite.binary().type(MyClass.class)
>
>
>
> https://ignite.apache.org/docs/latest/data-modeling/data-modeling
>
>
>
> On Tue, Nov 30, 2021 at 4:09 PM Rotondi, Antonio <
> antonio.rotondi@credit-suisse.com> wrote:
>
> Hello,
>
> I have a question regarding an application we have already in production
> and for which we are trying to clean as much as possible the dependencies
> between server node and client node ones resources.
>
> In particular we are seeing that it is not possible to set the binary
> types configuration in the server cluster configuration without adding that
> to all client nodes cluster configuration as well. This is a rather
> annoying dependency to manage as it will force the redeployment or patching
> of all the client nodes when a new type is added to the cache (think
> microservices) .
>
> I could not find a way to circumvent that as the client needs to connect
> tot the gird before it can call a remote task to pull the binary types
> configuration and the client will fail to start ignite with an error
> stating a difference between the server and client binary type
> configuration.
>
> This server resources dependencies in the client seems to be a general
> issue across the whole Ignite architecture, especially because
> configuration related ones are not included in class peer loading.
>
>
>
> Many thanks for any advise anyone can give me.
>
>
>
> Best regards,
>
> Antonio
>
>
>
>
>
>
> ===============================================================================
> Please access the attached hyperlink for an important data protection
> disclaimer:
> https://www.credit-suisse.com/uk/en/legal/privacy-statement.html
> If you have received any marketing or product promotion communications in
> this
> email, you can unsubscribe from receiving such communications by
> contacting the
> sender of this message or your regular Credit Suisse contact.
>
> ===============================================================================
>
>
>
>
> ==============================================================================
> Where required by law, and except for FX instruments, pre-trade mid-market
> mark
> is the arithmetic mean of bid and offer prices unless otherwise stated;
> other
> regulatory disclosures are at https://plus.credit-suisse.com
>
> ==============================================================================
>
>
>
>
> ==============================================================================
> Please access the attached hyperlink for an important electronic
> communications disclaimer:
> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
>
> ==============================================================================
>
>
>
> ===============================================================================
> Please access the attached hyperlink for an important data protection
> disclaimer:
> https://www.credit-suisse.com/uk/en/legal/privacy-statement.html
> If you have received any marketing or product promotion communications in
> this
> email, you can unsubscribe from receiving such communications by
> contacting the
> sender of this message or your regular Credit Suisse contact.
>
> ===============================================================================
>
>
> ==============================================================================
> Where required by law, and except for FX instruments, pre-trade mid-market
> mark
> is the arithmetic mean of bid and offer prices unless otherwise stated;
> other
> regulatory disclosures are at https://plus.credit-suisse.com
>
> ==============================================================================
>
>
> ==============================================================================
> Please access the attached hyperlink for an important electronic
> communications disclaimer:
> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
>
> ==============================================================================
>