You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by gcollins <ga...@gmail.com> on 2016/02/12 03:04:05 UTC

Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Hello,

We have an existing application which takes advantage of the Hazelcast SPI
interface. The Hazelcast SPI interface allowed us to create a quite complex
distributed service allowing us to manage our own memory, migrations and how
we would like to replicate (hazelcast just managed where partitions were on
the cluster and made sure that access on a single partition was always
single-threaded).

I have now been asked to migrate this existing service to Apache Ignite.
Having a quick read of the Ignite service documentation I don't see that I
could have anywhere near that level of control (it looks like custom
services are really just wrappers around IgniteCache). Is that correct? Or
am I just looking in the wrong place? I am probably missing something
obvious here.

thanks in advance,
Gareth Collins



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by gcollins <ga...@gmail.com>.
Can you point me an example of what you are suggesting? Or suggest
what classes I should look at as an entry point?

thanks in advance,
Gareth

On Tue, Feb 16, 2016 at 6:43 PM, dsetrakyan [via Apache Ignite Users]
<ml...@n6.nabble.com> wrote:
>
>
> On Tue, Feb 16, 2016 at 12:00 PM, collins <[hidden email]> wrote:
>
>> I guess the SpiQuery can help me part of the way potentially, but it
>> doesn't give me the control to manage internal memory structure,
>> replication, custom persistence etc.
>>
>> The Hazelcast API I am referring to is described in chapter 24 of the
>> Hazelcast Manual (Extending Hazelcast - User Defined Services). This
>> API was added when they released Hazelcast 3.0. As part of this
>> release, they also rewrote all their internal data structures (e.g.
>> Map, Queue etc) to use this same API so that you can write a service
>> that can do anything their internal data structures can do.
>
>
> Hm… I am not sure I fully understand. Why not just create caches or data
> structures internally within Ignite service and make use of them in your
> service logic? What am I missing?
>
> D.
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p3048.html
> To unsubscribe from Migrating From Hazelcast Service Interface To Apache
> Ignite Service Interface, click here.
> NAML




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p5811.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Tue, Feb 16, 2016 at 12:00 PM, collins <ga...@gmail.com>
wrote:

I guess the SpiQuery can help me part of the way potentially, but it
> doesn't give me the control to manage internal memory structure,
> replication, custom persistence etc.
>
> The Hazelcast API I am referring to is described in chapter 24 of the
> Hazelcast Manual (Extending Hazelcast - User Defined Services). This
> API was added when they released Hazelcast 3.0. As part of this
> release, they also rewrote all their internal data structures (e.g.
> Map, Queue etc) to use this same API so that you can write a service
> that can do anything their internal data structures can do.
>

Hm… I am not sure I fully understand. Why not just create caches or data
structures internally within Ignite service and make use of them in your
service logic? What am I missing?

D.

Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by Denis Magda <dm...@gridgain.com>.
Hi Gareth,

To control data location/disposal you need to write your own AffinityFunction (just refer to this interface). Using AffinityFunction you can select a range of nodes that will hold primary replicas and a range of nodes that will be stored the rest of data.

Probably even more simple scenario that may work for you is to store primary data (primary replica) in a particular cache and use CacheConfiguration.nodeFilter for cache deployment on particular nodes only. The same can be done for summary data if to place it in a separate cache as well.

—
Denis

> On Jun 23, 2016, at 2:50 AM, gcollins <ga...@gmail.com> wrote:
> 
> e.g. what to store on the primary and secondary replica. On the 
> primary replica the full data will be there, on the secondary replica 
> only summary data will be stored (as well as cached query results). 
> 
> My internal memory structure included big byte array blobs (or 
> off-heap memory blobs) which I will be appending to via an insert as 
> well as separate dictionaries to make sense of the binary blobs during 
> queries. 
> 
> thanks, 
> Gareth 
> 
> On Tue, Feb 16, 2016 at 4:07 PM, vkulichenko [via Apache Ignite Users] 
> <[hidden email] <x-msg://30/user/SendEmail.jtp?type=node&node=5812&i=0>> wrote:
> 
> > Hi Gareth, 
> > 
> > gcollins wrote 
> > I guess the SpiQuery can help me part of the way potentially, but it 
> > doesn't give me the control to manage internal memory structure, 
> > replication, custom persistence etc. 
> > 
> > What kind of control do you expect? Can you give an example? In general, 
> > almost everything in Ignite is customizable and pluggable. E.g., to 
> > customize backup node assignments you can implement your own 
> > AffinityFunction, for custom persistence implement CacheStore interface. 
> > 
> > Can you clarify what you mean by "internal memory structure"? Can you create 
> > an Ignite cache (or caches) under the hood and provide your own APIs and/or 
> > services around it? 
> > 
> > -Val 
> > 
> > ________________________________ 
> > If you reply to this email, your message will be added to the discussion 
> > below: 
> > http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p3045.html <http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p3045.html>
> > To unsubscribe from Migrating From Hazelcast Service Interface To Apache 
> > Ignite Service Interface, click here. 
> > NAML 
> 
> View this message in context: Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface <http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p5812.html>
> Sent from the Apache Ignite Users mailing list archive <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com <http://nabble.com/>.


Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by gcollins <ga...@gmail.com>.
e.g. what to store on the primary and secondary replica. On the
primary replica the full data will be there, on the secondary replica
only summary data will be stored (as well as cached query results).

My internal memory structure included big byte array blobs (or
off-heap memory blobs) which I will be appending to via an insert as
well as separate dictionaries to make sense of the binary blobs during
queries.

thanks,
Gareth

On Tue, Feb 16, 2016 at 4:07 PM, vkulichenko [via Apache Ignite Users]
<ml...@n6.nabble.com> wrote:
> Hi Gareth,
>
> gcollins wrote
> I guess the SpiQuery can help me part of the way potentially, but it
> doesn't give me the control to manage internal memory structure,
> replication, custom persistence etc.
>
> What kind of control do you expect? Can you give an example? In general,
> almost everything in Ignite is customizable and pluggable. E.g., to
> customize backup node assignments you can implement your own
> AffinityFunction, for custom persistence implement CacheStore interface.
>
> Can you clarify what you mean by "internal memory structure"? Can you create
> an Ignite cache (or caches) under the hood and provide your own APIs and/or
> services around it?
>
> -Val
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p3045.html
> To unsubscribe from Migrating From Hazelcast Service Interface To Apache
> Ignite Service Interface, click here.
> NAML




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p5812.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by vkulichenko <va...@gmail.com>.
Hi Gareth,


gcollins wrote
> I guess the SpiQuery can help me part of the way potentially, but it
> doesn't give me the control to manage internal memory structure,
> replication, custom persistence etc.

What kind of control do you expect? Can you give an example? In general,
almost everything in Ignite is customizable and pluggable. E.g., to
customize backup node assignments you can implement your own
AffinityFunction, for custom persistence implement CacheStore interface.

Can you clarify what you mean by "internal memory structure"? Can you create
an Ignite cache (or caches) under the hood and provide your own APIs and/or
services around it?

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p3045.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by gcollins <ga...@gmail.com>.
Hi Val,

Thanks again for the response!

I guess the SpiQuery can help me part of the way potentially, but it
doesn't give me the control to manage internal memory structure,
replication, custom persistence etc.

The Hazelcast API I am referring to is described in chapter 24 of the
Hazelcast Manual (Extending Hazelcast - User Defined Services). This
API was added when they released Hazelcast 3.0. As part of this
release, they also rewrote all their internal data structures (e.g.
Map, Queue etc) to use this same API so that you can write a service
that can do anything their internal data structures can do.

thanks,
Gareth



On Fri, Feb 12, 2016 at 4:54 PM, vkulichenko [via Apache Ignite Users]
<ml...@n6.nabble.com> wrote:
> Hi Gareth,
>
> First of all, Ignite provides IndexingSpi interface [1] that you can
> implement to support custom query language and custom indexing
> implementation. The SPI will be used when a special type of query, SpiQuery
> [2], is executed. Is this something that can help you? Let us know if you
> have follow up questions.
>
> Also it's not completely clear which Hazelcast APIs you are referring to.
> Since you mentioned partitions, it looks like you're using caches. Is there
> something on the cache API that is provided by Hazelcast, but not Ignite?
> Internally Ignite cache does not follow thread-per-partition model, but this
> is just an implementation detail. We use other mechanisms to ensure data
> consistency, and I am not sure I understand your concern here. Am I missing
> something?
>
> [1]
> https://ignite.apache.org/releases/1.5.0.final/javadoc/org/apache/ignite/spi/indexing/IndexingSpi.html
> [2]
> https://ignite.apache.org/releases/1.5.0.final/javadoc/org/apache/ignite/cache/query/SpiQuery.html
>
> -Val
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p2992.html
> To unsubscribe from Migrating From Hazelcast Service Interface To Apache
> Ignite Service Interface, click here.
> NAML




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p3041.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by vkulichenko <va...@gmail.com>.
Hi Gareth,

First of all, Ignite provides IndexingSpi interface [1] that you can
implement to support custom query language and custom indexing
implementation. The SPI will be used when a special type of query, SpiQuery
[2], is executed. Is this something that can help you? Let us know if you
have follow up questions.

Also it's not completely clear which Hazelcast APIs you are referring to.
Since you mentioned partitions, it looks like you're using caches. Is there
something on the cache API that is provided by Hazelcast, but not Ignite?
Internally Ignite cache does not follow thread-per-partition model, but this
is just an implementation detail. We use other mechanisms to ensure data
consistency, and I am not sure I understand your concern here. Am I missing
something?

[1]
https://ignite.apache.org/releases/1.5.0.final/javadoc/org/apache/ignite/spi/indexing/IndexingSpi.html
[2]
https://ignite.apache.org/releases/1.5.0.final/javadoc/org/apache/ignite/cache/query/SpiQuery.html

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p2992.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by gcollins <ga...@gmail.com>.
Thanks very much for the responses! Those are all important features of
Ignite, and I believe these features are one of the big reasons why we are
wanting to move away from Hazelcast (in Hazelcast, if you wanted to deploy
different services to different sets of nodes you need to set up different
Hazelcast clusters - ugh).

We effectively have implemented our own in-memory database with its own
specific query language. To minimize memory usage an "insert" into this
database will potentially update dictionaries, then append to a byte array
(or off-heap memory area) rather than create a new record with associated
overhead, then update Cassandra persistence (updating both summary and
detail tables). 

On a service restart, summary tables (with dictionaries and counts) are
automatically loaded into the right partitions. When we do a query, the
queries are done where the data is. If detail data is not already loaded we
load the partitions we needs.

We also do extra optimizing functions like in-memory compression (for
partitions which are queried but do not have inserts) and query result
caching on a per-partition basis. The partition may go and edit the query
(like modifying the query time range to match the partition time range) to
make it more likely we get a cache hit with subsequent queries.

Hazelcast just makes sure that partitions move around correctly and makes
sure that partition access is single-threaded. Since we can also manage what
migrates we select what really gets migrated to minimize the chance of OOM
exceptions (e.g. in a three node cluster, each node may be using 6 GB of
memory - if a node goes down and data migrates the remaining two nodes may
go to 9GB which could cause an OOM).

Does it sound at all possible I could do something similar in Ignite?
Perhaps would it be possible to write my own "IgniteCache" using lower level
building blocks to allow me to go do this?

thanks in advance,
Gareth





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p2984.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by Dmitriy Setrakyan <ds...@apache.org>.
In my view Ignite service grid supports a lot more than Hazelcast and
provides much simpler API for it.

   1. You can deploy service on any single node in the cluster, any cluster
   group, or on the whole grid.

   2. Ignite Service is not a wrapper around cache. It can have any user
   logic within a service and access any kind of data structure provided by
   Ignite. For example, your service can access several Ignite caches, or
   queues, issue computations on the grid, etc, etc.

   3. Ignite service grid supports a rich deployment model. In addition to
   deploying any number of services on any node, you can also use convenient
   service deploy methods on Ignite API to deploy cluster-singletons,
   node-singletons, and key-singletons.

   4. Ignite series can be accessed locally on the servers they are
   deployed on and remotely. In case of remote access, Ignite will still
   provide you with the same deployed service API, but will proxy the calls
   internally to the servers on which the services are deployed.

   5. You can use Ignite services in sticky or non-sticky mode. In case of
   sticky mode, Ignite will always proxy service API calls to the same remote
   server. For non-sticky usage, Ignite will automatically load-balance
   service invocations among the service that have the service deployed.

As Valentin suggested, if you describe your use case, will may be able to
suggest in more detail how it can be implemented with Ignite.

D.

On Thu, Feb 11, 2016 at 9:16 PM, vkulichenko <va...@gmail.com>
wrote:

> Hi Gareth,
>
> Ignite's Service Grid really differs from Hazelcast SPI. From what I see
> these are completely different features.
>
> Can you please elaborate what your service does? If we know more details
> about your use case, it would be easier to propose the best solution.
>
> -Val
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p2971.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Migrating From Hazelcast Service Interface To Apache Ignite Service Interface

Posted by vkulichenko <va...@gmail.com>.
Hi Gareth,

Ignite's Service Grid really differs from Hazelcast SPI. From what I see
these are completely different features.

Can you please elaborate what your service does? If we know more details
about your use case, it would be easier to propose the best solution.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Migrating-From-Hazelcast-Service-Interface-To-Apache-Ignite-Service-Interface-tp2970p2971.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.