Re: Affinity collocation and node rebalancing

[lhendra <lh...@apple.com>]

Denis, thank you for the detailed explanation 

One follow-up question, if you don't mind.
Is there a limit to how many tables you can chain in the affinity
collocation? We might have a use case where we want have 30 tables chained
together in affinity collocation (not too many columns each). Providing that
we'll have enough memory, is this something doable in Ignite?

Thanks,
Lily



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Affinity collocation and node rebalancing

[Denis Magda <dm...@apache.org>]

Thanks for raising the questions! We'll update the documentation
accordingly clarifying some tricky areas and scenarios.

-
Denis


On Thu, Aug 1, 2019 at 1:33 PM lhendra <lh...@apple.com> wrote:

> Denis, again this is really great - definitely something that I will take
> into consideration during the schema design.
> Really appreciate your thoughtful replies and explanations!
>
> Lily
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Affinity collocation and node rebalancing

[lhendra <lh...@apple.com>]

Denis, again this is really great - definitely something that I will take
into consideration during the schema design. 
Really appreciate your thoughtful replies and explanations! 

Lily



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Affinity collocation and node rebalancing

[Denis Magda <dm...@apache.org>]

It doesn't seem like an issue to me. The only one thing you should keep in
mind is related to the potential size of partitions.

Let's say you have Cities and Residents tables, and every resident is
collocated with its City ID. So, for New York you'll certainly have more
Residents than for Boston; thus,  the partition that stores all the
Residents of NY will be larger than the Boston's one. It can lead to the
situation when one node consumes more memory space only because it stores
more partitions of the larger size. In reality, and that's considered in
Ignite design, it's assumed that there will be many cities like NY and
Boston and all of them will be evenly distributed across different
partitions and, thus, all the nodes will occupy somehow equal memory space.

For those scenarios, when the even distribution doesn't work out (because
there are just two cities of the size of NY and you have 2 nodes that
consume too much space) you can come up with an alternate collocation
scheme when for Residents living in NY you use a complex affinity key like
(city_id, district_id) and with that help the district_id will also be
involved in the data distribution.

-
Denis


On Thu, Aug 1, 2019 at 10:09 AM lhendra <lh...@apple.com> wrote:

> Denis, thank you for the detailed explanation
>
> One follow-up question, if you don't mind.
> Is there a limit to how many tables you can chain in the affinity
> collocation? We might have a use case where we want have 30 tables chained
> together in affinity collocation (not too many columns each). Providing
> that
> we'll have enough memory, is this something doable in Ignite?
>
> Thanks,
> Lily
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>