You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by rpinzon <rp...@gmail.com> on 2018/07/12 23:13:07 UTC
Nodes distribution/nodes in parallel on one host using Yarn
Hi there,
I've been using Apache Ignite 1.8 in standalone mode for a while and as I'm
upgrading to 2.5.0 I'm trying to use YARN deployment
(https://apacheignite.readme.io/docs/yarn-deployment).
I deployed Ignite and it is running fine on YARN using ZooKeeper discovery
but when I increase the number of nodes (IGNITE_NODE_COUNT) seemingly they
start in the same host as the first node started until the host is out of
resources to start another one or when it reaches a total of 5 nodes started
in sequence.
As I didn't found any documentation about node distribution or number of
nodes in parallel on one host I'm concluding this based on some experiments
that I've made.
I found a parameter (/-m/) used with /start/ command
(https://apacheignite-tools.readme.io/v2.5/docs/start-command) in
ignitevisor that apparently do the trick and reinforces what I concluded
about reaching 5 nodes (it's the default value for /-m/).
I'd like to know if there is any option to change this policy when starting
the cluster or if the only option is starting manually through the visor.
Thanks in advance!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Nodes distribution/nodes in parallel on one host using Yarn
Posted by Renan Pinzon <rp...@gmail.com>.
Hi Ilya,
First of all thanks for your response!
I thought about increasing the resources to prevent more than one node on a
host but I believe that in this case it will be better to keep the cluster
running in standalone mode. Firstly because I won't make use of affinity
between Ignite and Spark cause I won't have resources to run Spark
executors on those hosts. Secondly because I'll decrease the capacity of
resources for data processing with Spark, so I believe it's better to give
resources that are not part of my Hadoop/Spark cluster.
Another solution that I thought about is using YARN's resource pool but
it's somewhat limited in relation to resource management per host.
I'll try to find which property is used during nodes startup to define the
maximum number of parallel nodes per host and fill an issue.
Regards,
Renan T. Pinzon
On Fri, Jul 13, 2018 at 6:02 AM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> I'm afraid you will have to debug it and fill and issue since there's not
> much expertise in YARN deployments here.
>
> One thing I see in the source that additional nodes might be brought up on
> the same host if it has extra resources (as determined by YARN resource
> management)
>
> I suggest increasing resources used by instance so that one server can
> only hose one instance.
>
> Note that you have to put in IGNITE_MEMORY_OVERHEAD_PER_NODE
> <https://apacheignite.readme.io/docs/yarn-deployment#section-configuration>
> memory used by all Ignite off-heap memory regions, i.e., all data stored in
> caches, as well as some padding.
>
> Regards,
>
> --
> Ilya Kasnacheev
>
> 2018-07-13 2:13 GMT+03:00 rpinzon <rp...@gmail.com>:
>
>> Hi there,
>>
>> I've been using Apache Ignite 1.8 in standalone mode for a while and as
>> I'm
>> upgrading to 2.5.0 I'm trying to use YARN deployment
>> (https://apacheignite.readme.io/docs/yarn-deployment).
>>
>> I deployed Ignite and it is running fine on YARN using ZooKeeper discovery
>> but when I increase the number of nodes (IGNITE_NODE_COUNT) seemingly they
>> start in the same host as the first node started until the host is out of
>> resources to start another one or when it reaches a total of 5 nodes
>> started
>> in sequence.
>>
>> As I didn't found any documentation about node distribution or number of
>> nodes in parallel on one host I'm concluding this based on some
>> experiments
>> that I've made.
>>
>> I found a parameter (/-m/) used with /start/ command
>> (https://apacheignite-tools.readme.io/v2.5/docs/start-command) in
>> ignitevisor that apparently do the trick and reinforces what I concluded
>> about reaching 5 nodes (it's the default value for /-m/).
>>
>> I'd like to know if there is any option to change this policy when
>> starting
>> the cluster or if the only option is starting manually through the visor.
>>
>> Thanks in advance!
>>
>>
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>
Re: Nodes distribution/nodes in parallel on one host using Yarn
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I'm afraid you will have to debug it and fill and issue since there's not
much expertise in YARN deployments here.
One thing I see in the source that additional nodes might be brought up on
the same host if it has extra resources (as determined by YARN resource
management)
I suggest increasing resources used by instance so that one server can only
hose one instance.
Note that you have to put in IGNITE_MEMORY_OVERHEAD_PER_NODE
<https://apacheignite.readme.io/docs/yarn-deployment#section-configuration>
memory used by all Ignite off-heap memory regions, i.e., all data stored in
caches, as well as some padding.
Regards,
--
Ilya Kasnacheev
2018-07-13 2:13 GMT+03:00 rpinzon <rp...@gmail.com>:
> Hi there,
>
> I've been using Apache Ignite 1.8 in standalone mode for a while and as I'm
> upgrading to 2.5.0 I'm trying to use YARN deployment
> (https://apacheignite.readme.io/docs/yarn-deployment).
>
> I deployed Ignite and it is running fine on YARN using ZooKeeper discovery
> but when I increase the number of nodes (IGNITE_NODE_COUNT) seemingly they
> start in the same host as the first node started until the host is out of
> resources to start another one or when it reaches a total of 5 nodes
> started
> in sequence.
>
> As I didn't found any documentation about node distribution or number of
> nodes in parallel on one host I'm concluding this based on some experiments
> that I've made.
>
> I found a parameter (/-m/) used with /start/ command
> (https://apacheignite-tools.readme.io/v2.5/docs/start-command) in
> ignitevisor that apparently do the trick and reinforces what I concluded
> about reaching 5 nodes (it's the default value for /-m/).
>
> I'd like to know if there is any option to change this policy when starting
> the cluster or if the only option is starting manually through the visor.
>
> Thanks in advance!
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>