You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Charlin S <Ch...@hotelhub.com> on 2021/04/01 04:02:05 UTC
Re: Ignite server not equally distribution load
Hello,
Thanks for the reply.
Two servers are formed as cluster and the details as below
Topology snapshot [ver=262, locNode=99bfa50b, servers=2, clients=14,
state=ACTIVE, CPUs=188, offheap=64.0GB, heap=97.0GB]
My code snippet in cache creation application
*Step 1* - *Ignite grid instance creation on client node*
IgniteConfiguration igniteGridIg = new IgniteConfiguration();
igniteGridIg.SpringConfigUrl = Path.Combine(igniteXmlfilePath);
IIgnite IgniteInstance = Ignition.Start( igniteGridIg );
*Step 2 - Cache creation* *with following Cache configuration *
CacheConfiguration cg = null;
cg = new CacheConfiguration(cacheName, new QueryEntity(typeof(string),
type));
cg.CacheMode = CacheMode.Replicated;
cg.EnableStatistics = true;
Apache.Ignite.Core.Cache.ICache<string, object> igniteModel = null;
igniteModel = IgniteInstance.GetOrCreateCache<string, object>(cg);
*Step 3 - Cache data store/push in cache*
using (var cacheDataStreamer = IgniteInstance.GetDataStreamer<string,
object>("cacheName"))
{
object dbObj = class model data
cacheDataStreamer.AddData(modelCacheKey, dbObj);
}
Kindly let me know if I need to add/set any more property for this.
Thanks & Regards,
Charlin
On Wed, 31 Mar 2021 at 22:01, Stephen Darlington <
stephen.darlington@gridgain.com> wrote:
> I can see two likely possibilities:
>
> Your two server nodes are not forming a cluster. What do you see in the
> logs? You should see something like this:
>
> [18:42:46] Topology snapshot [ver=9, locNode=a4b47587, servers=2,
> clients=0, state=ACTIVE, CPUs=8, offheap=12.0GB, heap=4.0GB]
>
> Please confirm that it’s using the correct number of servers.
>
> Alternatively, your keys (or affinity keys) are not very random. What kind
> of field are you using as a key? You’ve not shared your cache configuration
> so this is difficult to validate.
>
> On 31 Mar 2021, at 17:04, Charlin S <Ch...@hotelhub.com> wrote:
>
> Hello,
> Thanks for the reply.
> Based on our findings in logs, loads are going to one server.
> The server 10.212.120.66 is struggling to do its work: there are lots of
> "Possible too long JVM pause", "Query execution is too long" and "Long
> running query".
> On the other hand server 10.212.120.67 has a clean log without those
> messages.
> Yes, caches(165 cache models)are created by other windows applications and
> ASP.Net API for only reading data except few cache.
>
> Thanks & Regards,
> Charlin
>
>
>
> On Wed, 31 Mar 2021 at 20:00, Mikael <mi...@gmail.com> wrote:
>
>> Hi!
>>
>> What is it that is not distributed well, is it cache data/memory load or
>> computations/cpu load ?
>>
>> There are no caches in the config so I assume this is created by the
>> application (if there are any), do you use any custom collocation ?
>>
>> Mikael
>>
>> Do you use any custom collocation of any kind
>> On 2021-03-31 15:52, Charlin S wrote:
>>
>> Hello,
>> Thanks for your reply. I have attached my configuration files here.
>> Below code has been used to start ignite.
>>
>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>> igniteGridIg.SpringConfigUrl = Path.Combine(igniteXmlfilePath);
>> Ignition.Start(igniteGridIg);
>>
>> Thanks & Regards,
>> Charlin
>>
>>
>>
>> On Wed, 31 Mar 2021 at 17:44, Stephen Darlington <
>> stephen.darlington@gridgain.com> wrote:
>>
>>> What are you doing with Ignite? Are you sending compute tasks, cache
>>> operations, both? What’s your configuration?
>>>
>>> > On 31 Mar 2021, at 12:31, Charlin S <Ch...@hotelhub.com> wrote:
>>> >
>>> > Hi,
>>> >
>>> > I'm running an ASP.Net application with ignite 2.9.1 and node setup
>>> as 2 server nodes and 11 client nodes.
>>> > We are seeing most of the load on one server only it's supposed to
>>> distribute load between two servers.
>>> > How can we distributed load be equally distributed on both servers?
>>> >
>>> >
>>> > Thanks & Regards,
>>> > Charlin
>>>
>>>
>>>
>
>
Re: Ignite server not equally distribution load
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I recommend collecting (java) thread dumps on both server nodes and
comparing them, under load.
Then you can infer what is the difference in the load that they handle.
Regards,
--
Ilya Kasnacheev
пн, 5 апр. 2021 г. в 07:20, Charlin S <Ch...@hotelhub.com>:
> Hello,
> Thanks for your response,
> Yes, both the servers are on.
>
> Thanks & Regards,
> Charlin
>
>
> On Thu, 1 Apr 2021 at 20:29, Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
>> Hello!
>>
>> What is your baseline topology? Are both of your server nodes on it?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 1 апр. 2021 г. в 07:02, Charlin S <Ch...@hotelhub.com>:
>>
>>> Hello,
>>> Thanks for the reply.
>>> Two servers are formed as cluster and the details as below
>>> Topology snapshot [ver=262, locNode=99bfa50b, servers=2, clients=14,
>>> state=ACTIVE, CPUs=188, offheap=64.0GB, heap=97.0GB]
>>>
>>> My code snippet in cache creation application
>>>
>>> *Step 1* - *Ignite grid instance creation on client node*
>>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>>> igniteGridIg.SpringConfigUrl = Path.Combine(igniteXmlfilePath);
>>> IIgnite IgniteInstance = Ignition.Start( igniteGridIg );
>>>
>>> *Step 2 - Cache creation* *with following Cache configuration *
>>> CacheConfiguration cg = null;
>>> cg = new CacheConfiguration(cacheName, new QueryEntity(typeof(string),
>>> type));
>>> cg.CacheMode = CacheMode.Replicated;
>>> cg.EnableStatistics = true;
>>> Apache.Ignite.Core.Cache.ICache<string, object> igniteModel = null;
>>> igniteModel = IgniteInstance.GetOrCreateCache<string, object>(cg);
>>>
>>> *Step 3 - Cache data store/push in cache*
>>> using (var cacheDataStreamer = IgniteInstance.GetDataStreamer<string,
>>> object>("cacheName"))
>>> {
>>> object dbObj = class model data
>>> cacheDataStreamer.AddData(modelCacheKey, dbObj);
>>> }
>>>
>>> Kindly let me know if I need to add/set any more property for this.
>>>
>>> Thanks & Regards,
>>> Charlin
>>>
>>> On Wed, 31 Mar 2021 at 22:01, Stephen Darlington <
>>> stephen.darlington@gridgain.com> wrote:
>>>
>>>> I can see two likely possibilities:
>>>>
>>>> Your two server nodes are not forming a cluster. What do you see in the
>>>> logs? You should see something like this:
>>>>
>>>> [18:42:46] Topology snapshot [ver=9, locNode=a4b47587, servers=2,
>>>> clients=0, state=ACTIVE, CPUs=8, offheap=12.0GB, heap=4.0GB]
>>>>
>>>> Please confirm that it’s using the correct number of servers.
>>>>
>>>> Alternatively, your keys (or affinity keys) are not very random. What
>>>> kind of field are you using as a key? You’ve not shared your cache
>>>> configuration so this is difficult to validate.
>>>>
>>>> On 31 Mar 2021, at 17:04, Charlin S <Ch...@hotelhub.com> wrote:
>>>>
>>>> Hello,
>>>> Thanks for the reply.
>>>> Based on our findings in logs, loads are going to one server.
>>>> The server 10.212.120.66 is struggling to do its work: there are lots
>>>> of "Possible too long JVM pause", "Query execution is too long" and "Long
>>>> running query".
>>>> On the other hand server 10.212.120.67 has a clean log without those
>>>> messages.
>>>> Yes, caches(165 cache models)are created by other windows applications
>>>> and ASP.Net API for only reading data except few cache.
>>>>
>>>> Thanks & Regards,
>>>> Charlin
>>>>
>>>>
>>>>
>>>> On Wed, 31 Mar 2021 at 20:00, Mikael <mi...@gmail.com> wrote:
>>>>
>>>>> Hi!
>>>>>
>>>>> What is it that is not distributed well, is it cache data/memory load
>>>>> or computations/cpu load ?
>>>>>
>>>>> There are no caches in the config so I assume this is created by the
>>>>> application (if there are any), do you use any custom collocation ?
>>>>>
>>>>> Mikael
>>>>>
>>>>> Do you use any custom collocation of any kind
>>>>> On 2021-03-31 15:52, Charlin S wrote:
>>>>>
>>>>> Hello,
>>>>> Thanks for your reply. I have attached my configuration files here.
>>>>> Below code has been used to start ignite.
>>>>>
>>>>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>>>>> igniteGridIg.SpringConfigUrl = Path.Combine(igniteXmlfilePath);
>>>>> Ignition.Start(igniteGridIg);
>>>>>
>>>>> Thanks & Regards,
>>>>> Charlin
>>>>>
>>>>>
>>>>>
>>>>> On Wed, 31 Mar 2021 at 17:44, Stephen Darlington <
>>>>> stephen.darlington@gridgain.com> wrote:
>>>>>
>>>>>> What are you doing with Ignite? Are you sending compute tasks, cache
>>>>>> operations, both? What’s your configuration?
>>>>>>
>>>>>> > On 31 Mar 2021, at 12:31, Charlin S <Ch...@hotelhub.com> wrote:
>>>>>> >
>>>>>> > Hi,
>>>>>> >
>>>>>> > I'm running an ASP.Net application with ignite 2.9.1 and node
>>>>>> setup as 2 server nodes and 11 client nodes.
>>>>>> > We are seeing most of the load on one server only it's supposed to
>>>>>> distribute load between two servers.
>>>>>> > How can we distributed load be equally distributed on both servers?
>>>>>> >
>>>>>> >
>>>>>> > Thanks & Regards,
>>>>>> > Charlin
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
Re: Ignite server not equally distribution load
Posted by Charlin S <Ch...@hotelhub.com>.
Hello,
Thanks for your response,
Yes, both the servers are on.
Thanks & Regards,
Charlin
On Thu, 1 Apr 2021 at 20:29, Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> What is your baseline topology? Are both of your server nodes on it?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 1 апр. 2021 г. в 07:02, Charlin S <Ch...@hotelhub.com>:
>
>> Hello,
>> Thanks for the reply.
>> Two servers are formed as cluster and the details as below
>> Topology snapshot [ver=262, locNode=99bfa50b, servers=2, clients=14,
>> state=ACTIVE, CPUs=188, offheap=64.0GB, heap=97.0GB]
>>
>> My code snippet in cache creation application
>>
>> *Step 1* - *Ignite grid instance creation on client node*
>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>> igniteGridIg.SpringConfigUrl = Path.Combine(igniteXmlfilePath);
>> IIgnite IgniteInstance = Ignition.Start( igniteGridIg );
>>
>> *Step 2 - Cache creation* *with following Cache configuration *
>> CacheConfiguration cg = null;
>> cg = new CacheConfiguration(cacheName, new QueryEntity(typeof(string),
>> type));
>> cg.CacheMode = CacheMode.Replicated;
>> cg.EnableStatistics = true;
>> Apache.Ignite.Core.Cache.ICache<string, object> igniteModel = null;
>> igniteModel = IgniteInstance.GetOrCreateCache<string, object>(cg);
>>
>> *Step 3 - Cache data store/push in cache*
>> using (var cacheDataStreamer = IgniteInstance.GetDataStreamer<string,
>> object>("cacheName"))
>> {
>> object dbObj = class model data
>> cacheDataStreamer.AddData(modelCacheKey, dbObj);
>> }
>>
>> Kindly let me know if I need to add/set any more property for this.
>>
>> Thanks & Regards,
>> Charlin
>>
>> On Wed, 31 Mar 2021 at 22:01, Stephen Darlington <
>> stephen.darlington@gridgain.com> wrote:
>>
>>> I can see two likely possibilities:
>>>
>>> Your two server nodes are not forming a cluster. What do you see in the
>>> logs? You should see something like this:
>>>
>>> [18:42:46] Topology snapshot [ver=9, locNode=a4b47587, servers=2,
>>> clients=0, state=ACTIVE, CPUs=8, offheap=12.0GB, heap=4.0GB]
>>>
>>> Please confirm that it’s using the correct number of servers.
>>>
>>> Alternatively, your keys (or affinity keys) are not very random. What
>>> kind of field are you using as a key? You’ve not shared your cache
>>> configuration so this is difficult to validate.
>>>
>>> On 31 Mar 2021, at 17:04, Charlin S <Ch...@hotelhub.com> wrote:
>>>
>>> Hello,
>>> Thanks for the reply.
>>> Based on our findings in logs, loads are going to one server.
>>> The server 10.212.120.66 is struggling to do its work: there are lots of
>>> "Possible too long JVM pause", "Query execution is too long" and "Long
>>> running query".
>>> On the other hand server 10.212.120.67 has a clean log without those
>>> messages.
>>> Yes, caches(165 cache models)are created by other windows applications
>>> and ASP.Net API for only reading data except few cache.
>>>
>>> Thanks & Regards,
>>> Charlin
>>>
>>>
>>>
>>> On Wed, 31 Mar 2021 at 20:00, Mikael <mi...@gmail.com> wrote:
>>>
>>>> Hi!
>>>>
>>>> What is it that is not distributed well, is it cache data/memory load
>>>> or computations/cpu load ?
>>>>
>>>> There are no caches in the config so I assume this is created by the
>>>> application (if there are any), do you use any custom collocation ?
>>>>
>>>> Mikael
>>>>
>>>> Do you use any custom collocation of any kind
>>>> On 2021-03-31 15:52, Charlin S wrote:
>>>>
>>>> Hello,
>>>> Thanks for your reply. I have attached my configuration files here.
>>>> Below code has been used to start ignite.
>>>>
>>>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>>>> igniteGridIg.SpringConfigUrl = Path.Combine(igniteXmlfilePath);
>>>> Ignition.Start(igniteGridIg);
>>>>
>>>> Thanks & Regards,
>>>> Charlin
>>>>
>>>>
>>>>
>>>> On Wed, 31 Mar 2021 at 17:44, Stephen Darlington <
>>>> stephen.darlington@gridgain.com> wrote:
>>>>
>>>>> What are you doing with Ignite? Are you sending compute tasks, cache
>>>>> operations, both? What’s your configuration?
>>>>>
>>>>> > On 31 Mar 2021, at 12:31, Charlin S <Ch...@hotelhub.com> wrote:
>>>>> >
>>>>> > Hi,
>>>>> >
>>>>> > I'm running an ASP.Net application with ignite 2.9.1 and node
>>>>> setup as 2 server nodes and 11 client nodes.
>>>>> > We are seeing most of the load on one server only it's supposed to
>>>>> distribute load between two servers.
>>>>> > How can we distributed load be equally distributed on both servers?
>>>>> >
>>>>> >
>>>>> > Thanks & Regards,
>>>>> > Charlin
>>>>>
>>>>>
>>>>>
>>>
>>>
Re: Ignite server not equally distribution load
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
What is your baseline topology? Are both of your server nodes on it?
Regards,
--
Ilya Kasnacheev
чт, 1 апр. 2021 г. в 07:02, Charlin S <Ch...@hotelhub.com>:
> Hello,
> Thanks for the reply.
> Two servers are formed as cluster and the details as below
> Topology snapshot [ver=262, locNode=99bfa50b, servers=2, clients=14,
> state=ACTIVE, CPUs=188, offheap=64.0GB, heap=97.0GB]
>
> My code snippet in cache creation application
>
> *Step 1* - *Ignite grid instance creation on client node*
> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
> igniteGridIg.SpringConfigUrl = Path.Combine(igniteXmlfilePath);
> IIgnite IgniteInstance = Ignition.Start( igniteGridIg );
>
> *Step 2 - Cache creation* *with following Cache configuration *
> CacheConfiguration cg = null;
> cg = new CacheConfiguration(cacheName, new QueryEntity(typeof(string),
> type));
> cg.CacheMode = CacheMode.Replicated;
> cg.EnableStatistics = true;
> Apache.Ignite.Core.Cache.ICache<string, object> igniteModel = null;
> igniteModel = IgniteInstance.GetOrCreateCache<string, object>(cg);
>
> *Step 3 - Cache data store/push in cache*
> using (var cacheDataStreamer = IgniteInstance.GetDataStreamer<string,
> object>("cacheName"))
> {
> object dbObj = class model data
> cacheDataStreamer.AddData(modelCacheKey, dbObj);
> }
>
> Kindly let me know if I need to add/set any more property for this.
>
> Thanks & Regards,
> Charlin
>
> On Wed, 31 Mar 2021 at 22:01, Stephen Darlington <
> stephen.darlington@gridgain.com> wrote:
>
>> I can see two likely possibilities:
>>
>> Your two server nodes are not forming a cluster. What do you see in the
>> logs? You should see something like this:
>>
>> [18:42:46] Topology snapshot [ver=9, locNode=a4b47587, servers=2,
>> clients=0, state=ACTIVE, CPUs=8, offheap=12.0GB, heap=4.0GB]
>>
>> Please confirm that it’s using the correct number of servers.
>>
>> Alternatively, your keys (or affinity keys) are not very random. What
>> kind of field are you using as a key? You’ve not shared your cache
>> configuration so this is difficult to validate.
>>
>> On 31 Mar 2021, at 17:04, Charlin S <Ch...@hotelhub.com> wrote:
>>
>> Hello,
>> Thanks for the reply.
>> Based on our findings in logs, loads are going to one server.
>> The server 10.212.120.66 is struggling to do its work: there are lots of
>> "Possible too long JVM pause", "Query execution is too long" and "Long
>> running query".
>> On the other hand server 10.212.120.67 has a clean log without those
>> messages.
>> Yes, caches(165 cache models)are created by other windows applications
>> and ASP.Net API for only reading data except few cache.
>>
>> Thanks & Regards,
>> Charlin
>>
>>
>>
>> On Wed, 31 Mar 2021 at 20:00, Mikael <mi...@gmail.com> wrote:
>>
>>> Hi!
>>>
>>> What is it that is not distributed well, is it cache data/memory load or
>>> computations/cpu load ?
>>>
>>> There are no caches in the config so I assume this is created by the
>>> application (if there are any), do you use any custom collocation ?
>>>
>>> Mikael
>>>
>>> Do you use any custom collocation of any kind
>>> On 2021-03-31 15:52, Charlin S wrote:
>>>
>>> Hello,
>>> Thanks for your reply. I have attached my configuration files here.
>>> Below code has been used to start ignite.
>>>
>>> IgniteConfiguration igniteGridIg = new IgniteConfiguration();
>>> igniteGridIg.SpringConfigUrl = Path.Combine(igniteXmlfilePath);
>>> Ignition.Start(igniteGridIg);
>>>
>>> Thanks & Regards,
>>> Charlin
>>>
>>>
>>>
>>> On Wed, 31 Mar 2021 at 17:44, Stephen Darlington <
>>> stephen.darlington@gridgain.com> wrote:
>>>
>>>> What are you doing with Ignite? Are you sending compute tasks, cache
>>>> operations, both? What’s your configuration?
>>>>
>>>> > On 31 Mar 2021, at 12:31, Charlin S <Ch...@hotelhub.com> wrote:
>>>> >
>>>> > Hi,
>>>> >
>>>> > I'm running an ASP.Net application with ignite 2.9.1 and node setup
>>>> as 2 server nodes and 11 client nodes.
>>>> > We are seeing most of the load on one server only it's supposed to
>>>> distribute load between two servers.
>>>> > How can we distributed load be equally distributed on both servers?
>>>> >
>>>> >
>>>> > Thanks & Regards,
>>>> > Charlin
>>>>
>>>>
>>>>
>>
>>