You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by "Sunny Chan, CLSA" <su...@clsa.com> on 2020/02/26 10:06:36 UTC
Configuring Ignite Thread Pool's Core thread count/max thread
count/etc
Hello,
We are running Ignite cluster on bare metal on a relatively high core count machine (4x10 cores 20 threads), and looking some of the thread pool initialization code:
(IgnitionEx.java)
sysExecSvc = new IgniteThreadPoolExecutor(
"sys",
cfg.getIgniteInstanceName(),
cfg.getSystemThreadPoolSize(),
cfg.getSystemThreadPoolSize(),
DFLT_THREAD_KEEP_ALIVE_TIME,
new LinkedBlockingQueue<Runnable>(),
GridIoPolicy.SYSTEM_POOL);
Notice that the core thread pool size is equals to the max thread pool settings, which is by default same as the number of CPU cores. And in our cases, we won't be reusing any threads until we have enough request coming in to fill 80 threads. Also, we might want to tune the thread keep alive time to improve thread reuse.
We would like to propose to change ignite so that users can configure the core thread pool size in these Ignite thread pools. What is the best way to expose these parameters for user to modify?
Would the ignite dev team prefer exposing individual core thread size and others (ie. cfg.get/setSystemThreadPoolCoreSize(), cfg.get/setSystemThreadPoolKeepAliveTime(), ..) or should we use a thread pool configuration object? (e.g. cfg.getSystemThreadPoolConfiguration(ThreadPoolConfiguration config) where ThreadPoolConfiguration has get and set methods for core thread pool size, etc)?
Sunny Chan
Senior Lead Engineer, Executive Services
D +852 2600 8907 | M +852 6386 1835 | T +852 2600 8888
5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
[:1. Social Media Icons:CLSA_Social Media Icons_linkedin.png]<https://hk.linkedin.com/company/clsa>[:1. Social Media Icons:CLSA_Social Media Icons_twitter.png]<https://twitter.com/clsainsights?lang=en>[:1. Social Media Icons:CLSA_Social Media Icons_youtube.png]<https://www.youtube.com/channel/UC0qWp_lLnOcRYmBlCNQgZKA>[:1. Social Media Icons:CLSA_Social Media Icons_facebook.png]<https://www.facebook.com/clsacommunity/>
clsa.com<https://www.clsa.com/>
Insights. Liquidity. Capital.
[CLSA_RGB]<https://www.clsa.com/member>
A CITIC Securities Company
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
Re: Configuring Ignite Thread Pool's Core thread count/max thread count/etc
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
My opinion is that we have what we have and I recommend using the API
provided with Apache Ignite.
Regards,
--
Ilya Kasnacheev
чт, 27 февр. 2020 г. в 13:57, Sunny Chan, CLSA <su...@clsa.com>:
> Just to make it clear, you prefer the option 1 in my suggestion? That is
> listed out all the potential configurable for thread pool for each thread
> pools in Ignite?
>
>
>
> I do see that we have setDataStorageConfiguration where the parameters for
> data storage is grouped together into a configuration object, similar to my
> proposal 2 –would that be better?
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* Thursday, February 27, 2020 6:05 PM
> *To:* Sunny Chan, CLSA <su...@clsa.com>
> *Cc:* dev <de...@ignite.apache.org>
> *Subject:* Re: Configuring Ignite Thread Pool's Core thread count/max
> thread count/etc
>
>
>
> Hello!
>
>
>
> My suggestion is to use
> org.apache.ignite.configuration.IgniteConfiguration and avoid introducing
> your own classes, etc. Cut out the middle man.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> чт, 27 февр. 2020 г. в 05:36, Sunny Chan, CLSA <su...@clsa.com>:
>
> Hi Ilya,
>
> For the IgniteConfiguration interface, do we prefer:
>
> 1) IgniteConfiguration.setSystemThreadPoolCoreSize(),
> IgniteConfiguration.setSystemThreadPoolTimeOut(),IgniteConfiguration.setPublicThreadPoolCoreSize(),
> etc.
> 2) IgniteConfiguration.setSystemThreadPool(ThreadPoolConfiguration),
> IgniteConfiguration.setThreadPoolCoreSize(ThreadPoolConfiguration) and then
> we have ThreadPoolConfiguration.setCoreSize(), etc
>
> Thanks.
>
> -----Original Message-----
> From: Ilya Kasnacheev <il...@gmail.com>
> Sent: Wednesday, February 26, 2020 6:10 PM
> To: dev <de...@ignite.apache.org>
> Subject: Re: Configuring Ignite Thread Pool's Core thread count/max thread
> count/etc
>
> Hello!
>
> We recommend exposing IgniteConfiguration for user to modify. You never
> know in advance what kind of tuning will be needed, and don't want to play
> chinese whispers game with your users.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 26 февр. 2020 г. в 13:07, Sunny Chan, CLSA <su...@clsa.com>:
>
> > Hello,
> >
> >
> >
> > We are running Ignite cluster on bare metal on a relatively high core
> > count machine (4x10 cores 20 threads), and looking some of the thread
> pool
> > initialization code:
> >
> >
> >
> > (IgnitionEx.java)
> >
> > sysExecSvc = *new* IgniteThreadPoolExecutor(
> >
> > "sys",
> >
> > cfg.getIgniteInstanceName(),
> >
> > cfg.getSystemThreadPoolSize(),
> >
> > cfg.getSystemThreadPoolSize(),
> >
> > *DFLT_THREAD_KEEP_ALIVE_TIME*,
> >
> > *new* LinkedBlockingQueue<Runnable>(),
> >
> > GridIoPolicy.*SYSTEM_POOL*);
> >
> >
> >
> > Notice that the core thread pool size is equals to the max thread pool
> > settings, which is by default same as the number of CPU cores. And in our
> > cases, we won’t be reusing any threads until we have enough request
> coming
> > in to fill 80 threads. Also, we might want to tune the thread keep alive
> > time to improve thread reuse.
> >
> >
> >
> > We would like to propose to change ignite so that users can configure the
> > core thread pool size in these Ignite thread pools. What is the best way
> to
> > expose these parameters for user to modify?
> >
> >
> >
> > Would the ignite dev team prefer exposing individual core thread size and
> > others (ie. cfg.get/setSystemThreadPoolCoreSize(),
> > cfg.get/setSystemThreadPoolKeepAliveTime(), ..) or should we use a thread
> > pool configuration object? (e.g.
> > cfg.getSystemThreadPoolConfiguration(ThreadPoolConfiguration config)
> where
> > ThreadPoolConfiguration has get and set methods for core thread pool
> size,
> > etc)?
> >
> >
> >
> > *Sunny Chan*
> >
> > *Senior Lead Engineer, Executive Services*
> >
> > D +852 2600 8907 | M +852 6386 1835 | T +852 2600 8888
> >
> > 5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
> >
> >
> >
> > [image: :1. Social Media Icons:CLSA_Social Media Icons_linkedin.png]
> > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__hk.linkedin.com_company_clsa&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=Yaq04TdZjYF0IXiRHXdbJS_fU_YNho-DXGuYjRNAfRM&e=
> >[image: :1. Social Media
> > Icons:CLSA_Social Media Icons_twitter.png]
> > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_clsainsights-3Flang-3Den&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=60dVf5GhkOQ8CsuUVPmzqzQ3KBWI7Jl66i-gAajMqhw&e=
> >[image: :1. Social Media
> > Icons:CLSA_Social Media Icons_youtube.png]
> > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_channel_UC0qWp-5FlLnOcRYmBlCNQgZKA&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=mDwljLI4KYkfMCIaiGVeezUNf52sbK8OLOj90ISnrOU&e=
> >[image: :1.
> > Social Media Icons:CLSA_Social Media Icons_facebook.png]
> > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_clsacommunity_&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=FeoNstP309ftIhDxtwesvrG7_7h4hiVHichs7zv3_IQ&e=
> >
> >
> >
> >
> > *clsa.com* <https://www.clsa.com/>
> >
> > *Insights. Liquidity. Capital. *
> >
> >
> >
> > [image: CLSA_RGB] <https://www.clsa.com/member>
> >
> >
> >
> > *A CITIC Securities Company*
> >
> >
> >
> > The content of this communication is intended for the recipient and is
> > subject to CLSA Legal and Regulatory Notices.
> > These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> > you upon request.
> > Please consider before printing. CLSA is ISO14001 certified and committed
> > to reducing its impact on the environment.
> >
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> Please consider before printing. CLSA is ISO14001 certified and committed
> to reducing its impact on the environment.
>
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> Please consider before printing. CLSA is ISO14001 certified and committed
> to reducing its impact on the environment.
>
RE: Configuring Ignite Thread Pool's Core thread count/max thread
count/etc
Posted by "Sunny Chan, CLSA" <su...@clsa.com>.
Just to make it clear, you prefer the option 1 in my suggestion? That is listed out all the potential configurable for thread pool for each thread pools in Ignite?
I do see that we have setDataStorageConfiguration where the parameters for data storage is grouped together into a configuration object, similar to my proposal 2 –would that be better?
From: Ilya Kasnacheev <il...@gmail.com>
Sent: Thursday, February 27, 2020 6:05 PM
To: Sunny Chan, CLSA <su...@clsa.com>
Cc: dev <de...@ignite.apache.org>
Subject: Re: Configuring Ignite Thread Pool's Core thread count/max thread count/etc
Hello!
My suggestion is to use org.apache.ignite.configuration.IgniteConfiguration and avoid introducing your own classes, etc. Cut out the middle man.
Regards,
--
Ilya Kasnacheev
чт, 27 февр. 2020 г. в 05:36, Sunny Chan, CLSA <su...@clsa.com>>:
Hi Ilya,
For the IgniteConfiguration interface, do we prefer:
1) IgniteConfiguration.setSystemThreadPoolCoreSize(), IgniteConfiguration.setSystemThreadPoolTimeOut(),IgniteConfiguration.setPublicThreadPoolCoreSize(), etc.
2) IgniteConfiguration.setSystemThreadPool(ThreadPoolConfiguration), IgniteConfiguration.setThreadPoolCoreSize(ThreadPoolConfiguration) and then we have ThreadPoolConfiguration.setCoreSize(), etc
Thanks.
-----Original Message-----
From: Ilya Kasnacheev <il...@gmail.com>>
Sent: Wednesday, February 26, 2020 6:10 PM
To: dev <de...@ignite.apache.org>>
Subject: Re: Configuring Ignite Thread Pool's Core thread count/max thread count/etc
Hello!
We recommend exposing IgniteConfiguration for user to modify. You never know in advance what kind of tuning will be needed, and don't want to play chinese whispers game with your users.
Regards,
--
Ilya Kasnacheev
ср, 26 февр. 2020 г. в 13:07, Sunny Chan, CLSA <su...@clsa.com>>:
> Hello,
>
>
>
> We are running Ignite cluster on bare metal on a relatively high core
> count machine (4x10 cores 20 threads), and looking some of the thread pool
> initialization code:
>
>
>
> (IgnitionEx.java)
>
> sysExecSvc = *new* IgniteThreadPoolExecutor(
>
> "sys",
>
> cfg.getIgniteInstanceName(),
>
> cfg.getSystemThreadPoolSize(),
>
> cfg.getSystemThreadPoolSize(),
>
> *DFLT_THREAD_KEEP_ALIVE_TIME*,
>
> *new* LinkedBlockingQueue<Runnable>(),
>
> GridIoPolicy.*SYSTEM_POOL*);
>
>
>
> Notice that the core thread pool size is equals to the max thread pool
> settings, which is by default same as the number of CPU cores. And in our
> cases, we won’t be reusing any threads until we have enough request coming
> in to fill 80 threads. Also, we might want to tune the thread keep alive
> time to improve thread reuse.
>
>
>
> We would like to propose to change ignite so that users can configure the
> core thread pool size in these Ignite thread pools. What is the best way to
> expose these parameters for user to modify?
>
>
>
> Would the ignite dev team prefer exposing individual core thread size and
> others (ie. cfg.get/setSystemThreadPoolCoreSize(),
> cfg.get/setSystemThreadPoolKeepAliveTime(), ..) or should we use a thread
> pool configuration object? (e.g.
> cfg.getSystemThreadPoolConfiguration(ThreadPoolConfiguration config) where
> ThreadPoolConfiguration has get and set methods for core thread pool size,
> etc)?
>
>
>
> *Sunny Chan*
>
> *Senior Lead Engineer, Executive Services*
>
> D +852 2600 8907 | M +852 6386 1835 | T +852 2600 8888
>
> 5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
>
>
>
> [image: :1. Social Media Icons:CLSA_Social Media Icons_linkedin.png]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__hk.linkedin.com_company_clsa&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=Yaq04TdZjYF0IXiRHXdbJS_fU_YNho-DXGuYjRNAfRM&e= >[image: :1. Social Media
> Icons:CLSA_Social Media Icons_twitter.png]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_clsainsights-3Flang-3Den&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=60dVf5GhkOQ8CsuUVPmzqzQ3KBWI7Jl66i-gAajMqhw&e= >[image: :1. Social Media
> Icons:CLSA_Social Media Icons_youtube.png]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_channel_UC0qWp-5FlLnOcRYmBlCNQgZKA&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=mDwljLI4KYkfMCIaiGVeezUNf52sbK8OLOj90ISnrOU&e= >[image: :1.
> Social Media Icons:CLSA_Social Media Icons_facebook.png]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_clsacommunity_&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=FeoNstP309ftIhDxtwesvrG7_7h4hiVHichs7zv3_IQ&e= >
>
>
>
> *clsa.com<http://clsa.com>* <https://www.clsa.com/>
>
> *Insights. Liquidity. Capital. *
>
>
>
> [image: CLSA_RGB] <https://www.clsa.com/member>
>
>
>
> *A CITIC Securities Company*
>
>
>
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> Please consider before printing. CLSA is ISO14001 certified and committed
> to reducing its impact on the environment.
>
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
Re: Configuring Ignite Thread Pool's Core thread count/max thread count/etc
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
My suggestion is to use org.apache.ignite.configuration.IgniteConfiguration
and avoid introducing your own classes, etc. Cut out the middle man.
Regards,
--
Ilya Kasnacheev
чт, 27 февр. 2020 г. в 05:36, Sunny Chan, CLSA <su...@clsa.com>:
> Hi Ilya,
>
> For the IgniteConfiguration interface, do we prefer:
>
> 1) IgniteConfiguration.setSystemThreadPoolCoreSize(),
> IgniteConfiguration.setSystemThreadPoolTimeOut(),IgniteConfiguration.setPublicThreadPoolCoreSize(),
> etc.
> 2) IgniteConfiguration.setSystemThreadPool(ThreadPoolConfiguration),
> IgniteConfiguration.setThreadPoolCoreSize(ThreadPoolConfiguration) and then
> we have ThreadPoolConfiguration.setCoreSize(), etc
>
> Thanks.
>
> -----Original Message-----
> From: Ilya Kasnacheev <il...@gmail.com>
> Sent: Wednesday, February 26, 2020 6:10 PM
> To: dev <de...@ignite.apache.org>
> Subject: Re: Configuring Ignite Thread Pool's Core thread count/max thread
> count/etc
>
> Hello!
>
> We recommend exposing IgniteConfiguration for user to modify. You never
> know in advance what kind of tuning will be needed, and don't want to play
> chinese whispers game with your users.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 26 февр. 2020 г. в 13:07, Sunny Chan, CLSA <su...@clsa.com>:
>
> > Hello,
> >
> >
> >
> > We are running Ignite cluster on bare metal on a relatively high core
> > count machine (4x10 cores 20 threads), and looking some of the thread
> pool
> > initialization code:
> >
> >
> >
> > (IgnitionEx.java)
> >
> > sysExecSvc = *new* IgniteThreadPoolExecutor(
> >
> > "sys",
> >
> > cfg.getIgniteInstanceName(),
> >
> > cfg.getSystemThreadPoolSize(),
> >
> > cfg.getSystemThreadPoolSize(),
> >
> > *DFLT_THREAD_KEEP_ALIVE_TIME*,
> >
> > *new* LinkedBlockingQueue<Runnable>(),
> >
> > GridIoPolicy.*SYSTEM_POOL*);
> >
> >
> >
> > Notice that the core thread pool size is equals to the max thread pool
> > settings, which is by default same as the number of CPU cores. And in our
> > cases, we won’t be reusing any threads until we have enough request
> coming
> > in to fill 80 threads. Also, we might want to tune the thread keep alive
> > time to improve thread reuse.
> >
> >
> >
> > We would like to propose to change ignite so that users can configure the
> > core thread pool size in these Ignite thread pools. What is the best way
> to
> > expose these parameters for user to modify?
> >
> >
> >
> > Would the ignite dev team prefer exposing individual core thread size and
> > others (ie. cfg.get/setSystemThreadPoolCoreSize(),
> > cfg.get/setSystemThreadPoolKeepAliveTime(), ..) or should we use a thread
> > pool configuration object? (e.g.
> > cfg.getSystemThreadPoolConfiguration(ThreadPoolConfiguration config)
> where
> > ThreadPoolConfiguration has get and set methods for core thread pool
> size,
> > etc)?
> >
> >
> >
> > *Sunny Chan*
> >
> > *Senior Lead Engineer, Executive Services*
> >
> > D +852 2600 8907 | M +852 6386 1835 | T +852 2600 8888
> >
> > 5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
> >
> >
> >
> > [image: :1. Social Media Icons:CLSA_Social Media Icons_linkedin.png]
> > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__hk.linkedin.com_company_clsa&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=Yaq04TdZjYF0IXiRHXdbJS_fU_YNho-DXGuYjRNAfRM&e=
> >[image: :1. Social Media
> > Icons:CLSA_Social Media Icons_twitter.png]
> > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_clsainsights-3Flang-3Den&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=60dVf5GhkOQ8CsuUVPmzqzQ3KBWI7Jl66i-gAajMqhw&e=
> >[image: :1. Social Media
> > Icons:CLSA_Social Media Icons_youtube.png]
> > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_channel_UC0qWp-5FlLnOcRYmBlCNQgZKA&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=mDwljLI4KYkfMCIaiGVeezUNf52sbK8OLOj90ISnrOU&e=
> >[image: :1.
> > Social Media Icons:CLSA_Social Media Icons_facebook.png]
> > <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_clsacommunity_&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=FeoNstP309ftIhDxtwesvrG7_7h4hiVHichs7zv3_IQ&e=
> >
> >
> >
> >
> > *clsa.com* <https://www.clsa.com/>
> >
> > *Insights. Liquidity. Capital. *
> >
> >
> >
> > [image: CLSA_RGB] <https://www.clsa.com/member>
> >
> >
> >
> > *A CITIC Securities Company*
> >
> >
> >
> > The content of this communication is intended for the recipient and is
> > subject to CLSA Legal and Regulatory Notices.
> > These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> > you upon request.
> > Please consider before printing. CLSA is ISO14001 certified and committed
> > to reducing its impact on the environment.
> >
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> Please consider before printing. CLSA is ISO14001 certified and committed
> to reducing its impact on the environment.
>
RE: Configuring Ignite Thread Pool's Core thread count/max thread
count/etc
Posted by "Sunny Chan, CLSA" <su...@clsa.com>.
Hi Ilya,
For the IgniteConfiguration interface, do we prefer:
1) IgniteConfiguration.setSystemThreadPoolCoreSize(), IgniteConfiguration.setSystemThreadPoolTimeOut(),IgniteConfiguration.setPublicThreadPoolCoreSize(), etc.
2) IgniteConfiguration.setSystemThreadPool(ThreadPoolConfiguration), IgniteConfiguration.setThreadPoolCoreSize(ThreadPoolConfiguration) and then we have ThreadPoolConfiguration.setCoreSize(), etc
Thanks.
-----Original Message-----
From: Ilya Kasnacheev <il...@gmail.com>
Sent: Wednesday, February 26, 2020 6:10 PM
To: dev <de...@ignite.apache.org>
Subject: Re: Configuring Ignite Thread Pool's Core thread count/max thread count/etc
Hello!
We recommend exposing IgniteConfiguration for user to modify. You never know in advance what kind of tuning will be needed, and don't want to play chinese whispers game with your users.
Regards,
--
Ilya Kasnacheev
ср, 26 февр. 2020 г. в 13:07, Sunny Chan, CLSA <su...@clsa.com>:
> Hello,
>
>
>
> We are running Ignite cluster on bare metal on a relatively high core
> count machine (4x10 cores 20 threads), and looking some of the thread pool
> initialization code:
>
>
>
> (IgnitionEx.java)
>
> sysExecSvc = *new* IgniteThreadPoolExecutor(
>
> "sys",
>
> cfg.getIgniteInstanceName(),
>
> cfg.getSystemThreadPoolSize(),
>
> cfg.getSystemThreadPoolSize(),
>
> *DFLT_THREAD_KEEP_ALIVE_TIME*,
>
> *new* LinkedBlockingQueue<Runnable>(),
>
> GridIoPolicy.*SYSTEM_POOL*);
>
>
>
> Notice that the core thread pool size is equals to the max thread pool
> settings, which is by default same as the number of CPU cores. And in our
> cases, we won’t be reusing any threads until we have enough request coming
> in to fill 80 threads. Also, we might want to tune the thread keep alive
> time to improve thread reuse.
>
>
>
> We would like to propose to change ignite so that users can configure the
> core thread pool size in these Ignite thread pools. What is the best way to
> expose these parameters for user to modify?
>
>
>
> Would the ignite dev team prefer exposing individual core thread size and
> others (ie. cfg.get/setSystemThreadPoolCoreSize(),
> cfg.get/setSystemThreadPoolKeepAliveTime(), ..) or should we use a thread
> pool configuration object? (e.g.
> cfg.getSystemThreadPoolConfiguration(ThreadPoolConfiguration config) where
> ThreadPoolConfiguration has get and set methods for core thread pool size,
> etc)?
>
>
>
> *Sunny Chan*
>
> *Senior Lead Engineer, Executive Services*
>
> D +852 2600 8907 | M +852 6386 1835 | T +852 2600 8888
>
> 5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
>
>
>
> [image: :1. Social Media Icons:CLSA_Social Media Icons_linkedin.png]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__hk.linkedin.com_company_clsa&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=Yaq04TdZjYF0IXiRHXdbJS_fU_YNho-DXGuYjRNAfRM&e= >[image: :1. Social Media
> Icons:CLSA_Social Media Icons_twitter.png]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_clsainsights-3Flang-3Den&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=60dVf5GhkOQ8CsuUVPmzqzQ3KBWI7Jl66i-gAajMqhw&e= >[image: :1. Social Media
> Icons:CLSA_Social Media Icons_youtube.png]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.youtube.com_channel_UC0qWp-5FlLnOcRYmBlCNQgZKA&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=mDwljLI4KYkfMCIaiGVeezUNf52sbK8OLOj90ISnrOU&e= >[image: :1.
> Social Media Icons:CLSA_Social Media Icons_facebook.png]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_clsacommunity_&d=DwIFaQ&c=lxzXOFU02467FL7HOPRqCw&r=9dPMIMeLdcqo3dX29zZZ4USXrC7gp_0p-Fvzzp2WzgU&m=cblNaj4j3LVM7Djyn0ffKyR3bThUEDp7iqqm5mRgkys&s=FeoNstP309ftIhDxtwesvrG7_7h4hiVHichs7zv3_IQ&e= >
>
>
>
> *clsa.com* <https://www.clsa.com/>
>
> *Insights. Liquidity. Capital. *
>
>
>
> [image: CLSA_RGB] <https://www.clsa.com/member>
>
>
>
> *A CITIC Securities Company*
>
>
>
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> Please consider before printing. CLSA is ISO14001 certified and committed
> to reducing its impact on the environment.
>
The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request.
Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.
Re: Configuring Ignite Thread Pool's Core thread count/max thread count/etc
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
We recommend exposing IgniteConfiguration for user to modify. You never
know in advance what kind of tuning will be needed, and don't want to play
chinese whispers game with your users.
Regards,
--
Ilya Kasnacheev
ср, 26 февр. 2020 г. в 13:07, Sunny Chan, CLSA <su...@clsa.com>:
> Hello,
>
>
>
> We are running Ignite cluster on bare metal on a relatively high core
> count machine (4x10 cores 20 threads), and looking some of the thread pool
> initialization code:
>
>
>
> (IgnitionEx.java)
>
> sysExecSvc = *new* IgniteThreadPoolExecutor(
>
> "sys",
>
> cfg.getIgniteInstanceName(),
>
> cfg.getSystemThreadPoolSize(),
>
> cfg.getSystemThreadPoolSize(),
>
> *DFLT_THREAD_KEEP_ALIVE_TIME*,
>
> *new* LinkedBlockingQueue<Runnable>(),
>
> GridIoPolicy.*SYSTEM_POOL*);
>
>
>
> Notice that the core thread pool size is equals to the max thread pool
> settings, which is by default same as the number of CPU cores. And in our
> cases, we won’t be reusing any threads until we have enough request coming
> in to fill 80 threads. Also, we might want to tune the thread keep alive
> time to improve thread reuse.
>
>
>
> We would like to propose to change ignite so that users can configure the
> core thread pool size in these Ignite thread pools. What is the best way to
> expose these parameters for user to modify?
>
>
>
> Would the ignite dev team prefer exposing individual core thread size and
> others (ie. cfg.get/setSystemThreadPoolCoreSize(),
> cfg.get/setSystemThreadPoolKeepAliveTime(), ..) or should we use a thread
> pool configuration object? (e.g.
> cfg.getSystemThreadPoolConfiguration(ThreadPoolConfiguration config) where
> ThreadPoolConfiguration has get and set methods for core thread pool size,
> etc)?
>
>
>
> *Sunny Chan*
>
> *Senior Lead Engineer, Executive Services*
>
> D +852 2600 8907 | M +852 6386 1835 | T +852 2600 8888
>
> 5/F, One Island East, 18 Westlands Road, Island East, Hong Kong
>
>
>
> [image: :1. Social Media Icons:CLSA_Social Media Icons_linkedin.png]
> <https://hk.linkedin.com/company/clsa>[image: :1. Social Media
> Icons:CLSA_Social Media Icons_twitter.png]
> <https://twitter.com/clsainsights?lang=en>[image: :1. Social Media
> Icons:CLSA_Social Media Icons_youtube.png]
> <https://www.youtube.com/channel/UC0qWp_lLnOcRYmBlCNQgZKA>[image: :1.
> Social Media Icons:CLSA_Social Media Icons_facebook.png]
> <https://www.facebook.com/clsacommunity/>
>
>
>
> *clsa.com* <https://www.clsa.com/>
>
> *Insights. Liquidity. Capital. *
>
>
>
> [image: CLSA_RGB] <https://www.clsa.com/member>
>
>
>
> *A CITIC Securities Company*
>
>
>
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> Please consider before printing. CLSA is ISO14001 certified and committed
> to reducing its impact on the environment.
>