You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Vladimir Ozerov <vo...@gridgain.com> on 2016/12/15 09:22:01 UTC
Striped pool configuration
Folks,
As I see important performance improvements were merged to the *master *branch.
One of them is "striped pool" which can be used instead of current system
pool. I think it's configuration is not very good at the moment:
int IgniteConfiguration.setStripedPoolSize();
Problems:
1) It has non-zero default meaning that it will be used by default. It is
ok, but if user previously configured system pool size with
"IgniteConfiguration.setSystemPoolSize()", then his config will be ignored
silently. It looks wrong to me to have two configurable sizes for a single
pool.
2) What if decide to add striped pool optimization for other pools in
future, e.g. for public pool? How are we going to name configuration
properties?
I think we should do the following:
1) Remove *"int stripedPoolSize"* property.
2) Add *"boolean systemThreadPoolStriped"* property, set it to *true *by
default.
3) Rely on existing *"int systemThreadPoolSize"* property to get number of
threads.
Thoughts?
Vladimir.
Re: Striped pool configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Mon, Dec 26, 2016 at 2:08 AM, Yakov Zhdanov <yz...@apache.org> wrote:
> Dmitry, for example, running queries from old fashioned pool seems to
> perform better, but short-term cache updates perform better when run on
> striped pool.
>
Do we know why? It seems like a mystery to me.
Re: Striped pool configuration
Posted by Yakov Zhdanov <yz...@apache.org>.
Dmitry, for example, running queries from old fashioned pool seems to
perform better, but short-term cache updates perform better when run on
striped pool.
--Yakov
Re: Striped pool configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Yakov, I understand that striped pools may provide better performance. Can
you describe when it will have actually perform worse than a regular pool?
On Sat, Dec 17, 2016 at 12:44 AM, Yakov Zhdanov <yz...@apache.org> wrote:
> 3. if this is cache message and its target partition is determined it goes
> to particular stripe (put requests, lock requests, etc)
>
> If partition is not determined cache message goes to random stripe (update
> response, prepare request, commit request, etc) - btw this needs to be
> fixed and we need to spread partition info to these messages where
> applicable, eg finish txs on per partition basis - this way we eliminate
> contention on cache entries and partitions.
>
> Query requests go to system pool since they may iterate over entire cache.
>
> For some messages we still need to use sys pool - sometimes Ignite
> fallbacks to sync waiting in internal threads. This is obviously an issue
> that needs to be fixed. I have list of such places to fix.
>
> --Yakov
>
Re: Striped pool configuration
Posted by Yakov Zhdanov <yz...@apache.org>.
3. if this is cache message and its target partition is determined it goes
to particular stripe (put requests, lock requests, etc)
If partition is not determined cache message goes to random stripe (update
response, prepare request, commit request, etc) - btw this needs to be
fixed and we need to spread partition info to these messages where
applicable, eg finish txs on per partition basis - this way we eliminate
contention on cache entries and partitions.
Query requests go to system pool since they may iterate over entire cache.
For some messages we still need to use sys pool - sometimes Ignite
fallbacks to sync waiting in internal threads. This is obviously an issue
that needs to be fixed. I have list of such places to fix.
--Yakov
Re: Striped pool configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I am also confused. Is the striped-pool something that is attached to NIO
threads? Can someone describe the sequence of events here? For example,
1. Message is received
2. Message is deserialized by a NIO thread
3. ???
Thanks,
D.
On Fri, Dec 16, 2016 at 9:37 AM, Denis Magda <dm...@apache.org> wrote:
> Well, looks like I’m a bit confused with what is meant under the “striped
> pool”.
>
> Vladimir, referring to your explanation
>
> > Yes, "striped pool" is a thread pool where every thread operate on a
> single
> > queue.
>
> please answer on the following:
>
> - Does a queue belong to a specific cache partition? In other words, every
> partition will have its own processing thread which is similar to what HZ
> and VoltDB do.
>
> - If the answer to the question above is negative then how do we decide
> where a specific cache message should fall for processing?
>
>
> —
> Denis
>
> > On Dec 16, 2016, at 3:39 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
> >
> > Andrey,
> >
> > Yes, "striped pool" is a thread pool where every thread operate on a
> single
> > queue. This is the only similarity with FJP, as we do not perform real
> > fork-join in our pools. Having said that, I do not see why we might want
> to
> > compare our pool with FJP.
> >
> > As per throughput - it is improved because instead of accessing a
> > single *BlockingQueue
> > *based on *ReentrantLock*, every thread has separate queue. And it is
> > designed in a way, that under load NIO threads usually enqueue tasks and
> > worker threads dequeue tasks without blocking, and even without
> CAS-loops,
> > thanks to MPSC semantics.
> >
> > Vladimir.
> >
> > On Fri, Dec 16, 2016 at 2:21 PM, Andrey Mashenkov <
> > andrey.mashenkov@gmail.com> wrote:
> >
> >> Vladimir,
> >>
> >> As I understand "striped pool" is a array of single-threaded pools. Do
> you
> >> have an understanding why throughput increased up to 40%?
> >> It looks like it is due to every thread has its own task queue.
> >>
> >> As far as I know, there is ForkJoinPool in JDK, FJP implements
> >> ExecutorService interface, has striped tasks queue, has task-stealing
> >> mechanics.
> >>
> >> Can we run Ignite performance tests b\w using "striped pool" and FJP?
> >>
> >> On Fri, Dec 16, 2016 at 11:17 AM, Vladimir Ozerov <vozerov@gridgain.com
> >
> >> wrote:
> >>
> >>> Folks,
> >>> Can we move all discussions aside of pool config to separate threads,
> >>> please? :-)
> >>>
> >>> Dima,
> >>> I heard your concern about configuration complexity. But I do not see
> any
> >>> problem at all. Whether to use striped pool or not is a matter of
> >>> fine-grained tuning. We will set sensible defaults (i.e. enabled
> striped
> >>> pool), so 99% of users will never know that a concept of "stiped pool"
> >> even
> >>> exists.
> >>>
> >>> Striped and non-striped approaches have their own pros and cons.
> >>>
> >>> Striped:
> >>> + Better overall throughput (up to +40% compared to non-striped);
> >>> - Less predictable latency - user can have bad numbers even on moderate
> >>> load (e.g. consider updates on a large affinity co-located object
> graph).
> >>> Benchmarks demonstrate this clearly.
> >>> - Higher memory footprint, because in striped mode we pre-start all the
> >>> threads. On high-end machines it may end up in wasting of hundrends of
> >>> megabytes of RAM.
> >>> - As a result of previous point, it is not well-suited for clients,
> which
> >>> may require small memory footprint and low startup time.
> >>>
> >>> Non-striped:
> >>> - Worse throughput due to very high contention on BlockingQueue.
> >>> + No waste on idle threads.
> >>>
> >>> I would propose the following final design for this:
> >>>
> >>> - Introduce *"boolean systemThreadPoolStriped"* property;
> >>> - Set it to *"true"* for servers by default;
> >>> - Set it to *"false"* for clients by default.
> >>>
> >>> Yakov,
> >>> I still do not get your point about (system + public) pools approach.
> >>>
> >>> Vladimir.
> >>>
> >>> On Fri, Dec 16, 2016 at 9:29 AM, Yakov Zhdanov <yz...@apache.org>
> >>> wrote:
> >>>
> >>>>> In my view, we should hide most of such configuration intricacies
> from
> >>>> users,
> >>>> and select an appropriate thread pool for each task ourselves. Will
> >> this
> >>> be
> >>>> possible?
> >>>>
> >>>> We already do this. We take decision internally on where to route the
> >>>> request.
> >>>>
> >>>> SoE, my answers below.
> >>>>
> >>>>> How did the striped pool show itself for transactional updates that
> >>>> involve 2 phase commit? Any benefits or degradation?
> >>>>
> >>>> Even better than for atomic - up to ~50% improvement.
> >>>>
> >>>>> Here we should decide what’s more important for us - throughout or
> >>>> latency. If to execute SQL queries in the striped pool using multiple
> >>>> partitioned threads then, for sure, it will affect latency of other
> >>>> operations that are sitting in the queue waiting for their time to be
> >>>> processed but, on the other hand, the overall throughput of the
> >> platform
> >>>> should be improved because the operations will be less halted all the
> >>> time
> >>>> by synchronization needs.
> >>>>
> >>>>> VoltDB decided to go for with the throughput while our current
> >>>> architecture is mostly latency based.
> >>>>
> >>>> What do you mean by "using several partition threads"? I am pretty
> sure
> >>>> that if we do as you suggest we will have degradation here instead of
> >>>> boost:
> >>>>
> >>>> sql-query-put
> >>>> after: 77,344.83
> >>>> before: 53,578.78
> >>>> delta: 30.73%
> >>>>
> >>>> sql-query-put-offheap
> >>>> after 32,212.30
> >>>> before 25,322.43
> >>>> delta 21.39%
> >>>>
> >>>> --Yakov
> >>>
> >>
> >>
> >>
> >> --
> >> С уважением,
> >> Машенков Андрей Владимирович
> >> Тел. +7-921-932-61-82
> >>
> >> Best regards,
> >> Andrey V. Mashenkov
> >> Cerr: +7-921-932-61-82
> >>
>
>
Re: Striped pool configuration
Posted by Denis Magda <dm...@apache.org>.
Well, looks like I’m a bit confused with what is meant under the “striped pool”.
Vladimir, referring to your explanation
> Yes, "striped pool" is a thread pool where every thread operate on a single
> queue.
please answer on the following:
- Does a queue belong to a specific cache partition? In other words, every partition will have its own processing thread which is similar to what HZ and VoltDB do.
- If the answer to the question above is negative then how do we decide where a specific cache message should fall for processing?
—
Denis
> On Dec 16, 2016, at 3:39 AM, Vladimir Ozerov <vo...@gridgain.com> wrote:
>
> Andrey,
>
> Yes, "striped pool" is a thread pool where every thread operate on a single
> queue. This is the only similarity with FJP, as we do not perform real
> fork-join in our pools. Having said that, I do not see why we might want to
> compare our pool with FJP.
>
> As per throughput - it is improved because instead of accessing a
> single *BlockingQueue
> *based on *ReentrantLock*, every thread has separate queue. And it is
> designed in a way, that under load NIO threads usually enqueue tasks and
> worker threads dequeue tasks without blocking, and even without CAS-loops,
> thanks to MPSC semantics.
>
> Vladimir.
>
> On Fri, Dec 16, 2016 at 2:21 PM, Andrey Mashenkov <
> andrey.mashenkov@gmail.com> wrote:
>
>> Vladimir,
>>
>> As I understand "striped pool" is a array of single-threaded pools. Do you
>> have an understanding why throughput increased up to 40%?
>> It looks like it is due to every thread has its own task queue.
>>
>> As far as I know, there is ForkJoinPool in JDK, FJP implements
>> ExecutorService interface, has striped tasks queue, has task-stealing
>> mechanics.
>>
>> Can we run Ignite performance tests b\w using "striped pool" and FJP?
>>
>> On Fri, Dec 16, 2016 at 11:17 AM, Vladimir Ozerov <vo...@gridgain.com>
>> wrote:
>>
>>> Folks,
>>> Can we move all discussions aside of pool config to separate threads,
>>> please? :-)
>>>
>>> Dima,
>>> I heard your concern about configuration complexity. But I do not see any
>>> problem at all. Whether to use striped pool or not is a matter of
>>> fine-grained tuning. We will set sensible defaults (i.e. enabled striped
>>> pool), so 99% of users will never know that a concept of "stiped pool"
>> even
>>> exists.
>>>
>>> Striped and non-striped approaches have their own pros and cons.
>>>
>>> Striped:
>>> + Better overall throughput (up to +40% compared to non-striped);
>>> - Less predictable latency - user can have bad numbers even on moderate
>>> load (e.g. consider updates on a large affinity co-located object graph).
>>> Benchmarks demonstrate this clearly.
>>> - Higher memory footprint, because in striped mode we pre-start all the
>>> threads. On high-end machines it may end up in wasting of hundrends of
>>> megabytes of RAM.
>>> - As a result of previous point, it is not well-suited for clients, which
>>> may require small memory footprint and low startup time.
>>>
>>> Non-striped:
>>> - Worse throughput due to very high contention on BlockingQueue.
>>> + No waste on idle threads.
>>>
>>> I would propose the following final design for this:
>>>
>>> - Introduce *"boolean systemThreadPoolStriped"* property;
>>> - Set it to *"true"* for servers by default;
>>> - Set it to *"false"* for clients by default.
>>>
>>> Yakov,
>>> I still do not get your point about (system + public) pools approach.
>>>
>>> Vladimir.
>>>
>>> On Fri, Dec 16, 2016 at 9:29 AM, Yakov Zhdanov <yz...@apache.org>
>>> wrote:
>>>
>>>>> In my view, we should hide most of such configuration intricacies from
>>>> users,
>>>> and select an appropriate thread pool for each task ourselves. Will
>> this
>>> be
>>>> possible?
>>>>
>>>> We already do this. We take decision internally on where to route the
>>>> request.
>>>>
>>>> SoE, my answers below.
>>>>
>>>>> How did the striped pool show itself for transactional updates that
>>>> involve 2 phase commit? Any benefits or degradation?
>>>>
>>>> Even better than for atomic - up to ~50% improvement.
>>>>
>>>>> Here we should decide what’s more important for us - throughout or
>>>> latency. If to execute SQL queries in the striped pool using multiple
>>>> partitioned threads then, for sure, it will affect latency of other
>>>> operations that are sitting in the queue waiting for their time to be
>>>> processed but, on the other hand, the overall throughput of the
>> platform
>>>> should be improved because the operations will be less halted all the
>>> time
>>>> by synchronization needs.
>>>>
>>>>> VoltDB decided to go for with the throughput while our current
>>>> architecture is mostly latency based.
>>>>
>>>> What do you mean by "using several partition threads"? I am pretty sure
>>>> that if we do as you suggest we will have degradation here instead of
>>>> boost:
>>>>
>>>> sql-query-put
>>>> after: 77,344.83
>>>> before: 53,578.78
>>>> delta: 30.73%
>>>>
>>>> sql-query-put-offheap
>>>> after 32,212.30
>>>> before 25,322.43
>>>> delta 21.39%
>>>>
>>>> --Yakov
>>>
>>
>>
>>
>> --
>> С уважением,
>> Машенков Андрей Владимирович
>> Тел. +7-921-932-61-82
>>
>> Best regards,
>> Andrey V. Mashenkov
>> Cerr: +7-921-932-61-82
>>
Re: Striped pool configuration
Posted by Andrey Mashenkov <an...@gmail.com>.
Vladimir,
Got it. "Stripe pool" is really useful stuff for intensive message
processing.
However, I doubt it is usable for long-life jobs, e.g. SQL queries and some
compute jobs without job-stealing feature.
Also, having all theads pre-started by default can be an issue.
My point is FJP has no lacks that was disscussed above, and we could check
if we get benefits by using FJP in cases when "stripe pool" is seems not to
be suitable.
Of course we do not need harness that FJP provide for fork-join feature,
but some ideas can be adopted.
I remember another one, if threads in pool is sleeping and user put a job,
FJP has dedicated thread that make waking up other threads. So, pool starts
much more faster than any other pool.
On Fri, Dec 16, 2016 at 2:39 PM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> Andrey,
>
> Yes, "striped pool" is a thread pool where every thread operate on a single
> queue. This is the only similarity with FJP, as we do not perform real
> fork-join in our pools. Having said that, I do not see why we might want to
> compare our pool with FJP.
>
> As per throughput - it is improved because instead of accessing a
> single *BlockingQueue
> *based on *ReentrantLock*, every thread has separate queue. And it is
> designed in a way, that under load NIO threads usually enqueue tasks and
> worker threads dequeue tasks without blocking, and even without CAS-loops,
> thanks to MPSC semantics.
>
> Vladimir.
>
> On Fri, Dec 16, 2016 at 2:21 PM, Andrey Mashenkov <
> andrey.mashenkov@gmail.com> wrote:
>
> > Vladimir,
> >
> > As I understand "striped pool" is a array of single-threaded pools. Do
> you
> > have an understanding why throughput increased up to 40%?
> > It looks like it is due to every thread has its own task queue.
> >
> > As far as I know, there is ForkJoinPool in JDK, FJP implements
> > ExecutorService interface, has striped tasks queue, has task-stealing
> > mechanics.
> >
> > Can we run Ignite performance tests b\w using "striped pool" and FJP?
> >
> > On Fri, Dec 16, 2016 at 11:17 AM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> > > Folks,
> > > Can we move all discussions aside of pool config to separate threads,
> > > please? :-)
> > >
> > > Dima,
> > > I heard your concern about configuration complexity. But I do not see
> any
> > > problem at all. Whether to use striped pool or not is a matter of
> > > fine-grained tuning. We will set sensible defaults (i.e. enabled
> striped
> > > pool), so 99% of users will never know that a concept of "stiped pool"
> > even
> > > exists.
> > >
> > > Striped and non-striped approaches have their own pros and cons.
> > >
> > > Striped:
> > > + Better overall throughput (up to +40% compared to non-striped);
> > > - Less predictable latency - user can have bad numbers even on moderate
> > > load (e.g. consider updates on a large affinity co-located object
> graph).
> > > Benchmarks demonstrate this clearly.
> > > - Higher memory footprint, because in striped mode we pre-start all the
> > > threads. On high-end machines it may end up in wasting of hundrends of
> > > megabytes of RAM.
> > > - As a result of previous point, it is not well-suited for clients,
> which
> > > may require small memory footprint and low startup time.
> > >
> > > Non-striped:
> > > - Worse throughput due to very high contention on BlockingQueue.
> > > + No waste on idle threads.
> > >
> > > I would propose the following final design for this:
> > >
> > > - Introduce *"boolean systemThreadPoolStriped"* property;
> > > - Set it to *"true"* for servers by default;
> > > - Set it to *"false"* for clients by default.
> > >
> > > Yakov,
> > > I still do not get your point about (system + public) pools approach.
> > >
> > > Vladimir.
> > >
> > > On Fri, Dec 16, 2016 at 9:29 AM, Yakov Zhdanov <yz...@apache.org>
> > > wrote:
> > >
> > > > >In my view, we should hide most of such configuration intricacies
> from
> > > > users,
> > > > and select an appropriate thread pool for each task ourselves. Will
> > this
> > > be
> > > > possible?
> > > >
> > > > We already do this. We take decision internally on where to route the
> > > > request.
> > > >
> > > > SoE, my answers below.
> > > >
> > > > >How did the striped pool show itself for transactional updates that
> > > > involve 2 phase commit? Any benefits or degradation?
> > > >
> > > > Even better than for atomic - up to ~50% improvement.
> > > >
> > > > >Here we should decide what’s more important for us - throughout or
> > > > latency. If to execute SQL queries in the striped pool using multiple
> > > > partitioned threads then, for sure, it will affect latency of other
> > > > operations that are sitting in the queue waiting for their time to be
> > > > processed but, on the other hand, the overall throughput of the
> > platform
> > > > should be improved because the operations will be less halted all the
> > > time
> > > > by synchronization needs.
> > > >
> > > > >VoltDB decided to go for with the throughput while our current
> > > > architecture is mostly latency based.
> > > >
> > > > What do you mean by "using several partition threads"? I am pretty
> sure
> > > > that if we do as you suggest we will have degradation here instead of
> > > > boost:
> > > >
> > > > sql-query-put
> > > > after: 77,344.83
> > > > before: 53,578.78
> > > > delta: 30.73%
> > > >
> > > > sql-query-put-offheap
> > > > after 32,212.30
> > > > before 25,322.43
> > > > delta 21.39%
> > > >
> > > > --Yakov
> > >
> >
> >
> >
> > --
> > С уважением,
> > Машенков Андрей Владимирович
> > Тел. +7-921-932-61-82
> >
> > Best regards,
> > Andrey V. Mashenkov
> > Cerr: +7-921-932-61-82
> >
>
--
С уважением,
Машенков Андрей Владимирович
Тел. +7-921-932-61-82
Best regards,
Andrey V. Mashenkov
Cerr: +7-921-932-61-82
Re: Striped pool configuration
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Andrey,
Yes, "striped pool" is a thread pool where every thread operate on a single
queue. This is the only similarity with FJP, as we do not perform real
fork-join in our pools. Having said that, I do not see why we might want to
compare our pool with FJP.
As per throughput - it is improved because instead of accessing a
single *BlockingQueue
*based on *ReentrantLock*, every thread has separate queue. And it is
designed in a way, that under load NIO threads usually enqueue tasks and
worker threads dequeue tasks without blocking, and even without CAS-loops,
thanks to MPSC semantics.
Vladimir.
On Fri, Dec 16, 2016 at 2:21 PM, Andrey Mashenkov <
andrey.mashenkov@gmail.com> wrote:
> Vladimir,
>
> As I understand "striped pool" is a array of single-threaded pools. Do you
> have an understanding why throughput increased up to 40%?
> It looks like it is due to every thread has its own task queue.
>
> As far as I know, there is ForkJoinPool in JDK, FJP implements
> ExecutorService interface, has striped tasks queue, has task-stealing
> mechanics.
>
> Can we run Ignite performance tests b\w using "striped pool" and FJP?
>
> On Fri, Dec 16, 2016 at 11:17 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > Folks,
> > Can we move all discussions aside of pool config to separate threads,
> > please? :-)
> >
> > Dima,
> > I heard your concern about configuration complexity. But I do not see any
> > problem at all. Whether to use striped pool or not is a matter of
> > fine-grained tuning. We will set sensible defaults (i.e. enabled striped
> > pool), so 99% of users will never know that a concept of "stiped pool"
> even
> > exists.
> >
> > Striped and non-striped approaches have their own pros and cons.
> >
> > Striped:
> > + Better overall throughput (up to +40% compared to non-striped);
> > - Less predictable latency - user can have bad numbers even on moderate
> > load (e.g. consider updates on a large affinity co-located object graph).
> > Benchmarks demonstrate this clearly.
> > - Higher memory footprint, because in striped mode we pre-start all the
> > threads. On high-end machines it may end up in wasting of hundrends of
> > megabytes of RAM.
> > - As a result of previous point, it is not well-suited for clients, which
> > may require small memory footprint and low startup time.
> >
> > Non-striped:
> > - Worse throughput due to very high contention on BlockingQueue.
> > + No waste on idle threads.
> >
> > I would propose the following final design for this:
> >
> > - Introduce *"boolean systemThreadPoolStriped"* property;
> > - Set it to *"true"* for servers by default;
> > - Set it to *"false"* for clients by default.
> >
> > Yakov,
> > I still do not get your point about (system + public) pools approach.
> >
> > Vladimir.
> >
> > On Fri, Dec 16, 2016 at 9:29 AM, Yakov Zhdanov <yz...@apache.org>
> > wrote:
> >
> > > >In my view, we should hide most of such configuration intricacies from
> > > users,
> > > and select an appropriate thread pool for each task ourselves. Will
> this
> > be
> > > possible?
> > >
> > > We already do this. We take decision internally on where to route the
> > > request.
> > >
> > > SoE, my answers below.
> > >
> > > >How did the striped pool show itself for transactional updates that
> > > involve 2 phase commit? Any benefits or degradation?
> > >
> > > Even better than for atomic - up to ~50% improvement.
> > >
> > > >Here we should decide what’s more important for us - throughout or
> > > latency. If to execute SQL queries in the striped pool using multiple
> > > partitioned threads then, for sure, it will affect latency of other
> > > operations that are sitting in the queue waiting for their time to be
> > > processed but, on the other hand, the overall throughput of the
> platform
> > > should be improved because the operations will be less halted all the
> > time
> > > by synchronization needs.
> > >
> > > >VoltDB decided to go for with the throughput while our current
> > > architecture is mostly latency based.
> > >
> > > What do you mean by "using several partition threads"? I am pretty sure
> > > that if we do as you suggest we will have degradation here instead of
> > > boost:
> > >
> > > sql-query-put
> > > after: 77,344.83
> > > before: 53,578.78
> > > delta: 30.73%
> > >
> > > sql-query-put-offheap
> > > after 32,212.30
> > > before 25,322.43
> > > delta 21.39%
> > >
> > > --Yakov
> >
>
>
>
> --
> С уважением,
> Машенков Андрей Владимирович
> Тел. +7-921-932-61-82
>
> Best regards,
> Andrey V. Mashenkov
> Cerr: +7-921-932-61-82
>
Re: Striped pool configuration
Posted by Andrey Mashenkov <an...@gmail.com>.
Vladimir,
As I understand "striped pool" is a array of single-threaded pools. Do you
have an understanding why throughput increased up to 40%?
It looks like it is due to every thread has its own task queue.
As far as I know, there is ForkJoinPool in JDK, FJP implements
ExecutorService interface, has striped tasks queue, has task-stealing
mechanics.
Can we run Ignite performance tests b\w using "striped pool" and FJP?
On Fri, Dec 16, 2016 at 11:17 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> Folks,
> Can we move all discussions aside of pool config to separate threads,
> please? :-)
>
> Dima,
> I heard your concern about configuration complexity. But I do not see any
> problem at all. Whether to use striped pool or not is a matter of
> fine-grained tuning. We will set sensible defaults (i.e. enabled striped
> pool), so 99% of users will never know that a concept of "stiped pool" even
> exists.
>
> Striped and non-striped approaches have their own pros and cons.
>
> Striped:
> + Better overall throughput (up to +40% compared to non-striped);
> - Less predictable latency - user can have bad numbers even on moderate
> load (e.g. consider updates on a large affinity co-located object graph).
> Benchmarks demonstrate this clearly.
> - Higher memory footprint, because in striped mode we pre-start all the
> threads. On high-end machines it may end up in wasting of hundrends of
> megabytes of RAM.
> - As a result of previous point, it is not well-suited for clients, which
> may require small memory footprint and low startup time.
>
> Non-striped:
> - Worse throughput due to very high contention on BlockingQueue.
> + No waste on idle threads.
>
> I would propose the following final design for this:
>
> - Introduce *"boolean systemThreadPoolStriped"* property;
> - Set it to *"true"* for servers by default;
> - Set it to *"false"* for clients by default.
>
> Yakov,
> I still do not get your point about (system + public) pools approach.
>
> Vladimir.
>
> On Fri, Dec 16, 2016 at 9:29 AM, Yakov Zhdanov <yz...@apache.org>
> wrote:
>
> > >In my view, we should hide most of such configuration intricacies from
> > users,
> > and select an appropriate thread pool for each task ourselves. Will this
> be
> > possible?
> >
> > We already do this. We take decision internally on where to route the
> > request.
> >
> > SoE, my answers below.
> >
> > >How did the striped pool show itself for transactional updates that
> > involve 2 phase commit? Any benefits or degradation?
> >
> > Even better than for atomic - up to ~50% improvement.
> >
> > >Here we should decide what’s more important for us - throughout or
> > latency. If to execute SQL queries in the striped pool using multiple
> > partitioned threads then, for sure, it will affect latency of other
> > operations that are sitting in the queue waiting for their time to be
> > processed but, on the other hand, the overall throughput of the platform
> > should be improved because the operations will be less halted all the
> time
> > by synchronization needs.
> >
> > >VoltDB decided to go for with the throughput while our current
> > architecture is mostly latency based.
> >
> > What do you mean by "using several partition threads"? I am pretty sure
> > that if we do as you suggest we will have degradation here instead of
> > boost:
> >
> > sql-query-put
> > after: 77,344.83
> > before: 53,578.78
> > delta: 30.73%
> >
> > sql-query-put-offheap
> > after 32,212.30
> > before 25,322.43
> > delta 21.39%
> >
> > --Yakov
>
--
С уважением,
Машенков Андрей Владимирович
Тел. +7-921-932-61-82
Best regards,
Andrey V. Mashenkov
Cerr: +7-921-932-61-82
Re: Striped pool configuration
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Folks,
Can we move all discussions aside of pool config to separate threads,
please? :-)
Dima,
I heard your concern about configuration complexity. But I do not see any
problem at all. Whether to use striped pool or not is a matter of
fine-grained tuning. We will set sensible defaults (i.e. enabled striped
pool), so 99% of users will never know that a concept of "stiped pool" even
exists.
Striped and non-striped approaches have their own pros and cons.
Striped:
+ Better overall throughput (up to +40% compared to non-striped);
- Less predictable latency - user can have bad numbers even on moderate
load (e.g. consider updates on a large affinity co-located object graph).
Benchmarks demonstrate this clearly.
- Higher memory footprint, because in striped mode we pre-start all the
threads. On high-end machines it may end up in wasting of hundrends of
megabytes of RAM.
- As a result of previous point, it is not well-suited for clients, which
may require small memory footprint and low startup time.
Non-striped:
- Worse throughput due to very high contention on BlockingQueue.
+ No waste on idle threads.
I would propose the following final design for this:
- Introduce *"boolean systemThreadPoolStriped"* property;
- Set it to *"true"* for servers by default;
- Set it to *"false"* for clients by default.
Yakov,
I still do not get your point about (system + public) pools approach.
Vladimir.
On Fri, Dec 16, 2016 at 9:29 AM, Yakov Zhdanov <yz...@apache.org> wrote:
> >In my view, we should hide most of such configuration intricacies from
> users,
> and select an appropriate thread pool for each task ourselves. Will this be
> possible?
>
> We already do this. We take decision internally on where to route the
> request.
>
> SoE, my answers below.
>
> >How did the striped pool show itself for transactional updates that
> involve 2 phase commit? Any benefits or degradation?
>
> Even better than for atomic - up to ~50% improvement.
>
> >Here we should decide what’s more important for us - throughout or
> latency. If to execute SQL queries in the striped pool using multiple
> partitioned threads then, for sure, it will affect latency of other
> operations that are sitting in the queue waiting for their time to be
> processed but, on the other hand, the overall throughput of the platform
> should be improved because the operations will be less halted all the time
> by synchronization needs.
>
> >VoltDB decided to go for with the throughput while our current
> architecture is mostly latency based.
>
> What do you mean by "using several partition threads"? I am pretty sure
> that if we do as you suggest we will have degradation here instead of
> boost:
>
> sql-query-put
> after: 77,344.83
> before: 53,578.78
> delta: 30.73%
>
> sql-query-put-offheap
> after 32,212.30
> before 25,322.43
> delta 21.39%
>
> --Yakov
Re: Striped pool configuration
Posted by Yakov Zhdanov <yz...@apache.org>.
>In my view, we should hide most of such configuration intricacies from users,
and select an appropriate thread pool for each task ourselves. Will this be
possible?
We already do this. We take decision internally on where to route the
request.
SoE, my answers below.
>How did the striped pool show itself for transactional updates that
involve 2 phase commit? Any benefits or degradation?
Even better than for atomic - up to ~50% improvement.
>Here we should decide what’s more important for us - throughout or
latency. If to execute SQL queries in the striped pool using multiple
partitioned threads then, for sure, it will affect latency of other
operations that are sitting in the queue waiting for their time to be
processed but, on the other hand, the overall throughput of the platform
should be improved because the operations will be less halted all the time
by synchronization needs.
>VoltDB decided to go for with the throughput while our current
architecture is mostly latency based.
What do you mean by "using several partition threads"? I am pretty sure
that if we do as you suggest we will have degradation here instead of boost:
sql-query-put
after: 77,344.83
before: 53,578.78
delta: 30.73%
sql-query-put-offheap
after 32,212.30
before 25,322.43
delta 21.39%
--Yakov
Re: Striped pool configuration
Posted by Denis Magda <dm...@apache.org>.
Yakov, thanks.
Here are some thoughts from the Son of the Earth :)
> Another reason for not switching to striped pool completely is that stripes
> are good for cache updates and single or batched get operations since they
> reduce inter-thread communication and synchronization to minimum
How did the striped pool show itself for transactional updates that involve 2 phase commit? Any benefits or degradation?
> however I
> think that we should run, for example, SQL queries in old fashioned pools -
> first, they can fallback to scanning random data, second, we pause cache
> updates in some partition for query execution time.
Here we should decide what’s more important for us - throughout or latency. If to execute SQL queries in the striped pool using multiple partitioned threads then, for sure, it will affect latency of other operations that are sitting in the queue waiting for their time to be processed but, on the other hand, the overall throughput of the platform should be improved because the operations will be less halted all the time by synchronization needs.
VoltDB decided to go for with the throughput while our current architecture is mostly latency based.
—
Denis
> On Dec 15, 2016, at 11:22 AM, Yakov Zhdanov <yz...@gridgain.com> wrote:
>
> Denis, there was no discussion on dev list. Do not worry, Son of the Earth,
> you are all right :) I thought I would send letter a bit later when I
> finish the full scope of optimizations I have in plan.
>
> Currently, I use striped pool only for cache messages that are targeted to
> some defined partition - cache update benchmarks showed very good growth in
> throughput.
>
> I tried to switch public and system pool to striped mode, but I get a lot
> of hangs on TC - sometimes due to issues in tests sometimes due to issues
> in Ignite. For example, under some circumstances we can synchronously wait
> in system thread for some message to come. When message handler gets to the
> same stripe Ignite hangs. I have a list of such places discovered so far.
> After I fix them I will try re-approach the pools.
>
> Another reason for not switching to striped pool completely is that stripes
> are good for cache updates and single or batched get operations since they
> reduce inter-thread communication and synchronization to minimum, however I
> think that we should run, for example, SQL queries in old fashioned pools -
> first, they can fallback to scanning random data, second, we pause cache
> updates in some partition for query execution time. I was thinking of
> splitting CPU resources between striped pool and system pool in some ratio.
> I will follow up on this in a couple of days and will explain what has been
> done and what I want to do.
>
> Vladimir, I hope I responded to your points.
>
> Thanks!
>
> --Yakov
Re: Striped pool configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Yakov,
I think that our thread pool configuration is getting overly complex. I am
reading your reasoning for having both, old and striped thread pools, and
even I get confused, even though I have written some of this code myself.
In my view, we should hide most of such configuration intricacies from
users, and select an appropriate thread pool for each task ourselves. Will
this be possible?
D.
On Thu, Dec 15, 2016 at 12:16 PM, Yakov Zhdanov <yz...@apache.org> wrote:
> >In this case users will not be able to perform synchronous cache
> operations from compute tasks, right?
>
> why? you are still free to do this.
>
> --Yakov
>
Re: Striped pool configuration
Posted by Yakov Zhdanov <yz...@apache.org>.
>In this case users will not be able to perform synchronous cache
operations from compute tasks, right?
why? you are still free to do this.
--Yakov
Re: Striped pool configuration
Posted by Vladimir Ozerov <vo...@gridgain.com>.
In this case users will not be able to perform synchronous cache operations
from compute tasks, right?
On Thu, Dec 15, 2016 at 11:05 PM, Yakov Zhdanov <yz...@gridgain.com>
wrote:
> do you agree with the point that we need to keep pub and sys pool together
> with striped pool for now?
>
> --Yakov
>
--
Vladimir Ozerov
Senior Software Architect
GridGain Systems
www.gridgain.com
*+7 (960) 283 98 40*
Fwd: Striped pool configuration
Posted by Yakov Zhdanov <yz...@gridgain.com>.
do you agree with the point that we need to keep pub and sys pool together
with striped pool for now?
--Yakov
Re: Striped pool configuration
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Yakov,
Not really :-) I asked on how to better expose it's configuration to API,
not about it's internals.
On Thu, Dec 15, 2016 at 10:22 PM, Yakov Zhdanov <yz...@gridgain.com>
wrote:
> Denis, there was no discussion on dev list. Do not worry, Son of the Earth,
> you are all right :) I thought I would send letter a bit later when I
> finish the full scope of optimizations I have in plan.
>
> Currently, I use striped pool only for cache messages that are targeted to
> some defined partition - cache update benchmarks showed very good growth in
> throughput.
>
> I tried to switch public and system pool to striped mode, but I get a lot
> of hangs on TC - sometimes due to issues in tests sometimes due to issues
> in Ignite. For example, under some circumstances we can synchronously wait
> in system thread for some message to come. When message handler gets to the
> same stripe Ignite hangs. I have a list of such places discovered so far.
> After I fix them I will try re-approach the pools.
>
> Another reason for not switching to striped pool completely is that stripes
> are good for cache updates and single or batched get operations since they
> reduce inter-thread communication and synchronization to minimum, however I
> think that we should run, for example, SQL queries in old fashioned pools -
> first, they can fallback to scanning random data, second, we pause cache
> updates in some partition for query execution time. I was thinking of
> splitting CPU resources between striped pool and system pool in some ratio.
> I will follow up on this in a couple of days and will explain what has been
> done and what I want to do.
>
> Vladimir, I hope I responded to your points.
>
> Thanks!
>
> --Yakov
>
--
Vladimir Ozerov
Senior Software Architect
GridGain Systems
www.gridgain.com
*+7 (960) 283 98 40*
Fwd: Striped pool configuration
Posted by Yakov Zhdanov <yz...@gridgain.com>.
Denis, there was no discussion on dev list. Do not worry, Son of the Earth,
you are all right :) I thought I would send letter a bit later when I
finish the full scope of optimizations I have in plan.
Currently, I use striped pool only for cache messages that are targeted to
some defined partition - cache update benchmarks showed very good growth in
throughput.
I tried to switch public and system pool to striped mode, but I get a lot
of hangs on TC - sometimes due to issues in tests sometimes due to issues
in Ignite. For example, under some circumstances we can synchronously wait
in system thread for some message to come. When message handler gets to the
same stripe Ignite hangs. I have a list of such places discovered so far.
After I fix them I will try re-approach the pools.
Another reason for not switching to striped pool completely is that stripes
are good for cache updates and single or batched get operations since they
reduce inter-thread communication and synchronization to minimum, however I
think that we should run, for example, SQL queries in old fashioned pools -
first, they can fallback to scanning random data, second, we pause cache
updates in some partition for query execution time. I was thinking of
splitting CPU resources between striped pool and system pool in some ratio.
I will follow up on this in a couple of days and will explain what has been
done and what I want to do.
Vladimir, I hope I responded to your points.
Thanks!
--Yakov
Re: Striped pool configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Igniters,
Why not just move to striped pools and remove this confusion? At this point
we are forcing users to understand what a striped pool is, which seems
unnecessary in my view.
D.
On Thu, Dec 15, 2016 at 8:09 AM, Denis Magda <dm...@apache.org> wrote:
> Vladimir,
>
> > On Dec 15, 2016, at 1:22 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
> >
> > One of them is "striped pool" which can be used instead of current system
> > pool.
>
> Would you mind explaining the whole purpose of this pool and why it was
> decided to introduce it? If I missed some discussion happened on the dev
> list earlier please point out to this.
>
> —
> Denis
Re: Striped pool configuration
Posted by Denis Magda <dm...@apache.org>.
Vladimir,
> On Dec 15, 2016, at 1:22 AM, Vladimir Ozerov <vo...@gridgain.com> wrote:
>
> One of them is "striped pool" which can be used instead of current system
> pool.
Would you mind explaining the whole purpose of this pool and why it was decided to introduce it? If I missed some discussion happened on the dev list earlier please point out to this.
—
Denis