You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by Connie Yau <co...@microsoft.com> on 2016/04/05 00:15:41 UTC
Changing Thread management to Tasks
Hi,
I've been working on trying to move Lucene.NET to .NET Core and one of the issues I am having is migrating ConcurrentMergeScheduler and Threads. I want to replace the ConcurrentMergeScheduler with the default TaskScheduler and Threads with Tasks. However, I noticed that there is a lot of logic to alter merges based on the priority of their threads, etc.
* What is the reason we manage priority threads?
o If the default TaskScheduler is not sufficient, there is a QueuedTaskScheduler<https://code.msdn.microsoft.com/Samples-for-Parallel-b4b76364/sourcecode?fileId=44488&pathId=462437453> that manages priority
Thanks,
Connie
Re: Changing Thread management to Tasks
Posted by Petar Repac <pe...@gmail.com>.
public class FastLogger
{
private static readonly ILog log = LogManager.GetLogger(typeof(FastLogger));
private static readonly bool isDebugEnabled = log.IsDebugEnabled;
public void MyMethod()
{
if(isDebugEnabled)
{
log.Debug("Entry number: " + i + " is " + entry[i]);
}
}
}
If *log.IsDebugEnabled* is false then JIT compiler removes the if ()
statement completely so there is no performance hit at all.
Downside: you can not re-enable logging while running.
More here: https://logging.apache.org/log4net/release/faq.html
Maybe something similar could be used to lower (remove?) injecting
logging dependency perf hit.
On Tue, Apr 12, 2016 at 4:03 AM, Rory Plaire <co...@gmail.com> wrote:
> I agree with being restrained in taking on dependencies. Using a bridge
> pattern and providing an interface for a user-supplied logging library,
> defaulting to a null logger would be more flexible, keep the dependencies
> to a minimum, allow users to keep the logging library they like, and have a
> vanishing effect on performance if no logging is desired is a well-used
> approach for libraries.
>
> On Mon, Apr 11, 2016 at 11:44 AM, Oren Eini (Ayende Rahien) <
> ayende@ayende.com> wrote:
>
> > Please note that adding dependencies is likely to cause pain for
> downstream
> > users.
> > Especially since logging can very easily kill system perofrmance.
> >
> > *Hibernating Rhinos Ltd *
> >
> > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> >
> > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> >
> >
> >
> > On Mon, Apr 11, 2016 at 9:39 PM, Patric Forsgard
> > <patric@tasteful.se.invalid
> > > wrote:
> >
> > > I think it is better to use the
> Microsoft.Extensions.Logging.Abstractions
> > > or anything like the
> > >
> > >
> >
> https://github.com/nhibernate/nhibernate-core/blob/master/src/NHibernate/Logging.cs
> > > to not bind the logging to any specific logging provider.
> > >
> > > // Patric
> > >
> > >
> > > On 11 April 2016 at 20:20, Itamar Syn-Hershko <it...@code972.com>
> > wrote:
> > >
> > > > Now IoC, but I'm in favor of adding a dependency of a lightweight
> > logger
> > > > (Serilog?) that we can expose to all our classes.
> > > >
> > > > --
> > > >
> > > > Itamar Syn-Hershko
> > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > > Freelance Developer & Consultant
> > > > Lucene.NET committer and PMC member
> > > >
> > > > On Mon, Apr 11, 2016 at 7:12 AM, Connie Yau <co...@microsoft.com>
> > > wrote:
> > > >
> > > > > Thanks Oren for pointing this out to me!
> > > > >
> > > > > Are there any logging interfaces or classes (in Lucene.Net) I could
> > > > > dependency inject into the MergeScheduler I am writing to log
> > messages
> > > > > rather than just swallowing the exceptions if an exception gets
> > thrown
> > > > > during the merge or execution of the thread action.
> > > > >
> > > > > Cheers,
> > > > > Connie
> > > > >
> > > > > -----Original Message-----
> > > > > From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> > > > > Sent: Thursday, April 7, 2016 1:51 PM
> > > > > To: dev@lucenenet.apache.org
> > > > > Cc: Daniel Plaisted <da...@microsoft.com>
> > > > > Subject: Re: Changing Thread management to Tasks
> > > > >
> > > > > See here:
> > > > >
> > > > >
> > > >
> > >
> >
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2flucenenet.apache.org%2fdocs%2f3.0.3%2fdc%2fdd1%2f_concurrent_merge_scheduler_8cs_source.html&data=01%7c01%7cconniey%40microsoft.com%7c6f2cdd270b214019b79408d35f266df0%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=46yECAyFkWJmbvYBYSbvJBBRd%2bzpSxvyfe3fZ5YjP7Q%3d
> > > > >
> > > > > We had to write this:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/ayende/ravendb/blob/master/Raven.Database/Indexing/ErrorLoggingConcurrentMergeScheduler.cs
> > > > >
> > > > >
> > > > > *Hibernating Rhinos Ltd *
> > > > >
> > > > > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> > > > >
> > > > > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Apr 7, 2016 at 10:30 PM, Connie Yau <conniey@microsoft.com
> >
> > > > wrote:
> > > > >
> > > > > > Hi Oren,
> > > > > >
> > > > > > Thanks for letting me know about this! When you say "default
> merge
> > > > > > scheduler", do you mean ConcurrentMergeScheduler? Would you be
> able
> > > to
> > > > > > tell me about a scenario where the default merge scheduler would
> > leak
> > > > > > exceptions? Also, are there a set of unit tests that test this?
> > > > > >
> > > > > > Thanks,
> > > > > > Connie
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> > > > > > Sent: Monday, April 4, 2016 10:58 PM
> > > > > > To: dev@lucenenet.apache.org
> > > > > > Cc: Daniel Plaisted <da...@microsoft.com>
> > > > > > Subject: Re: Changing Thread management to Tasks
> > > > > >
> > > > > > Please note, one of the things that we had to do with this is to
> > make
> > > > > > sure that this doesn't leak exceptions.
> > > > > > The default merge scheduler is known to leak exceptions to the
> > > thread,
> > > > > > killing the process. Need to be sure to handle this
> > > > > >
> > > > > > *Hibernating Rhinos Ltd *
> > > > > >
> > > > > > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> > > > > >
> > > > > > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 5, 2016 at 1:15 AM, Connie Yau <
> conniey@microsoft.com>
> > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I've been working on trying to move
> > > > > > >
> > > https://na01.safelinks.protection.outlook.com/?url=Lucene.NET&data=0
> > > > > > > 1%
> > > > > > > 7c01%7cconniey%40microsoft.com
> > > %7c574774f60d874dcc43da08d35d174289%7c
> > > > > > > 72
> > > > > > >
> > > f988bf86f141af91ab2d7cd011db47%7c1&sdata=lcp54tq4j93yifdy8sWFmDNS2Oi
> > > > > > > YZ MGl5EDtRcJJ%2bzU%3d to .NET Core and one of the issues I am
> > > > > > > having is
> > > > > > migrating ConcurrentMergeScheduler and Threads. I want to
> replace
> > > the
> > > > > > ConcurrentMergeScheduler with the default TaskScheduler and
> Threads
> > > > > > with Tasks. However, I noticed that there is a lot of logic to
> > alter
> > > > > > merges based on the priority of their threads, etc.
> > > > > > >
> > > > > > > * What is the reason we manage priority threads?
> > > > > > >
> > > > > > > o If the default TaskScheduler is not sufficient, there is a
> > > > > > > QueuedTaskScheduler<
> > > > > > >
> > > >
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fcode.
> > > > > > > msdn.microsoft.com
> > > %2fSamples-for-Parallel-b4b76364%2fsourcecode%3ffi
> > > > > > > le
> > > > > > >
> > > Id%3d44488%26pathId%3d462437453&data=01%7c01%7cconniey%40microsoft.c
> > > > > > > om
> > > > > > >
> > > %7c574774f60d874dcc43da08d35d174289%7c72f988bf86f141af91ab2d7cd011db
> > > > > > > 47
> %7c1&sdata=%2bqPmfU%2fxQ0hSPSpBWKMJYTkBCkkkOMh1PQ1j7kjwlDA%3d>
> > > > > > > that manages priority
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Connie
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Changing Thread management to Tasks
Posted by Rory Plaire <co...@gmail.com>.
I agree with being restrained in taking on dependencies. Using a bridge
pattern and providing an interface for a user-supplied logging library,
defaulting to a null logger would be more flexible, keep the dependencies
to a minimum, allow users to keep the logging library they like, and have a
vanishing effect on performance if no logging is desired is a well-used
approach for libraries.
On Mon, Apr 11, 2016 at 11:44 AM, Oren Eini (Ayende Rahien) <
ayende@ayende.com> wrote:
> Please note that adding dependencies is likely to cause pain for downstream
> users.
> Especially since logging can very easily kill system perofrmance.
>
> *Hibernating Rhinos Ltd *
>
> Oren Eini* l CEO l *Mobile: + 972-52-548-6969
>
> Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
>
>
>
> On Mon, Apr 11, 2016 at 9:39 PM, Patric Forsgard
> <patric@tasteful.se.invalid
> > wrote:
>
> > I think it is better to use the Microsoft.Extensions.Logging.Abstractions
> > or anything like the
> >
> >
> https://github.com/nhibernate/nhibernate-core/blob/master/src/NHibernate/Logging.cs
> > to not bind the logging to any specific logging provider.
> >
> > // Patric
> >
> >
> > On 11 April 2016 at 20:20, Itamar Syn-Hershko <it...@code972.com>
> wrote:
> >
> > > Now IoC, but I'm in favor of adding a dependency of a lightweight
> logger
> > > (Serilog?) that we can expose to all our classes.
> > >
> > > --
> > >
> > > Itamar Syn-Hershko
> > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > Freelance Developer & Consultant
> > > Lucene.NET committer and PMC member
> > >
> > > On Mon, Apr 11, 2016 at 7:12 AM, Connie Yau <co...@microsoft.com>
> > wrote:
> > >
> > > > Thanks Oren for pointing this out to me!
> > > >
> > > > Are there any logging interfaces or classes (in Lucene.Net) I could
> > > > dependency inject into the MergeScheduler I am writing to log
> messages
> > > > rather than just swallowing the exceptions if an exception gets
> thrown
> > > > during the merge or execution of the thread action.
> > > >
> > > > Cheers,
> > > > Connie
> > > >
> > > > -----Original Message-----
> > > > From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> > > > Sent: Thursday, April 7, 2016 1:51 PM
> > > > To: dev@lucenenet.apache.org
> > > > Cc: Daniel Plaisted <da...@microsoft.com>
> > > > Subject: Re: Changing Thread management to Tasks
> > > >
> > > > See here:
> > > >
> > > >
> > >
> >
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2flucenenet.apache.org%2fdocs%2f3.0.3%2fdc%2fdd1%2f_concurrent_merge_scheduler_8cs_source.html&data=01%7c01%7cconniey%40microsoft.com%7c6f2cdd270b214019b79408d35f266df0%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=46yECAyFkWJmbvYBYSbvJBBRd%2bzpSxvyfe3fZ5YjP7Q%3d
> > > >
> > > > We had to write this:
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/ayende/ravendb/blob/master/Raven.Database/Indexing/ErrorLoggingConcurrentMergeScheduler.cs
> > > >
> > > >
> > > > *Hibernating Rhinos Ltd *
> > > >
> > > > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> > > >
> > > > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> > > >
> > > >
> > > >
> > > > On Thu, Apr 7, 2016 at 10:30 PM, Connie Yau <co...@microsoft.com>
> > > wrote:
> > > >
> > > > > Hi Oren,
> > > > >
> > > > > Thanks for letting me know about this! When you say "default merge
> > > > > scheduler", do you mean ConcurrentMergeScheduler? Would you be able
> > to
> > > > > tell me about a scenario where the default merge scheduler would
> leak
> > > > > exceptions? Also, are there a set of unit tests that test this?
> > > > >
> > > > > Thanks,
> > > > > Connie
> > > > >
> > > > > -----Original Message-----
> > > > > From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> > > > > Sent: Monday, April 4, 2016 10:58 PM
> > > > > To: dev@lucenenet.apache.org
> > > > > Cc: Daniel Plaisted <da...@microsoft.com>
> > > > > Subject: Re: Changing Thread management to Tasks
> > > > >
> > > > > Please note, one of the things that we had to do with this is to
> make
> > > > > sure that this doesn't leak exceptions.
> > > > > The default merge scheduler is known to leak exceptions to the
> > thread,
> > > > > killing the process. Need to be sure to handle this
> > > > >
> > > > > *Hibernating Rhinos Ltd *
> > > > >
> > > > > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> > > > >
> > > > > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Apr 5, 2016 at 1:15 AM, Connie Yau <co...@microsoft.com>
> > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I've been working on trying to move
> > > > > >
> > https://na01.safelinks.protection.outlook.com/?url=Lucene.NET&data=0
> > > > > > 1%
> > > > > > 7c01%7cconniey%40microsoft.com
> > %7c574774f60d874dcc43da08d35d174289%7c
> > > > > > 72
> > > > > >
> > f988bf86f141af91ab2d7cd011db47%7c1&sdata=lcp54tq4j93yifdy8sWFmDNS2Oi
> > > > > > YZ MGl5EDtRcJJ%2bzU%3d to .NET Core and one of the issues I am
> > > > > > having is
> > > > > migrating ConcurrentMergeScheduler and Threads. I want to replace
> > the
> > > > > ConcurrentMergeScheduler with the default TaskScheduler and Threads
> > > > > with Tasks. However, I noticed that there is a lot of logic to
> alter
> > > > > merges based on the priority of their threads, etc.
> > > > > >
> > > > > > * What is the reason we manage priority threads?
> > > > > >
> > > > > > o If the default TaskScheduler is not sufficient, there is a
> > > > > > QueuedTaskScheduler<
> > > > > >
> > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fcode.
> > > > > > msdn.microsoft.com
> > %2fSamples-for-Parallel-b4b76364%2fsourcecode%3ffi
> > > > > > le
> > > > > >
> > Id%3d44488%26pathId%3d462437453&data=01%7c01%7cconniey%40microsoft.c
> > > > > > om
> > > > > >
> > %7c574774f60d874dcc43da08d35d174289%7c72f988bf86f141af91ab2d7cd011db
> > > > > > 47 %7c1&sdata=%2bqPmfU%2fxQ0hSPSpBWKMJYTkBCkkkOMh1PQ1j7kjwlDA%3d>
> > > > > > that manages priority
> > > > > >
> > > > > > Thanks,
> > > > > > Connie
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Changing Thread management to Tasks
Posted by "Oren Eini (Ayende Rahien)" <ay...@ayende.com>.
Please note that adding dependencies is likely to cause pain for downstream
users.
Especially since logging can very easily kill system perofrmance.
*Hibernating Rhinos Ltd *
Oren Eini* l CEO l *Mobile: + 972-52-548-6969
Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
On Mon, Apr 11, 2016 at 9:39 PM, Patric Forsgard <patric@tasteful.se.invalid
> wrote:
> I think it is better to use the Microsoft.Extensions.Logging.Abstractions
> or anything like the
>
> https://github.com/nhibernate/nhibernate-core/blob/master/src/NHibernate/Logging.cs
> to not bind the logging to any specific logging provider.
>
> // Patric
>
>
> On 11 April 2016 at 20:20, Itamar Syn-Hershko <it...@code972.com> wrote:
>
> > Now IoC, but I'm in favor of adding a dependency of a lightweight logger
> > (Serilog?) that we can expose to all our classes.
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
> > On Mon, Apr 11, 2016 at 7:12 AM, Connie Yau <co...@microsoft.com>
> wrote:
> >
> > > Thanks Oren for pointing this out to me!
> > >
> > > Are there any logging interfaces or classes (in Lucene.Net) I could
> > > dependency inject into the MergeScheduler I am writing to log messages
> > > rather than just swallowing the exceptions if an exception gets thrown
> > > during the merge or execution of the thread action.
> > >
> > > Cheers,
> > > Connie
> > >
> > > -----Original Message-----
> > > From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> > > Sent: Thursday, April 7, 2016 1:51 PM
> > > To: dev@lucenenet.apache.org
> > > Cc: Daniel Plaisted <da...@microsoft.com>
> > > Subject: Re: Changing Thread management to Tasks
> > >
> > > See here:
> > >
> > >
> >
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2flucenenet.apache.org%2fdocs%2f3.0.3%2fdc%2fdd1%2f_concurrent_merge_scheduler_8cs_source.html&data=01%7c01%7cconniey%40microsoft.com%7c6f2cdd270b214019b79408d35f266df0%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=46yECAyFkWJmbvYBYSbvJBBRd%2bzpSxvyfe3fZ5YjP7Q%3d
> > >
> > > We had to write this:
> > >
> > >
> > >
> >
> https://github.com/ayende/ravendb/blob/master/Raven.Database/Indexing/ErrorLoggingConcurrentMergeScheduler.cs
> > >
> > >
> > > *Hibernating Rhinos Ltd *
> > >
> > > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> > >
> > > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> > >
> > >
> > >
> > > On Thu, Apr 7, 2016 at 10:30 PM, Connie Yau <co...@microsoft.com>
> > wrote:
> > >
> > > > Hi Oren,
> > > >
> > > > Thanks for letting me know about this! When you say "default merge
> > > > scheduler", do you mean ConcurrentMergeScheduler? Would you be able
> to
> > > > tell me about a scenario where the default merge scheduler would leak
> > > > exceptions? Also, are there a set of unit tests that test this?
> > > >
> > > > Thanks,
> > > > Connie
> > > >
> > > > -----Original Message-----
> > > > From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> > > > Sent: Monday, April 4, 2016 10:58 PM
> > > > To: dev@lucenenet.apache.org
> > > > Cc: Daniel Plaisted <da...@microsoft.com>
> > > > Subject: Re: Changing Thread management to Tasks
> > > >
> > > > Please note, one of the things that we had to do with this is to make
> > > > sure that this doesn't leak exceptions.
> > > > The default merge scheduler is known to leak exceptions to the
> thread,
> > > > killing the process. Need to be sure to handle this
> > > >
> > > > *Hibernating Rhinos Ltd *
> > > >
> > > > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> > > >
> > > > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> > > >
> > > >
> > > >
> > > > On Tue, Apr 5, 2016 at 1:15 AM, Connie Yau <co...@microsoft.com>
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I've been working on trying to move
> > > > >
> https://na01.safelinks.protection.outlook.com/?url=Lucene.NET&data=0
> > > > > 1%
> > > > > 7c01%7cconniey%40microsoft.com
> %7c574774f60d874dcc43da08d35d174289%7c
> > > > > 72
> > > > >
> f988bf86f141af91ab2d7cd011db47%7c1&sdata=lcp54tq4j93yifdy8sWFmDNS2Oi
> > > > > YZ MGl5EDtRcJJ%2bzU%3d to .NET Core and one of the issues I am
> > > > > having is
> > > > migrating ConcurrentMergeScheduler and Threads. I want to replace
> the
> > > > ConcurrentMergeScheduler with the default TaskScheduler and Threads
> > > > with Tasks. However, I noticed that there is a lot of logic to alter
> > > > merges based on the priority of their threads, etc.
> > > > >
> > > > > * What is the reason we manage priority threads?
> > > > >
> > > > > o If the default TaskScheduler is not sufficient, there is a
> > > > > QueuedTaskScheduler<
> > > > >
> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fcode.
> > > > > msdn.microsoft.com
> %2fSamples-for-Parallel-b4b76364%2fsourcecode%3ffi
> > > > > le
> > > > >
> Id%3d44488%26pathId%3d462437453&data=01%7c01%7cconniey%40microsoft.c
> > > > > om
> > > > >
> %7c574774f60d874dcc43da08d35d174289%7c72f988bf86f141af91ab2d7cd011db
> > > > > 47 %7c1&sdata=%2bqPmfU%2fxQ0hSPSpBWKMJYTkBCkkkOMh1PQ1j7kjwlDA%3d>
> > > > > that manages priority
> > > > >
> > > > > Thanks,
> > > > > Connie
> > > > >
> > > >
> > >
> >
>
Re: Changing Thread management to Tasks
Posted by Patric Forsgard <pa...@tasteful.se.INVALID>.
I think it is better to use the Microsoft.Extensions.Logging.Abstractions
or anything like the
https://github.com/nhibernate/nhibernate-core/blob/master/src/NHibernate/Logging.cs
to not bind the logging to any specific logging provider.
// Patric
On 11 April 2016 at 20:20, Itamar Syn-Hershko <it...@code972.com> wrote:
> Now IoC, but I'm in favor of adding a dependency of a lightweight logger
> (Serilog?) that we can expose to all our classes.
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
> On Mon, Apr 11, 2016 at 7:12 AM, Connie Yau <co...@microsoft.com> wrote:
>
> > Thanks Oren for pointing this out to me!
> >
> > Are there any logging interfaces or classes (in Lucene.Net) I could
> > dependency inject into the MergeScheduler I am writing to log messages
> > rather than just swallowing the exceptions if an exception gets thrown
> > during the merge or execution of the thread action.
> >
> > Cheers,
> > Connie
> >
> > -----Original Message-----
> > From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> > Sent: Thursday, April 7, 2016 1:51 PM
> > To: dev@lucenenet.apache.org
> > Cc: Daniel Plaisted <da...@microsoft.com>
> > Subject: Re: Changing Thread management to Tasks
> >
> > See here:
> >
> >
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2flucenenet.apache.org%2fdocs%2f3.0.3%2fdc%2fdd1%2f_concurrent_merge_scheduler_8cs_source.html&data=01%7c01%7cconniey%40microsoft.com%7c6f2cdd270b214019b79408d35f266df0%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=46yECAyFkWJmbvYBYSbvJBBRd%2bzpSxvyfe3fZ5YjP7Q%3d
> >
> > We had to write this:
> >
> >
> >
> https://github.com/ayende/ravendb/blob/master/Raven.Database/Indexing/ErrorLoggingConcurrentMergeScheduler.cs
> >
> >
> > *Hibernating Rhinos Ltd *
> >
> > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> >
> > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> >
> >
> >
> > On Thu, Apr 7, 2016 at 10:30 PM, Connie Yau <co...@microsoft.com>
> wrote:
> >
> > > Hi Oren,
> > >
> > > Thanks for letting me know about this! When you say "default merge
> > > scheduler", do you mean ConcurrentMergeScheduler? Would you be able to
> > > tell me about a scenario where the default merge scheduler would leak
> > > exceptions? Also, are there a set of unit tests that test this?
> > >
> > > Thanks,
> > > Connie
> > >
> > > -----Original Message-----
> > > From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> > > Sent: Monday, April 4, 2016 10:58 PM
> > > To: dev@lucenenet.apache.org
> > > Cc: Daniel Plaisted <da...@microsoft.com>
> > > Subject: Re: Changing Thread management to Tasks
> > >
> > > Please note, one of the things that we had to do with this is to make
> > > sure that this doesn't leak exceptions.
> > > The default merge scheduler is known to leak exceptions to the thread,
> > > killing the process. Need to be sure to handle this
> > >
> > > *Hibernating Rhinos Ltd *
> > >
> > > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> > >
> > > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> > >
> > >
> > >
> > > On Tue, Apr 5, 2016 at 1:15 AM, Connie Yau <co...@microsoft.com>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > I've been working on trying to move
> > > > https://na01.safelinks.protection.outlook.com/?url=Lucene.NET&data=0
> > > > 1%
> > > > 7c01%7cconniey%40microsoft.com%7c574774f60d874dcc43da08d35d174289%7c
> > > > 72
> > > > f988bf86f141af91ab2d7cd011db47%7c1&sdata=lcp54tq4j93yifdy8sWFmDNS2Oi
> > > > YZ MGl5EDtRcJJ%2bzU%3d to .NET Core and one of the issues I am
> > > > having is
> > > migrating ConcurrentMergeScheduler and Threads. I want to replace the
> > > ConcurrentMergeScheduler with the default TaskScheduler and Threads
> > > with Tasks. However, I noticed that there is a lot of logic to alter
> > > merges based on the priority of their threads, etc.
> > > >
> > > > * What is the reason we manage priority threads?
> > > >
> > > > o If the default TaskScheduler is not sufficient, there is a
> > > > QueuedTaskScheduler<
> > > >
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fcode.
> > > > msdn.microsoft.com%2fSamples-for-Parallel-b4b76364%2fsourcecode%3ffi
> > > > le
> > > > Id%3d44488%26pathId%3d462437453&data=01%7c01%7cconniey%40microsoft.c
> > > > om
> > > > %7c574774f60d874dcc43da08d35d174289%7c72f988bf86f141af91ab2d7cd011db
> > > > 47 %7c1&sdata=%2bqPmfU%2fxQ0hSPSpBWKMJYTkBCkkkOMh1PQ1j7kjwlDA%3d>
> > > > that manages priority
> > > >
> > > > Thanks,
> > > > Connie
> > > >
> > >
> >
>
Re: Changing Thread management to Tasks
Posted by Itamar Syn-Hershko <it...@code972.com>.
Now IoC, but I'm in favor of adding a dependency of a lightweight logger
(Serilog?) that we can expose to all our classes.
--
Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member
On Mon, Apr 11, 2016 at 7:12 AM, Connie Yau <co...@microsoft.com> wrote:
> Thanks Oren for pointing this out to me!
>
> Are there any logging interfaces or classes (in Lucene.Net) I could
> dependency inject into the MergeScheduler I am writing to log messages
> rather than just swallowing the exceptions if an exception gets thrown
> during the merge or execution of the thread action.
>
> Cheers,
> Connie
>
> -----Original Message-----
> From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> Sent: Thursday, April 7, 2016 1:51 PM
> To: dev@lucenenet.apache.org
> Cc: Daniel Plaisted <da...@microsoft.com>
> Subject: Re: Changing Thread management to Tasks
>
> See here:
>
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2flucenenet.apache.org%2fdocs%2f3.0.3%2fdc%2fdd1%2f_concurrent_merge_scheduler_8cs_source.html&data=01%7c01%7cconniey%40microsoft.com%7c6f2cdd270b214019b79408d35f266df0%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=46yECAyFkWJmbvYBYSbvJBBRd%2bzpSxvyfe3fZ5YjP7Q%3d
>
> We had to write this:
>
>
> https://github.com/ayende/ravendb/blob/master/Raven.Database/Indexing/ErrorLoggingConcurrentMergeScheduler.cs
>
>
> *Hibernating Rhinos Ltd *
>
> Oren Eini* l CEO l *Mobile: + 972-52-548-6969
>
> Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
>
>
>
> On Thu, Apr 7, 2016 at 10:30 PM, Connie Yau <co...@microsoft.com> wrote:
>
> > Hi Oren,
> >
> > Thanks for letting me know about this! When you say "default merge
> > scheduler", do you mean ConcurrentMergeScheduler? Would you be able to
> > tell me about a scenario where the default merge scheduler would leak
> > exceptions? Also, are there a set of unit tests that test this?
> >
> > Thanks,
> > Connie
> >
> > -----Original Message-----
> > From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> > Sent: Monday, April 4, 2016 10:58 PM
> > To: dev@lucenenet.apache.org
> > Cc: Daniel Plaisted <da...@microsoft.com>
> > Subject: Re: Changing Thread management to Tasks
> >
> > Please note, one of the things that we had to do with this is to make
> > sure that this doesn't leak exceptions.
> > The default merge scheduler is known to leak exceptions to the thread,
> > killing the process. Need to be sure to handle this
> >
> > *Hibernating Rhinos Ltd *
> >
> > Oren Eini* l CEO l *Mobile: + 972-52-548-6969
> >
> > Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
> >
> >
> >
> > On Tue, Apr 5, 2016 at 1:15 AM, Connie Yau <co...@microsoft.com>
> wrote:
> >
> > > Hi,
> > >
> > > I've been working on trying to move
> > > https://na01.safelinks.protection.outlook.com/?url=Lucene.NET&data=0
> > > 1%
> > > 7c01%7cconniey%40microsoft.com%7c574774f60d874dcc43da08d35d174289%7c
> > > 72
> > > f988bf86f141af91ab2d7cd011db47%7c1&sdata=lcp54tq4j93yifdy8sWFmDNS2Oi
> > > YZ MGl5EDtRcJJ%2bzU%3d to .NET Core and one of the issues I am
> > > having is
> > migrating ConcurrentMergeScheduler and Threads. I want to replace the
> > ConcurrentMergeScheduler with the default TaskScheduler and Threads
> > with Tasks. However, I noticed that there is a lot of logic to alter
> > merges based on the priority of their threads, etc.
> > >
> > > * What is the reason we manage priority threads?
> > >
> > > o If the default TaskScheduler is not sufficient, there is a
> > > QueuedTaskScheduler<
> > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fcode.
> > > msdn.microsoft.com%2fSamples-for-Parallel-b4b76364%2fsourcecode%3ffi
> > > le
> > > Id%3d44488%26pathId%3d462437453&data=01%7c01%7cconniey%40microsoft.c
> > > om
> > > %7c574774f60d874dcc43da08d35d174289%7c72f988bf86f141af91ab2d7cd011db
> > > 47 %7c1&sdata=%2bqPmfU%2fxQ0hSPSpBWKMJYTkBCkkkOMh1PQ1j7kjwlDA%3d>
> > > that manages priority
> > >
> > > Thanks,
> > > Connie
> > >
> >
>
RE: Changing Thread management to Tasks
Posted by Connie Yau <co...@microsoft.com>.
Thanks Oren for pointing this out to me!
Are there any logging interfaces or classes (in Lucene.Net) I could dependency inject into the MergeScheduler I am writing to log messages rather than just swallowing the exceptions if an exception gets thrown during the merge or execution of the thread action.
Cheers,
Connie
-----Original Message-----
From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
Sent: Thursday, April 7, 2016 1:51 PM
To: dev@lucenenet.apache.org
Cc: Daniel Plaisted <da...@microsoft.com>
Subject: Re: Changing Thread management to Tasks
See here:
https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2flucenenet.apache.org%2fdocs%2f3.0.3%2fdc%2fdd1%2f_concurrent_merge_scheduler_8cs_source.html&data=01%7c01%7cconniey%40microsoft.com%7c6f2cdd270b214019b79408d35f266df0%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=46yECAyFkWJmbvYBYSbvJBBRd%2bzpSxvyfe3fZ5YjP7Q%3d
We had to write this:
https://github.com/ayende/ravendb/blob/master/Raven.Database/Indexing/ErrorLoggingConcurrentMergeScheduler.cs
*Hibernating Rhinos Ltd *
Oren Eini* l CEO l *Mobile: + 972-52-548-6969
Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
On Thu, Apr 7, 2016 at 10:30 PM, Connie Yau <co...@microsoft.com> wrote:
> Hi Oren,
>
> Thanks for letting me know about this! When you say "default merge
> scheduler", do you mean ConcurrentMergeScheduler? Would you be able to
> tell me about a scenario where the default merge scheduler would leak
> exceptions? Also, are there a set of unit tests that test this?
>
> Thanks,
> Connie
>
> -----Original Message-----
> From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> Sent: Monday, April 4, 2016 10:58 PM
> To: dev@lucenenet.apache.org
> Cc: Daniel Plaisted <da...@microsoft.com>
> Subject: Re: Changing Thread management to Tasks
>
> Please note, one of the things that we had to do with this is to make
> sure that this doesn't leak exceptions.
> The default merge scheduler is known to leak exceptions to the thread,
> killing the process. Need to be sure to handle this
>
> *Hibernating Rhinos Ltd *
>
> Oren Eini* l CEO l *Mobile: + 972-52-548-6969
>
> Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
>
>
>
> On Tue, Apr 5, 2016 at 1:15 AM, Connie Yau <co...@microsoft.com> wrote:
>
> > Hi,
> >
> > I've been working on trying to move
> > https://na01.safelinks.protection.outlook.com/?url=Lucene.NET&data=0
> > 1%
> > 7c01%7cconniey%40microsoft.com%7c574774f60d874dcc43da08d35d174289%7c
> > 72
> > f988bf86f141af91ab2d7cd011db47%7c1&sdata=lcp54tq4j93yifdy8sWFmDNS2Oi
> > YZ MGl5EDtRcJJ%2bzU%3d to .NET Core and one of the issues I am
> > having is
> migrating ConcurrentMergeScheduler and Threads. I want to replace the
> ConcurrentMergeScheduler with the default TaskScheduler and Threads
> with Tasks. However, I noticed that there is a lot of logic to alter
> merges based on the priority of their threads, etc.
> >
> > * What is the reason we manage priority threads?
> >
> > o If the default TaskScheduler is not sufficient, there is a
> > QueuedTaskScheduler<
> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fcode.
> > msdn.microsoft.com%2fSamples-for-Parallel-b4b76364%2fsourcecode%3ffi
> > le
> > Id%3d44488%26pathId%3d462437453&data=01%7c01%7cconniey%40microsoft.c
> > om
> > %7c574774f60d874dcc43da08d35d174289%7c72f988bf86f141af91ab2d7cd011db
> > 47 %7c1&sdata=%2bqPmfU%2fxQ0hSPSpBWKMJYTkBCkkkOMh1PQ1j7kjwlDA%3d>
> > that manages priority
> >
> > Thanks,
> > Connie
> >
>
Re: Changing Thread management to Tasks
Posted by "Oren Eini (Ayende Rahien)" <ay...@ayende.com>.
See here:
https://lucenenet.apache.org/docs/3.0.3/dc/dd1/_concurrent_merge_scheduler_8cs_source.html
We had to write this:
https://github.com/ayende/ravendb/blob/master/Raven.Database/Indexing/ErrorLoggingConcurrentMergeScheduler.cs
*Hibernating Rhinos Ltd *
Oren Eini* l CEO l *Mobile: + 972-52-548-6969
Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
On Thu, Apr 7, 2016 at 10:30 PM, Connie Yau <co...@microsoft.com> wrote:
> Hi Oren,
>
> Thanks for letting me know about this! When you say "default merge
> scheduler", do you mean ConcurrentMergeScheduler? Would you be able to tell
> me about a scenario where the default merge scheduler would leak
> exceptions? Also, are there a set of unit tests that test this?
>
> Thanks,
> Connie
>
> -----Original Message-----
> From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
> Sent: Monday, April 4, 2016 10:58 PM
> To: dev@lucenenet.apache.org
> Cc: Daniel Plaisted <da...@microsoft.com>
> Subject: Re: Changing Thread management to Tasks
>
> Please note, one of the things that we had to do with this is to make sure
> that this doesn't leak exceptions.
> The default merge scheduler is known to leak exceptions to the thread,
> killing the process. Need to be sure to handle this
>
> *Hibernating Rhinos Ltd *
>
> Oren Eini* l CEO l *Mobile: + 972-52-548-6969
>
> Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
>
>
>
> On Tue, Apr 5, 2016 at 1:15 AM, Connie Yau <co...@microsoft.com> wrote:
>
> > Hi,
> >
> > I've been working on trying to move
> > https://na01.safelinks.protection.outlook.com/?url=Lucene.NET&data=01%
> > 7c01%7cconniey%40microsoft.com%7c574774f60d874dcc43da08d35d174289%7c72
> > f988bf86f141af91ab2d7cd011db47%7c1&sdata=lcp54tq4j93yifdy8sWFmDNS2OiYZ
> > MGl5EDtRcJJ%2bzU%3d to .NET Core and one of the issues I am having is
> migrating ConcurrentMergeScheduler and Threads. I want to replace the
> ConcurrentMergeScheduler with the default TaskScheduler and Threads with
> Tasks. However, I noticed that there is a lot of logic to alter merges
> based on the priority of their threads, etc.
> >
> > * What is the reason we manage priority threads?
> >
> > o If the default TaskScheduler is not sufficient, there is a
> > QueuedTaskScheduler<
> > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fcode.
> > msdn.microsoft.com%2fSamples-for-Parallel-b4b76364%2fsourcecode%3ffile
> > Id%3d44488%26pathId%3d462437453&data=01%7c01%7cconniey%40microsoft.com
> > %7c574774f60d874dcc43da08d35d174289%7c72f988bf86f141af91ab2d7cd011db47
> > %7c1&sdata=%2bqPmfU%2fxQ0hSPSpBWKMJYTkBCkkkOMh1PQ1j7kjwlDA%3d>
> > that manages priority
> >
> > Thanks,
> > Connie
> >
>
RE: Changing Thread management to Tasks
Posted by Connie Yau <co...@microsoft.com>.
Hi Oren,
Thanks for letting me know about this! When you say "default merge scheduler", do you mean ConcurrentMergeScheduler? Would you be able to tell me about a scenario where the default merge scheduler would leak exceptions? Also, are there a set of unit tests that test this?
Thanks,
Connie
-----Original Message-----
From: Oren Eini (Ayende Rahien) [mailto:ayende@ayende.com]
Sent: Monday, April 4, 2016 10:58 PM
To: dev@lucenenet.apache.org
Cc: Daniel Plaisted <da...@microsoft.com>
Subject: Re: Changing Thread management to Tasks
Please note, one of the things that we had to do with this is to make sure that this doesn't leak exceptions.
The default merge scheduler is known to leak exceptions to the thread, killing the process. Need to be sure to handle this
*Hibernating Rhinos Ltd *
Oren Eini* l CEO l *Mobile: + 972-52-548-6969
Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
On Tue, Apr 5, 2016 at 1:15 AM, Connie Yau <co...@microsoft.com> wrote:
> Hi,
>
> I've been working on trying to move
> https://na01.safelinks.protection.outlook.com/?url=Lucene.NET&data=01%
> 7c01%7cconniey%40microsoft.com%7c574774f60d874dcc43da08d35d174289%7c72
> f988bf86f141af91ab2d7cd011db47%7c1&sdata=lcp54tq4j93yifdy8sWFmDNS2OiYZ
> MGl5EDtRcJJ%2bzU%3d to .NET Core and one of the issues I am having is migrating ConcurrentMergeScheduler and Threads. I want to replace the ConcurrentMergeScheduler with the default TaskScheduler and Threads with Tasks. However, I noticed that there is a lot of logic to alter merges based on the priority of their threads, etc.
>
> * What is the reason we manage priority threads?
>
> o If the default TaskScheduler is not sufficient, there is a
> QueuedTaskScheduler<
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fcode.
> msdn.microsoft.com%2fSamples-for-Parallel-b4b76364%2fsourcecode%3ffile
> Id%3d44488%26pathId%3d462437453&data=01%7c01%7cconniey%40microsoft.com
> %7c574774f60d874dcc43da08d35d174289%7c72f988bf86f141af91ab2d7cd011db47
> %7c1&sdata=%2bqPmfU%2fxQ0hSPSpBWKMJYTkBCkkkOMh1PQ1j7kjwlDA%3d>
> that manages priority
>
> Thanks,
> Connie
>
Re: Changing Thread management to Tasks
Posted by "Oren Eini (Ayende Rahien)" <ay...@ayende.com>.
Please note, one of the things that we had to do with this is to make sure
that this doesn't leak exceptions.
The default merge scheduler is known to leak exceptions to the thread,
killing the process. Need to be sure to handle this
*Hibernating Rhinos Ltd *
Oren Eini* l CEO l *Mobile: + 972-52-548-6969
Office: +972-4-622-7811 *l *Fax: +972-153-4-622-7811
On Tue, Apr 5, 2016 at 1:15 AM, Connie Yau <co...@microsoft.com> wrote:
> Hi,
>
> I've been working on trying to move Lucene.NET to .NET Core and one of the
> issues I am having is migrating ConcurrentMergeScheduler and Threads. I
> want to replace the ConcurrentMergeScheduler with the default TaskScheduler
> and Threads with Tasks. However, I noticed that there is a lot of logic to
> alter merges based on the priority of their threads, etc.
>
> * What is the reason we manage priority threads?
>
> o If the default TaskScheduler is not sufficient, there is a
> QueuedTaskScheduler<
> https://code.msdn.microsoft.com/Samples-for-Parallel-b4b76364/sourcecode?fileId=44488&pathId=462437453>
> that manages priority
>
> Thanks,
> Connie
>