You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Shai Erera <se...@gmail.com> on 2010/12/22 20:14:47 UTC
LuceneTestCase.threadCleanup incorrectly reports left running threads
Hi
I noticed that some tests report threads are left running, even when those
tests never create and start a Thread. Digging deeper I found out that the
tests report "Signal Dispatcher" and "Attach handler" as two threads that
are left running. If I run the test from eclipse, then a "ReaderThread" and
"Signal Dispatcher" are reported. ReaderThread belongs to JUnit framework
and the other two are initiated by some framework, and definitely not from
our tests.
So I was thinking if instead of reporting those threads, we should inspect
each running Thread's stacktrace and report it only if it contains an
"org.apache.lucene/solr" package. Otherwise it cannot be started from our
tests?
What do you think?
Shai
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Uwe Schindler <uw...@thetaphi.de>.
Jaja. Dont want to discuss about that. :'(
Merry Xmas! :)
"Robert Muir" <rc...@gmail.com> schrieb:
>On Sat, Dec 25, 2010 at 9:24 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>>
>http://download.oracle.com/javase/1.4.2/docs/guide/security/CryptoSpec.html
>>
>> Under "architecture", same document exists for later jvms. This is
>the same type of document that e.g. lists default charsets.
>>
>
>I don't see anywhere in this document stating that MD5 is mandatory. I
>only see things such as documentation on which algorithm SUN's
>particular implementation includes, and what standards must be met if
>an implementation does support MD5.
>
>So this is not at all like Charsets, which clearly says in the
>javadoc: "Every implementation of the Java platform is required to
>support the following standard charsets."
>http://download.oracle.com/javase/1.4.2/docs/api/java/nio/charset/Charset.html
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>For additional commands, e-mail: dev-help@lucene.apache.org
--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Robert Muir <rc...@gmail.com>.
On Sat, Dec 25, 2010 at 9:24 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> http://download.oracle.com/javase/1.4.2/docs/guide/security/CryptoSpec.html
>
> Under "architecture", same document exists for later jvms. This is the same type of document that e.g. lists default charsets.
>
I don't see anywhere in this document stating that MD5 is mandatory. I
only see things such as documentation on which algorithm SUN's
particular implementation includes, and what standards must be met if
an implementation does support MD5.
So this is not at all like Charsets, which clearly says in the
javadoc: "Every implementation of the Java platform is required to
support the following standard charsets."
http://download.oracle.com/javase/1.4.2/docs/api/java/nio/charset/Charset.html
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Uwe Schindler <uw...@thetaphi.de>.
http://download.oracle.com/javase/1.4.2/docs/guide/security/CryptoSpec.html
Under "architecture", same document exists for later jvms. This is the same type of document that e.g. lists default charsets.
Uwe
"Robert Muir" <rc...@gmail.com> schrieb:
>On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>> Md5 is guaranteed to be there (like utf8 as charset). This is
>documented in crypto Api, which algorithms are available for digest.
>>
>
>where is this documented? its not in the javadocs.
>
>anyway, we shouldn't be doing this:
>* this algorithm might not exist on J2ME etc (still java), you need to
>install an extra crypto add-on.
>* we shouldnt start up an expensive PKI infrastructure on mac os X,
>including spawning a new thread, just to hash a string. thats absurd.
>* we pay all these costs ... for md5! its not even a good hash!
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>For additional commands, e-mail: dev-help@lucene.apache.org
--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Robert Muir <rc...@gmail.com>.
On Sun, Dec 26, 2010 at 5:17 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> About MD5: At least for Lucene 3.x we must stay with MD5, as changing the
> hash algorithm would change the lock file name, so an index locked with 3.0
> would have a different lock file than an index locked with 3.x (for external
> lock directory). This would lead to problems for users upgrading several
> servers writing to shared indexes during the upgrades. You cannot be sure
> that the same index is always locked when you have 2 different lucene
> versions. This is just a warning, maybe it’s not important, but its
> important.
Its not that important, as the hash algorithm uses getBytes() <--
default charset.
So, this functionality is already broken anyway, as it will hash the
same string to different hashes across different JVMs who happen to
have a different default charset.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Shai Erera <se...@gmail.com>.
Opening an issue sounds great. It kinda hijacked that thread already :)
We can argue there whether FSLockFactory should or shouldn't allow locking
outside the index directory. Perhaps we can put on the application the
burden of generating a unique lock name, if it intends to use it outside the
index directory.
Shai
On Sun, Dec 26, 2010 at 12:17 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Hi Shai,
>
>
>
> Thanks for the insight. I still disagree about the lock file placement. If
> your application needs such functionality you can always program your own
> lock factory. But even if we add support for external dirs, the creation of
> the lock file name and prefix should be done inside the lock factory
> (currently, the lock id – the MD5 – is created in the directory instance).
> This has pros for e.g. RAMDirectory, but is just unclean at the moment.
> Especially the special handling of the lock id when the directory path for
> the lock file differs from the index directory.
>
>
>
> About MD5: At least for Lucene 3.x we must stay with MD5, as changing the
> hash algorithm would change the lock file name, so an index locked with 3.0
> would have a different lock file than an index locked with 3.x (for external
> lock directory). This would lead to problems for users upgrading several
> servers writing to shared indexes during the upgrades. You cannot be sure
> that the same index is always locked when you have 2 different lucene
> versions. This is just a warning, maybe it’s not important, but its
> important.
>
>
>
> I would propose the following (for 3.x): Remove the static MessageDigest
> from FSDirectory and replace it by a lazy-init one. Because for the common
> case, the message digest is never used and should not be initialized. On
> first use, create one (which only happens if you specify a separate lock
> directory). Normal Lucene users and Solr will never hit this. We can stay
> with MD5. About availability: As nitpicked yesterday with Robert, we can
> still assume MD5 is always available for any JVM, because without MD5, tons
> of software applications will break (just think of sessions in web app
> servers). Harmony has it of course, Android. So we should simply use it (and
> please don’t copy an MD5 impl to util!) and throw exception when not
> available. For this seldom case (normal user have their locks in index dir),
> this is fine. Everything else would break safe upgrade pathes.
>
>
>
> I think we should open an issue and discuss about the LockFactory
> simplyfications (the code is currently very broken as it relies on strange
> sophisticated backwards assumptions, just my opinion).
>
>
>
> Uwe
>
> -----
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de
>
> eMail: uwe@thetaphi.de
>
>
>
> *From:* Shai Erera [mailto:serera@gmail.com]
> *Sent:* Sunday, December 26, 2010 7:43 AM
>
> *To:* dev@lucene.apache.org
> *Subject:* Re: LuceneTestCase.threadCleanup incorrectly reports left
> running threads
>
>
>
> Thanks for the explanation Uwe. I got confused with that and the process
> name we add to the temp lock file.
>
> About what you say, we have a scenario where we use a FSLockFactory outside
> an IndexWriter scope, but still within a search application scope. We lock a
> directory so that several processes can be synced on it. Another use case is
> when you manage several indexes, that should be locked at once -- the lock
> file may reside external to all of them.
>
> So I would not want to see LockFactory suddenly assuming and enforcing the
> lock file to be created in the index directory, or rely on IndexWriter
> passing it something etc. I think the way LF works today, allowing you to
> place the lock file wherever you want gives more freedom to people and opens
> up the door for more use cases.
>
> I don't think though that we should rely on MD5. A simple hash function
> should be enough IMO.
>
> Shai
>
> On Sun, Dec 26, 2010 at 12:41 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>
> Hi Shai,
>
> the md5 hash generated has nothing to do with concurrency anymore (the
> concurrency thing was this NativeFSLock test method already removed). The
> thing is the following:
>
> In early lucene versions, the lock files were put into TEMP directory.
> Later
> the lock factories allowed, to put the lock files into arbitrary folders.
> For these both cases, the lock file name got an MD5 hash of the index
> directory appended/prepended. In later Lucene versions the default for lock
> files was changed to be the index folder. For backwards compatibility
> reasons, with 2.9 and 3.0 you still had the possibility to instantiate a
> LockFactory using a non-null path (using the ctor with a directory name).
> FSLockFactory was programmed to support both cases (null directory or
> explicit directory). When the lock directory is the same like the index
> directory, the lock file got no hash appended. For the rare case that
> somebody used a different folder (e.g. a temp directory), FSLockFactory was
> falling back to the "old" behavior of adding the hash to the lock file
> name.
>
> The magic for the md5 magic lock prefix is done if
> FSDirectory#setLockFactory(). It checks for lockFactory extends
> FSLockFactory and if yes then checks, that the LockFactories path name is
> the same like the FSDir's or null. In that case it sets the lock prefix to
> null. Otherwise the lock prefix is generated by calling the magic MD5
> creating method (Directory#getLockId()).
>
> In my opinion, in 3.x we should deprecate the separate path for the lock
> file (Directory#getLockId()) and enforce the lockfile always to be placed
> in
> the index dir. LockFactory should not get a directory at all, but instead
> should get the index dir on locking. For FS locks it would place the
> write.lock file in the supplied folder and for other locks (like per-JVM
> locks for RAMDirs) it could e.g. lookup the index dir in some map or
> whatever. To place the lockfile somewhere else, you should be able to use
> FileSwitchDirectory (currently not possible).
>
> Most tests in Lucene use the default (null lock dir in LockFactory), but
> some tests for SimpleFSLockFactory & Co use the explicit directory names
> and
> therefore generate MD5 hashes to test the special behavior.
>
> For compatibility reasons we have to still use MD5 (to prevent different
> lock file names after Lucene upgrade when FSDir is locked by another JVM
> with older Lucene version). For 4.0 I would remove this stupidity and only
> allow lock files in index directory.
>
> I hope I explained this stuff so everybody understand it, its really a
> little bit confusing (how its implemented), but its "sophisticated
> backwards" (haha). I would like to get rid of it and then we have no digest
> code anymore.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
> > -----Original Message-----
> > From: Shai Erera [mailto:serera@gmail.com]
> > Sent: Saturday, December 25, 2010 3:04 PM
> > To: dev@lucene.apache.org
> > Subject: Re: LuceneTestCase.threadCleanup incorrectly reports left
> running
> > threads
> >
> > Actually, the MD5 thingy is an attempt to generate a unique temp lock ID,
> > IIRC. so this piece of code can disappear entirely now that the tests
> > concurrency is better.
> >
> > As for the other threads that are left running, I couldn't track down yet
> the
> > warning from the benchmark tests, but I'd love to get rid of those false
> > warnings. I thought the stack trace could at least tell us who spawned
> the
> > thread, but obviously it's not always clear.
> >
> > Shai
> >
> > On Saturday, December 25, 2010, Robert Muir <rc...@gmail.com> wrote:
> > > On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de>
> > wrote:
> > >> Md5 is guaranteed to be there (like utf8 as charset). This is
> documented in
> > crypto Api, which algorithms are available for digest.
> > >>
> > >
> > > where is this documented? its not in the javadocs.
> > >
> > > anyway, we shouldn't be doing this:
> > > * this algorithm might not exist on J2ME etc (still java), you need to
> > > install an extra crypto add-on.
> > > * we shouldnt start up an expensive PKI infrastructure on mac os X,
> > > including spawning a new thread, just to hash a string. thats absurd.
> > > * we pay all these costs ... for md5! its not even a good hash!
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > > additional commands, e-mail: dev-help@lucene.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> > commands, e-mail: dev-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
>
RE: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Shai,
Thanks for the insight. I still disagree about the lock file placement. If
your application needs such functionality you can always program your own
lock factory. But even if we add support for external dirs, the creation of
the lock file name and prefix should be done inside the lock factory
(currently, the lock id - the MD5 - is created in the directory instance).
This has pros for e.g. RAMDirectory, but is just unclean at the moment.
Especially the special handling of the lock id when the directory path for
the lock file differs from the index directory.
About MD5: At least for Lucene 3.x we must stay with MD5, as changing the
hash algorithm would change the lock file name, so an index locked with 3.0
would have a different lock file than an index locked with 3.x (for external
lock directory). This would lead to problems for users upgrading several
servers writing to shared indexes during the upgrades. You cannot be sure
that the same index is always locked when you have 2 different lucene
versions. This is just a warning, maybe it's not important, but its
important.
I would propose the following (for 3.x): Remove the static MessageDigest
from FSDirectory and replace it by a lazy-init one. Because for the common
case, the message digest is never used and should not be initialized. On
first use, create one (which only happens if you specify a separate lock
directory). Normal Lucene users and Solr will never hit this. We can stay
with MD5. About availability: As nitpicked yesterday with Robert, we can
still assume MD5 is always available for any JVM, because without MD5, tons
of software applications will break (just think of sessions in web app
servers). Harmony has it of course, Android. So we should simply use it (and
please don't copy an MD5 impl to util!) and throw exception when not
available. For this seldom case (normal user have their locks in index dir),
this is fine. Everything else would break safe upgrade pathes.
I think we should open an issue and discuss about the LockFactory
simplyfications (the code is currently very broken as it relies on strange
sophisticated backwards assumptions, just my opinion).
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
<http://www.thetaphi.de/> http://www.thetaphi.de
eMail: uwe@thetaphi.de
From: Shai Erera [mailto:serera@gmail.com]
Sent: Sunday, December 26, 2010 7:43 AM
To: dev@lucene.apache.org
Subject: Re: LuceneTestCase.threadCleanup incorrectly reports left running
threads
Thanks for the explanation Uwe. I got confused with that and the process
name we add to the temp lock file.
About what you say, we have a scenario where we use a FSLockFactory outside
an IndexWriter scope, but still within a search application scope. We lock a
directory so that several processes can be synced on it. Another use case is
when you manage several indexes, that should be locked at once -- the lock
file may reside external to all of them.
So I would not want to see LockFactory suddenly assuming and enforcing the
lock file to be created in the index directory, or rely on IndexWriter
passing it something etc. I think the way LF works today, allowing you to
place the lock file wherever you want gives more freedom to people and opens
up the door for more use cases.
I don't think though that we should rely on MD5. A simple hash function
should be enough IMO.
Shai
On Sun, Dec 26, 2010 at 12:41 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
Hi Shai,
the md5 hash generated has nothing to do with concurrency anymore (the
concurrency thing was this NativeFSLock test method already removed). The
thing is the following:
In early lucene versions, the lock files were put into TEMP directory. Later
the lock factories allowed, to put the lock files into arbitrary folders.
For these both cases, the lock file name got an MD5 hash of the index
directory appended/prepended. In later Lucene versions the default for lock
files was changed to be the index folder. For backwards compatibility
reasons, with 2.9 and 3.0 you still had the possibility to instantiate a
LockFactory using a non-null path (using the ctor with a directory name).
FSLockFactory was programmed to support both cases (null directory or
explicit directory). When the lock directory is the same like the index
directory, the lock file got no hash appended. For the rare case that
somebody used a different folder (e.g. a temp directory), FSLockFactory was
falling back to the "old" behavior of adding the hash to the lock file name.
The magic for the md5 magic lock prefix is done if
FSDirectory#setLockFactory(). It checks for lockFactory extends
FSLockFactory and if yes then checks, that the LockFactories path name is
the same like the FSDir's or null. In that case it sets the lock prefix to
null. Otherwise the lock prefix is generated by calling the magic MD5
creating method (Directory#getLockId()).
In my opinion, in 3.x we should deprecate the separate path for the lock
file (Directory#getLockId()) and enforce the lockfile always to be placed in
the index dir. LockFactory should not get a directory at all, but instead
should get the index dir on locking. For FS locks it would place the
write.lock file in the supplied folder and for other locks (like per-JVM
locks for RAMDirs) it could e.g. lookup the index dir in some map or
whatever. To place the lockfile somewhere else, you should be able to use
FileSwitchDirectory (currently not possible).
Most tests in Lucene use the default (null lock dir in LockFactory), but
some tests for SimpleFSLockFactory & Co use the explicit directory names and
therefore generate MD5 hashes to test the special behavior.
For compatibility reasons we have to still use MD5 (to prevent different
lock file names after Lucene upgrade when FSDir is locked by another JVM
with older Lucene version). For 4.0 I would remove this stupidity and only
allow lock files in index directory.
I hope I explained this stuff so everybody understand it, its really a
little bit confusing (how its implemented), but its "sophisticated
backwards" (haha). I would like to get rid of it and then we have no digest
code anymore.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Shai Erera [mailto:serera@gmail.com]
> Sent: Saturday, December 25, 2010 3:04 PM
> To: dev@lucene.apache.org
> Subject: Re: LuceneTestCase.threadCleanup incorrectly reports left running
> threads
>
> Actually, the MD5 thingy is an attempt to generate a unique temp lock ID,
> IIRC. so this piece of code can disappear entirely now that the tests
> concurrency is better.
>
> As for the other threads that are left running, I couldn't track down yet
the
> warning from the benchmark tests, but I'd love to get rid of those false
> warnings. I thought the stack trace could at least tell us who spawned the
> thread, but obviously it's not always clear.
>
> Shai
>
> On Saturday, December 25, 2010, Robert Muir <rc...@gmail.com> wrote:
> > On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de>
> wrote:
> >> Md5 is guaranteed to be there (like utf8 as charset). This is
documented in
> crypto Api, which algorithms are available for digest.
> >>
> >
> > where is this documented? its not in the javadocs.
> >
> > anyway, we shouldn't be doing this:
> > * this algorithm might not exist on J2ME etc (still java), you need to
> > install an extra crypto add-on.
> > * we shouldnt start up an expensive PKI infrastructure on mac os X,
> > including spawning a new thread, just to hash a string. thats absurd.
> > * we pay all these costs ... for md5! its not even a good hash!
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Shai Erera <se...@gmail.com>.
Thanks for the explanation Uwe. I got confused with that and the process
name we add to the temp lock file.
About what you say, we have a scenario where we use a FSLockFactory outside
an IndexWriter scope, but still within a search application scope. We lock a
directory so that several processes can be synced on it. Another use case is
when you manage several indexes, that should be locked at once -- the lock
file may reside external to all of them.
So I would not want to see LockFactory suddenly assuming and enforcing the
lock file to be created in the index directory, or rely on IndexWriter
passing it something etc. I think the way LF works today, allowing you to
place the lock file wherever you want gives more freedom to people and opens
up the door for more use cases.
I don't think though that we should rely on MD5. A simple hash function
should be enough IMO.
Shai
On Sun, Dec 26, 2010 at 12:41 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Hi Shai,
>
> the md5 hash generated has nothing to do with concurrency anymore (the
> concurrency thing was this NativeFSLock test method already removed). The
> thing is the following:
>
> In early lucene versions, the lock files were put into TEMP directory.
> Later
> the lock factories allowed, to put the lock files into arbitrary folders.
> For these both cases, the lock file name got an MD5 hash of the index
> directory appended/prepended. In later Lucene versions the default for lock
> files was changed to be the index folder. For backwards compatibility
> reasons, with 2.9 and 3.0 you still had the possibility to instantiate a
> LockFactory using a non-null path (using the ctor with a directory name).
> FSLockFactory was programmed to support both cases (null directory or
> explicit directory). When the lock directory is the same like the index
> directory, the lock file got no hash appended. For the rare case that
> somebody used a different folder (e.g. a temp directory), FSLockFactory was
> falling back to the "old" behavior of adding the hash to the lock file
> name.
>
> The magic for the md5 magic lock prefix is done if
> FSDirectory#setLockFactory(). It checks for lockFactory extends
> FSLockFactory and if yes then checks, that the LockFactories path name is
> the same like the FSDir's or null. In that case it sets the lock prefix to
> null. Otherwise the lock prefix is generated by calling the magic MD5
> creating method (Directory#getLockId()).
>
> In my opinion, in 3.x we should deprecate the separate path for the lock
> file (Directory#getLockId()) and enforce the lockfile always to be placed
> in
> the index dir. LockFactory should not get a directory at all, but instead
> should get the index dir on locking. For FS locks it would place the
> write.lock file in the supplied folder and for other locks (like per-JVM
> locks for RAMDirs) it could e.g. lookup the index dir in some map or
> whatever. To place the lockfile somewhere else, you should be able to use
> FileSwitchDirectory (currently not possible).
>
> Most tests in Lucene use the default (null lock dir in LockFactory), but
> some tests for SimpleFSLockFactory & Co use the explicit directory names
> and
> therefore generate MD5 hashes to test the special behavior.
>
> For compatibility reasons we have to still use MD5 (to prevent different
> lock file names after Lucene upgrade when FSDir is locked by another JVM
> with older Lucene version). For 4.0 I would remove this stupidity and only
> allow lock files in index directory.
>
> I hope I explained this stuff so everybody understand it, its really a
> little bit confusing (how its implemented), but its "sophisticated
> backwards" (haha). I would like to get rid of it and then we have no digest
> code anymore.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
> > -----Original Message-----
> > From: Shai Erera [mailto:serera@gmail.com]
> > Sent: Saturday, December 25, 2010 3:04 PM
> > To: dev@lucene.apache.org
> > Subject: Re: LuceneTestCase.threadCleanup incorrectly reports left
> running
> > threads
> >
> > Actually, the MD5 thingy is an attempt to generate a unique temp lock ID,
> > IIRC. so this piece of code can disappear entirely now that the tests
> > concurrency is better.
> >
> > As for the other threads that are left running, I couldn't track down yet
> the
> > warning from the benchmark tests, but I'd love to get rid of those false
> > warnings. I thought the stack trace could at least tell us who spawned
> the
> > thread, but obviously it's not always clear.
> >
> > Shai
> >
> > On Saturday, December 25, 2010, Robert Muir <rc...@gmail.com> wrote:
> > > On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de>
> > wrote:
> > >> Md5 is guaranteed to be there (like utf8 as charset). This is
> documented in
> > crypto Api, which algorithms are available for digest.
> > >>
> > >
> > > where is this documented? its not in the javadocs.
> > >
> > > anyway, we shouldn't be doing this:
> > > * this algorithm might not exist on J2ME etc (still java), you need to
> > > install an extra crypto add-on.
> > > * we shouldnt start up an expensive PKI infrastructure on mac os X,
> > > including spawning a new thread, just to hash a string. thats absurd.
> > > * we pay all these costs ... for md5! its not even a good hash!
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > > additional commands, e-mail: dev-help@lucene.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> > commands, e-mail: dev-help@lucene.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
RE: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Shai,
the md5 hash generated has nothing to do with concurrency anymore (the
concurrency thing was this NativeFSLock test method already removed). The
thing is the following:
In early lucene versions, the lock files were put into TEMP directory. Later
the lock factories allowed, to put the lock files into arbitrary folders.
For these both cases, the lock file name got an MD5 hash of the index
directory appended/prepended. In later Lucene versions the default for lock
files was changed to be the index folder. For backwards compatibility
reasons, with 2.9 and 3.0 you still had the possibility to instantiate a
LockFactory using a non-null path (using the ctor with a directory name).
FSLockFactory was programmed to support both cases (null directory or
explicit directory). When the lock directory is the same like the index
directory, the lock file got no hash appended. For the rare case that
somebody used a different folder (e.g. a temp directory), FSLockFactory was
falling back to the "old" behavior of adding the hash to the lock file name.
The magic for the md5 magic lock prefix is done if
FSDirectory#setLockFactory(). It checks for lockFactory extends
FSLockFactory and if yes then checks, that the LockFactories path name is
the same like the FSDir's or null. In that case it sets the lock prefix to
null. Otherwise the lock prefix is generated by calling the magic MD5
creating method (Directory#getLockId()).
In my opinion, in 3.x we should deprecate the separate path for the lock
file (Directory#getLockId()) and enforce the lockfile always to be placed in
the index dir. LockFactory should not get a directory at all, but instead
should get the index dir on locking. For FS locks it would place the
write.lock file in the supplied folder and for other locks (like per-JVM
locks for RAMDirs) it could e.g. lookup the index dir in some map or
whatever. To place the lockfile somewhere else, you should be able to use
FileSwitchDirectory (currently not possible).
Most tests in Lucene use the default (null lock dir in LockFactory), but
some tests for SimpleFSLockFactory & Co use the explicit directory names and
therefore generate MD5 hashes to test the special behavior.
For compatibility reasons we have to still use MD5 (to prevent different
lock file names after Lucene upgrade when FSDir is locked by another JVM
with older Lucene version). For 4.0 I would remove this stupidity and only
allow lock files in index directory.
I hope I explained this stuff so everybody understand it, its really a
little bit confusing (how its implemented), but its "sophisticated
backwards" (haha). I would like to get rid of it and then we have no digest
code anymore.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Shai Erera [mailto:serera@gmail.com]
> Sent: Saturday, December 25, 2010 3:04 PM
> To: dev@lucene.apache.org
> Subject: Re: LuceneTestCase.threadCleanup incorrectly reports left running
> threads
>
> Actually, the MD5 thingy is an attempt to generate a unique temp lock ID,
> IIRC. so this piece of code can disappear entirely now that the tests
> concurrency is better.
>
> As for the other threads that are left running, I couldn't track down yet
the
> warning from the benchmark tests, but I'd love to get rid of those false
> warnings. I thought the stack trace could at least tell us who spawned the
> thread, but obviously it's not always clear.
>
> Shai
>
> On Saturday, December 25, 2010, Robert Muir <rc...@gmail.com> wrote:
> > On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de>
> wrote:
> >> Md5 is guaranteed to be there (like utf8 as charset). This is
documented in
> crypto Api, which algorithms are available for digest.
> >>
> >
> > where is this documented? its not in the javadocs.
> >
> > anyway, we shouldn't be doing this:
> > * this algorithm might not exist on J2ME etc (still java), you need to
> > install an extra crypto add-on.
> > * we shouldnt start up an expensive PKI infrastructure on mac os X,
> > including spawning a new thread, just to hash a string. thats absurd.
> > * we pay all these costs ... for md5! its not even a good hash!
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Robert Muir <rc...@gmail.com>.
On Sat, Dec 25, 2010 at 9:03 AM, Shai Erera <se...@gmail.com> wrote:
> As for the other threads that are left running, I couldn't track down
> yet the warning from the benchmark tests,
I havent looked yet, but i think benchmark depends on/uses the demo
html parser (what a strange dependency)... and the demo html parser
has a problem: https://issues.apache.org/jira/browse/LUCENE-2823
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Shai Erera <se...@gmail.com>.
Actually, the MD5 thingy is an attempt to generate a unique temp lock
ID, IIRC. so this piece of code can disappear entirely now that the
tests concurrency is better.
As for the other threads that are left running, I couldn't track down
yet the warning from the benchmark tests, but I'd love to get rid of
those false warnings. I thought the stack trace could at least tell us
who spawned the thread, but obviously it's not always clear.
Shai
On Saturday, December 25, 2010, Robert Muir <rc...@gmail.com> wrote:
> On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>> Md5 is guaranteed to be there (like utf8 as charset). This is documented in crypto Api, which algorithms are available for digest.
>>
>
> where is this documented? its not in the javadocs.
>
> anyway, we shouldn't be doing this:
> * this algorithm might not exist on J2ME etc (still java), you need to
> install an extra crypto add-on.
> * we shouldnt start up an expensive PKI infrastructure on mac os X,
> including spawning a new thread, just to hash a string. thats absurd.
> * we pay all these costs ... for md5! its not even a good hash!
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Digy <di...@gmail.com>.
Wouldn't something like that be enough?
string hash = Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(dirName));
since MD5 is only used to lower the risk of collision
http://web.archiveorange.com/archive/v/cTKYIAO6UX4CnyDgeEV8
DIGY
-----Original Message-----
From: Simon Willnauer [mailto:simon.willnauer@googlemail.com]
Sent: Saturday, December 25, 2010 6:30 PM
To: dev@lucene.apache.org
Subject: Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
On Sat, Dec 25, 2010 at 3:05 PM, Earwin Burrfoot <ea...@gmail.com> wrote:
> I think we can take some public algos like lookup3 / murmurhash2/3,
> and stuff them into Lucene utils.
>
> Java implementations (very simple and fast ones) exist for both of these.
>
> I.e. lookup3 done by Yonik (http://people.apache.org/~yonik/code/hash/),
> murmurhash2 - by Andrzej Bialecki (
> http://www.getopt.org/murmur/MurmurHash.java ) used in Hadoop?
> non-compatible with original?,
> a version by Derek Young (
> http://dmy999.com/article/50/murmurhash-2-java-port ),
> a version by Ted Dunning for Mahout,
> a random version off Google by Viliam Holub (
> http://d3s.mff.cuni.cz/~holub/sw/javamurmurhash/ )
>
> lookup3 defines versions returning 32/64bit hashes, murmurhash2 -
> 32/64bit, murmurhash3 - 32/64/128bit
Earwin, I like your suggestion - that seems to be very straight
forward and will work on every JVM as far as I can see. No need to
rely on anything else or include a whole crypto lib.
simon
>
> They are just generally useful for many text-processing apps.
>
> On Sat, Dec 25, 2010 at 16:20, Robert Muir <rc...@gmail.com> wrote:
>> On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>>> Md5 is guaranteed to be there (like utf8 as charset). This is documented in crypto Api, which algorithms are available for digest.
>>>
>>
>> where is this documented? its not in the javadocs.
>>
>> anyway, we shouldn't be doing this:
>> * this algorithm might not exist on J2ME etc (still java), you need to
>> install an extra crypto add-on.
>> * we shouldnt start up an expensive PKI infrastructure on mac os X,
>> including spawning a new thread, just to hash a string. thats absurd.
>> * we pay all these costs ... for md5! its not even a good hash!
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
>
>
> --
> Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
> Phone: +7 (495) 683-567-4
> ICQ: 104465785
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Simon Willnauer <si...@googlemail.com>.
On Sat, Dec 25, 2010 at 3:05 PM, Earwin Burrfoot <ea...@gmail.com> wrote:
> I think we can take some public algos like lookup3 / murmurhash2/3,
> and stuff them into Lucene utils.
>
> Java implementations (very simple and fast ones) exist for both of these.
>
> I.e. lookup3 done by Yonik (http://people.apache.org/~yonik/code/hash/),
> murmurhash2 - by Andrzej Bialecki (
> http://www.getopt.org/murmur/MurmurHash.java ) used in Hadoop?
> non-compatible with original?,
> a version by Derek Young (
> http://dmy999.com/article/50/murmurhash-2-java-port ),
> a version by Ted Dunning for Mahout,
> a random version off Google by Viliam Holub (
> http://d3s.mff.cuni.cz/~holub/sw/javamurmurhash/ )
>
> lookup3 defines versions returning 32/64bit hashes, murmurhash2 -
> 32/64bit, murmurhash3 - 32/64/128bit
Earwin, I like your suggestion - that seems to be very straight
forward and will work on every JVM as far as I can see. No need to
rely on anything else or include a whole crypto lib.
simon
>
> They are just generally useful for many text-processing apps.
>
> On Sat, Dec 25, 2010 at 16:20, Robert Muir <rc...@gmail.com> wrote:
>> On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>>> Md5 is guaranteed to be there (like utf8 as charset). This is documented in crypto Api, which algorithms are available for digest.
>>>
>>
>> where is this documented? its not in the javadocs.
>>
>> anyway, we shouldn't be doing this:
>> * this algorithm might not exist on J2ME etc (still java), you need to
>> install an extra crypto add-on.
>> * we shouldnt start up an expensive PKI infrastructure on mac os X,
>> including spawning a new thread, just to hash a string. thats absurd.
>> * we pay all these costs ... for md5! its not even a good hash!
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
>
>
> --
> Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
> Phone: +7 (495) 683-567-4
> ICQ: 104465785
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Earwin Burrfoot <ea...@gmail.com>.
I think we can take some public algos like lookup3 / murmurhash2/3,
and stuff them into Lucene utils.
Java implementations (very simple and fast ones) exist for both of these.
I.e. lookup3 done by Yonik (http://people.apache.org/~yonik/code/hash/),
murmurhash2 - by Andrzej Bialecki (
http://www.getopt.org/murmur/MurmurHash.java ) used in Hadoop?
non-compatible with original?,
a version by Derek Young (
http://dmy999.com/article/50/murmurhash-2-java-port ),
a version by Ted Dunning for Mahout,
a random version off Google by Viliam Holub (
http://d3s.mff.cuni.cz/~holub/sw/javamurmurhash/ )
lookup3 defines versions returning 32/64bit hashes, murmurhash2 -
32/64bit, murmurhash3 - 32/64/128bit
They are just generally useful for many text-processing apps.
On Sat, Dec 25, 2010 at 16:20, Robert Muir <rc...@gmail.com> wrote:
> On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>> Md5 is guaranteed to be there (like utf8 as charset). This is documented in crypto Api, which algorithms are available for digest.
>>
>
> where is this documented? its not in the javadocs.
>
> anyway, we shouldn't be doing this:
> * this algorithm might not exist on J2ME etc (still java), you need to
> install an extra crypto add-on.
> * we shouldnt start up an expensive PKI infrastructure on mac os X,
> including spawning a new thread, just to hash a string. thats absurd.
> * we pay all these costs ... for md5! its not even a good hash!
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
--
Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
Phone: +7 (495) 683-567-4
ICQ: 104465785
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Robert Muir <rc...@gmail.com>.
On Sat, Dec 25, 2010 at 4:04 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Md5 is guaranteed to be there (like utf8 as charset). This is documented in crypto Api, which algorithms are available for digest.
>
where is this documented? its not in the javadocs.
anyway, we shouldn't be doing this:
* this algorithm might not exist on J2ME etc (still java), you need to
install an extra crypto add-on.
* we shouldnt start up an expensive PKI infrastructure on mac os X,
including spawning a new thread, just to hash a string. thats absurd.
* we pay all these costs ... for md5! its not even a good hash!
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Uwe Schindler <uw...@thetaphi.de>.
Md5 is guaranteed to be there (like utf8 as charset). This is documented in crypto Api, which algorithms are available for digest.
Uwe
"Robert Muir" <rc...@gmail.com> schrieb:
>On Thu, Dec 23, 2010 at 2:09 AM, Shai Erera <se...@gmail.com> wrote:
>> I don't know where this Timer is created, but I'll dig more.
>>
>> At any rate, I think your patch is good, and perhaps we should add
>the
>> stacktrace print as well, to help with the debugging?
>>
>> Shai
>
>OK, i'll commit the patch to rid of false positives (threads created
>before our tests even start).
>
>But I'm not sure about the stacktrace, it doesn't seem to be useful?
>
>For example I was trying to debug why we create so many rogue threads
>on Mac OS X, and it looks like:
> [junit] WARNING: test method: 'testDemo' left thread running:
>Thread[Poller SunPKCS11-Darwin,1,main]
> [junit] [java.lang.Thread.sleep(Native Method),
>sun.security.pkcs11.SunPKCS11$TokenPoller.run(SunPKCS11.java:692),
>java.lang.Thread.run(Thread.java:680)]
>
>This doesn't provide any hint that the real reason this thread is
>started is FSDirectory's use of digester for getLockID().
>
>I don't think we should use the java crypto API the way we do in
>FSDirectory. Apparently whether or not it has an algorithm like MD5
>depends on the java implementation (its not guaranteed), so I think we
>should just include MD5Digest from bouncycastle or something like that
>to avoid this.
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>For additional commands, e-mail: dev-help@lucene.apache.org
--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Robert Muir <rc...@gmail.com>.
On Thu, Dec 23, 2010 at 2:09 AM, Shai Erera <se...@gmail.com> wrote:
> I don't know where this Timer is created, but I'll dig more.
>
> At any rate, I think your patch is good, and perhaps we should add the
> stacktrace print as well, to help with the debugging?
>
> Shai
OK, i'll commit the patch to rid of false positives (threads created
before our tests even start).
But I'm not sure about the stacktrace, it doesn't seem to be useful?
For example I was trying to debug why we create so many rogue threads
on Mac OS X, and it looks like:
[junit] WARNING: test method: 'testDemo' left thread running:
Thread[Poller SunPKCS11-Darwin,1,main]
[junit] [java.lang.Thread.sleep(Native Method),
sun.security.pkcs11.SunPKCS11$TokenPoller.run(SunPKCS11.java:692),
java.lang.Thread.run(Thread.java:680)]
This doesn't provide any hint that the real reason this thread is
started is FSDirectory's use of digester for getLockID().
I don't think we should use the java crypto API the way we do in
FSDirectory. Apparently whether or not it has an algorithm like MD5
depends on the java implementation (its not guaranteed), so I think we
should just include MD5Digest from bouncycastle or something like that
to avoid this.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Shai Erera <se...@gmail.com>.
I ran the test from both eclipse and Ant, and got similar warnings.
With your patch most of the 'false alarms' do not show up again, but I still
see a strange failure. I add this to after the System.err.print("left thread
running"): System.err.println(Arrays.toString(t.getStackTrace())); -- it
prints the stack trace. And here is what I get:
[junit] ------------- Standard Error -----------------
[junit] WARNING: test method: 'testIndexAndSearchTasks' left thread
running: Thread[file lock watchdog,6,main]
[junit] [java.lang.Object.wait(Native Method),
java.lang.Object.wait(Object.java:167),
java.util.Timer$TimerImpl.run(Timer.java:226)]
[junit] RESOURCE LEAK: test method: 'testIndexAndSearchTasks' left 1
thread(s) running
[junit] NOTE: reproduce with: ant test -Dtestcase=TestPerfTasksLogic
-Dtestmethod=testIndexAndSearchTasks
-Dtests.seed=-792089523312439823:1164084411683706634
I don't know where this Timer is created, but I'll dig more.
At any rate, I think your patch is good, and perhaps we should add the
stacktrace print as well, to help with the debugging?
Shai
On Wed, Dec 22, 2010 at 9:35 PM, Robert Muir <rc...@gmail.com> wrote:
> static {
> + // just a hack for things like eclipse threads
> + for (Thread t : Thread.getAllStackTraces().keySet()) {
> + rogueThreads.put(t, true);
> + }
> + }
>
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Robert Muir <rc...@gmail.com>.
here is a (imperfect) patch for eclipse, can you try this? any threads
running at this point are not our own.
Index: lucene/src/test/org/apache/lucene/util/LuceneTestCase.java
===================================================================
--- lucene/src/test/org/apache/lucene/util/LuceneTestCase.java
(revision 1051872)
+++ lucene/src/test/org/apache/lucene/util/LuceneTestCase.java (working copy)
@@ -522,6 +522,13 @@
// jvm-wide list of 'rogue threads' we found, so they only get reported once.
private final static IdentityHashMap<Thread,Boolean> rogueThreads =
new IdentityHashMap<Thread,Boolean>();
+ static {
+ // just a hack for things like eclipse threads
+ for (Thread t : Thread.getAllStackTraces().keySet()) {
+ rogueThreads.put(t, true);
+ }
+ }
+
/**
* Looks for leftover running threads, trying to kill them off,
* so they don't fail future tests.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: LuceneTestCase.threadCleanup incorrectly reports left running threads
Posted by Robert Muir <rc...@gmail.com>.
On Wed, Dec 22, 2010 at 2:14 PM, Shai Erera <se...@gmail.com> wrote:
> Hi
>
> I noticed that some tests report threads are left running, even when those
> tests never create and start a Thread. Digging deeper I found out that the
> tests report "Signal Dispatcher" and "Attach handler" as two threads that
> are left running. If I run the test from eclipse, then a "ReaderThread" and
> "Signal Dispatcher" are reported. ReaderThread belongs to JUnit framework
> and the other two are initiated by some framework, and definitely not from
> our tests.
>
> So I was thinking if instead of reporting those threads, we should inspect
> each running Thread's stacktrace and report it only if it contains an
> "org.apache.lucene/solr" package. Otherwise it cannot be started from our
> tests?
>
> What do you think?
are you running the tests from eclipse or something in this case (i
think i've seen these from eclipse)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org