You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Stas Chetvertkov <sc...@oilspace.com> on 2002/10/01 18:16:00 UTC
Multithread searching problem on Linux
Hi All,
I am building a search engine based on Lucene. Recently I created a test
simulating multiple users searching in the same index simultaneously and
found out that quite often JVM crashes with 'Hotspot Virtual Machine Error :
11'. I couldnot reproduce this bug on Windows box, but observed it a lot on
Red Hat Linux 7.3 with different versions of Sun's 1.3 JVM, including the
most recent one (1.3.1_04 at the moment).
I am attaching a simple test that generates hotspot error in 90% of cases.
In our code we have to create new IndexSearcher for every search because
indices are updated in real time.
The only workaround I found for this problem so far is reducing the number
of searching threads which doesnot seem to be a good solution.
Had anyone encountered problems like this one?
Regards,
Stas.
Re: Multithread searching problem on Linux
Posted by karl øie <ka...@gan.no>.
there have been numerous problems/bad features with the hot-spot mode
on sun's linux vm, the reason for this is that hot-spotting is
optimizing your code by doing very weird stuff with your code :-)
anyway i'm glad -classic works well for you. the bad performance is a
known problem with the linux-kernel java threading system. When it
comes to threads windows jvms outperforms linux jvms because of kernel
internal things i don't even try to understand...
i have used the 1.3.1 linux jvm from ibm with great stability, how does
the ibm jvm perform against the sun jvm when it comes to thread
performance?
there is also a 1.3 jvm from a group called "blackdown" that is free
and optimized for linux. there was some talking in the news about it
being very good at threading... you could try it.. (
http://www.blackdown.org/ )
mvh karl øie
On onsdag, okt 2, 2002, at 15:34 Europe/Oslo, Stas Chetvertkov wrote:
> Yes, it works without errors with classic JVM, but if it was not so
> painfully slow :(
>
> Anyway, I'll check what is faster - classic JVM with multiple thread
> search
> or Hotspot
> with 1 searching thread (as we have now).
>
> Thanks,
> Stas.
>
>> Try to run your vm in classic mode "java -classic" to disable the
>> hotspot features...
>
>> mvh karl øie
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Multithread searching problem on Linux
Posted by karl øie <ka...@gan.no>.
if you still have problems, take a look at this note found in the
newest tomcat release... it might help.
mvh karl øie
> -------------------------------
> Linux and Sun JDK 1.2.x - 1.3.x:
> -------------------------------
>
> Virtual machine crashes can be experienced when using certain
combinations of
> kernel / glibc under Linux with Sun Hotspot 1.2 to 1.3. The crashes
were
> reported to occur mostly on startup. Sun JDK 1.4 does not exhibit the
problems,
> and neither does IBM JDK for Linux.
>
> The problems can be fixed by reducing the default stack size. At bash
shell,
> do "ulimit -s 2048"; use "limit stacksize 2048" for tcsh.
>
> GLIBC 2.2 / Linux 2.4 users should also define an environment
variable:
> export LD_ASSUME_KERNEL=2.2.5
On onsdag, okt 2, 2002, at 15:34 Europe/Oslo, Stas Chetvertkov wrote:
> Yes, it works without errors with classic JVM, but if it was not so
> painfully slow :(
>
> Anyway, I'll check what is faster - classic JVM with multiple thread
> search
> or Hotspot
> with 1 searching thread (as we have now).
>
> Thanks,
> Stas.
>
>> Try to run your vm in classic mode "java -classic" to disable the
>> hotspot features...
>
>> mvh karl øie
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Multithread searching problem on Linux
Posted by Stas Chetvertkov <sc...@oilspace.com>.
Yes, it works without errors with classic JVM, but if it was not so
painfully slow :(
Anyway, I'll check what is faster - classic JVM with multiple thread search
or Hotspot
with 1 searching thread (as we have now).
Thanks,
Stas.
> Try to run your vm in classic mode "java -classic" to disable the
> hotspot features...
> mvh karl øie
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Multithread searching problem on Linux
Posted by karl øie <ka...@gan.no>.
Try to run your vm in classic mode "java -classic" to disable the
hotspot features...
mvh karl øie
On tirsdag, okt 1, 2002, at 18:16 Europe/Oslo, Stas Chetvertkov wrote:
> Hi All,
>
> I am building a search engine based on Lucene. Recently I created a
> test
> simulating multiple users searching in the same index simultaneously
> and
> found out that quite often JVM crashes with 'Hotspot Virtual Machine
> Error :
> 11'. I couldnot reproduce this bug on Windows box, but observed it a
> lot on
> Red Hat Linux 7.3 with different versions of Sun's 1.3 JVM, including
> the
> most recent one (1.3.1_04 at the moment).
>
> I am attaching a simple test that generates hotspot error in 90% of
> cases.
> In our code we have to create new IndexSearcher for every search
> because
> indices are updated in real time.
>
> The only workaround I found for this problem so far is reducing the
> number
> of searching threads which doesnot seem to be a good solution.
>
> Had anyone encountered problems like this one?
>
> Regards,
> Stas.
> <SearchTest.java>--
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Multithread searching problem on Linux
Posted by Stas Chetvertkov <sc...@oilspace.com>.
> Thursday? :)
quarter ;)
> I haven't encountered this with Lucene, but I have seen it with a
> number of other applications.
> These are JVM problems.
>
> Try IBM's 1.3.1 JVM for Linux, or the recently released JRockit 7.0.
Everything works perfectly on both IBM's JVM and Sun's 1.4. Unfortunately we
have a complicated working production system, and moving the whole thing to
another JVM and making sure that everything works fine would take a while.
Eventually we'll move to 1.4, but not now.
This is surely Sun's problem. I had sent them a bug report, but it takes for
them up to 3 weeks to response.
I just hoped that someone encountered similar problem and found a good
solution :)
Thanks,
Stas.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Multithread searching problem on Linux
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Thursday? :)
I haven't encountered this with Lucene, but I have seen it with a
number of other applications.
These are JVM problems.
Try IBM's 1.3.1 JVM for Linux, or the recently released JRockit 7.0.
Otis
--- Stas Chetvertkov <sc...@oilspace.com> wrote:
> Hi All,
>
> I am building a search engine based on Lucene. Recently I created a
> test
> simulating multiple users searching in the same index simultaneously
> and
> found out that quite often JVM crashes with 'Hotspot Virtual Machine
> Error :
> 11'. I couldnot reproduce this bug on Windows box, but observed it a
> lot on
> Red Hat Linux 7.3 with different versions of Sun's 1.3 JVM, including
> the
> most recent one (1.3.1_04 at the moment).
>
> I am attaching a simple test that generates hotspot error in 90% of
> cases.
> In our code we have to create new IndexSearcher for every search
> because
> indices are updated in real time.
>
> The only workaround I found for this problem so far is reducing the
> number
> of searching threads which doesnot seem to be a good solution.
>
> Had anyone encountered problems like this one?
>
> Regards,
> Stas.
>
> ATTACHMENT part 2 application/octet-stream name=SearchTest.java
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
__________________________________________________
Do you Yahoo!?
New DSL Internet Access from SBC & Yahoo!
http://sbc.yahoo.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>