JVM Crash in Lucene

[Dan Gould <dl...@cs.brown.edu>]

Hi--

I'm relatively new to Lucene.  When I run my app, I get a JVM error. 
This gets called a lot, but only fails every once in awhile (maybe 1 in 
100 calls?)

I filed a report with Sun, but I don't expect to hear anything from them. 
So, I was wondering if any Lucene experts have run across this or have any 
ideas for workarounds (if not, I'll have to take lucene out of my system 
and manage the termvectors myself):

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0x0000002a95a5c300, pid=26545, tid=1085655392
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# V  [libjvm.so+0x4fa300]
#
# An error report file with more information is saved as 
hs_err_pid26545.log

That error file is huge, but the salient part seems to be:

Stack: [0x0000000040a5c000,0x0000000040b5d000),  sp=0x0000000040b59f70, 
free sp
ace=1015k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native 
code)
V  [libjvm.so+0x4fa300]
V  [libjvm.so+0x4fa3c2]
V  [libjvm.so+0x4fa876]
V  [libjvm.so+0x58051e]
V  [libjvm.so+0x58017c]
V  [libjvm.so+0x2b968b]
V  [libjvm.so+0x2b67ba]
V  [libjvm.so+0x25f3c7]
V  [libjvm.so+0x2bed06]
V  [libjvm.so+0x2be6b2]
V  [libjvm.so+0x6163f9]
V  [libjvm.so+0x610ffe]
V  [libjvm.so+0x554745]


Current CompileTask:
opto:1621 
org.apache.lucene.index.IndexReader$1.doBody()Ljava/lang/Object;
(99 bytes)

My code that is causing the JVM crash is:

         //Get the terms and frequencies
         IndexReader indexreader = null;
         TermFreqVector tv = null;
         try {
             indexreader = indexManager.getIndexReader();
             tv = indexManager.getTermVector(indexreader, item);
             if(null==tv) {
                 if(log.isErrorEnabled()) log.error("No term vector for item " + item.getId());
                 //??? better handling
                 return;
             }
         }
         finally {
             //XXX should close earlier
             if(null!=indexreader) {
                 try {
                     indexreader.close();
                 }
                 catch(IOException ex) {
                     if(log.isDebugEnabled()) log.debug("IOException cleaning up indexreader ", ex);
                 }
             }
         }

         //Replace the String[] with its equivalent long[]... (and tally up total occurs)
         String[] termstrings = tv.getTerms();
         int[] freqs = tv.getTermFrequencies();

Thank you for your help,
Dan

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: JVM Crash in Lucene

[Yonik Seeley <ys...@gmail.com>]

The only problems I've had with 1.5 JVM crashes and Lucene was related
to stack overflow... try increasing the stack size and see of anything
different happens.

My crashes happened while trying to use Luke to open a 4GB index with
thousands of indexed fields.

-Yonik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: JVM Crash in Lucene

[Chris Hostetter <ho...@fucit.org>]

: I'm relatively new to Lucene.  When I run my app, I get a JVM error.
: This gets called a lot, but only fails every once in awhile (maybe 1 in
: 100 calls?)

i'm not that familiar with TermFreqVectors, and I have no idea what
indexManager is, but I'm suprised this works at all ... I thought calling
the getters on a TermFreqVector required it to read from the underlying
files (which would require that the reader still be open).

Is it possible that that's your problem? .. is there perhaps some strange
GC interaction issue that results in the data TermFreqVector is looking
for to go bye-bye because you closed the reader?

(it sounds silly when I read it back to myself ... but it's the best guess
i've got just reading your code)

: My code that is causing the JVM crash is:
:
:          //Get the terms and frequencies
:          IndexReader indexreader = null;
:          TermFreqVector tv = null;
:          try {
:              indexreader = indexManager.getIndexReader();
:              tv = indexManager.getTermVector(indexreader, item);
:              if(null==tv) {
:                  if(log.isErrorEnabled()) log.error("No term vector for item " + item.getId());
:                  //??? better handling
:                  return;
:              }
:          }
:          finally {
:              //XXX should close earlier
:              if(null!=indexreader) {
:                  try {
:                      indexreader.close();
:                  }
:                  catch(IOException ex) {
:                      if(log.isDebugEnabled()) log.debug("IOException cleaning up indexreader ", ex);
:                  }
:              }
:          }
:
:          //Replace the String[] with its equivalent long[]... (and tally up total occurs)
:          String[] termstrings = tv.getTerms();
:          int[] freqs = tv.getTermFrequencies();




-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org