You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Lourival Júnior <ju...@gmail.com> on 2006/08/03 16:33:31 UTC
NullPointException
Why when I delete some segments that reach the
db.default.fetcth.intervalthe search application gets the
nullPointerException? Periodically I have to
recrawl my Site. And delete old segments is a problem. Someone have a
suggestion?
Regards
--
Lourival Junior
Universidade Federal do Pará
Curso de Bacharelado em Sistemas de Informação
http://www.ufpa.br/cbsi
Msn: junior_ufpa@hotmail.com
Re: NullPointException
Posted by Matthew Holt <mh...@redhat.com>.
Couldn't you just merge all the segments into one, then reindex the one
segment and delete the rest?
Or does reindexing not generate a fresh index, but just adds the new
indexed material the old index (that still contains references to the
old segments)?
Marko Bauhardt wrote:
>
> Am 03.08.2006 um 18:52 schrieb Lourival Júnior:
>
>>
>>
>> My questions:
>> Why it occurs? How can I know which segments can be deleted?
>>
>
> You must know which segment are indexed. You can not index all
> segments and after that delete these segments.
> The Indexer index the name of the segment that the searcher knows
> where is the parsed text for a given indexed Term. But the segment
> name is only stored. You can not search on a segment name. But you can
> write some lines of code that prints the segment names which are
> stored in index. After that you know which segment are indexed. But i
> think this way is not really cool.
>
> Sorry
> Marko
>
>
>
Re: NullPointException
Posted by Marko Bauhardt <mb...@media-style.com>.
Am 03.08.2006 um 18:52 schrieb Lourival Júnior:
>
>
> My questions:
> Why it occurs? How can I know which segments can be deleted?
>
You must know which segment are indexed. You can not index all
segments and after that delete these segments.
The Indexer index the name of the segment that the searcher knows
where is the parsed text for a given indexed Term. But the segment
name is only stored. You can not search on a segment name. But you
can write some lines of code that prints the segment names which are
stored in index. After that you know which segment are indexed. But i
think this way is not really cool.
Sorry
Marko
Re: NullPointException
Posted by Lourival Júnior <ju...@gmail.com>.
All right. Take a look to this output of the segread command:
060803 132735 PARSED? STARTED FINISHED
COUNT DIR NAME
060803 132735 true 20060717-14:41:58 20060717-14:41:58
1 crawl-legislacao_copia/segments/20060717144154
060803 132735 true 20060717-14:42:03 20060717-14:43:22
77 crawl-legislacao_copia/segments/20060717144201
060803 132735 true 20060717-14:43:29 20060717-15:08:10
1464 crawl-legislacao_copia/segments/20060717144327
060803 132735 true 20060717-15:08:17 20060717-15:11:58
223 crawl-legislacao_copia/segments/20060717150815
060803 132736 true 20060718-09:02:56 20060718-09:03:10
5 crawl-legislacao_copia/segments/20060718090250
060803 132736 true 20060803-10:55:18 20060803-12:53:49
1541 crawl-legislacao_copia/segments/20060803105509
060803 132736 true 20060803-13:07:15 20060803-13:07:20
4 crawl-legislacao_copia/segments/20060803130707
060803 132736 TOTAL: 3315 entries in 7 segments.
My db.default.fetch.interval is 15. Before I run a recrawl script I had 5
segments ( 200607* ) and the Index points to 1537 documents. After run the
recrawl 2 segments was created and then the script index all. When I
analyzed the index generated I see it had 1541 documents. But how can you
see the segments 200607* are old and can be deleted. I done this:
rm -rf segments/200607*
Then I get de NPE. I right I must to re-index the 2 remain segments. I've
done this. So, I analize again the index and it has only 1417!
My questions:
Why it occurs? How can I know which segments can be deleted?
I hope you can help me
On 8/3/06, Marko Bauhardt <mb...@media-style.com> wrote:
>
> Hi,
> if you delete segments then be sure that you doesnt have an index
> from this segment.
> The segment contains the parsed content and the index is the index
> from this content. If you delete the segment and you doing a search
> on this index, a NPE occurs because no summary (parsed content) are
> found.
>
>
> HTH
> Marko
>
>
>
> Am 03.08.2006 um 16:33 schrieb Lourival Júnior:
>
> > Why when I delete some segments that reach the
> > db.default.fetcth.intervalthe search application gets the
> > nullPointerException? Periodically I have to
> > recrawl my Site. And delete old segments is a problem. Someone have a
> > suggestion?
> >
> > Regards
> >
> > --
> > Lourival Junior
> > Universidade Federal do Pará
> > Curso de Bacharelado em Sistemas de Informação
> > http://www.ufpa.br/cbsi
> > Msn: junior_ufpa@hotmail.com
>
>
--
Lourival Junior
Universidade Federal do Pará
Curso de Bacharelado em Sistemas de Informação
http://www.ufpa.br/cbsi
Msn: junior_ufpa@hotmail.com
Re: NullPointException
Posted by Marko Bauhardt <mb...@media-style.com>.
Hi,
if you delete segments then be sure that you doesnt have an index
from this segment.
The segment contains the parsed content and the index is the index
from this content. If you delete the segment and you doing a search
on this index, a NPE occurs because no summary (parsed content) are
found.
HTH
Marko
Am 03.08.2006 um 16:33 schrieb Lourival Júnior:
> Why when I delete some segments that reach the
> db.default.fetcth.intervalthe search application gets the
> nullPointerException? Periodically I have to
> recrawl my Site. And delete old segments is a problem. Someone have a
> suggestion?
>
> Regards
>
> --
> Lourival Junior
> Universidade Federal do Pará
> Curso de Bacharelado em Sistemas de Informação
> http://www.ufpa.br/cbsi
> Msn: junior_ufpa@hotmail.com