Some questions about index hard commit and intellij dev setup

[Huaxiang Sun <hu...@gmail.com>]

Hi Devs,

    I am a newbie to solr/lucene project and have some questions about
index hard commit. Excuse me if these have been asked before.

    1. When hard commit happens, will it drain up entries in the index
queue?

    2. How exactly is index file written? I.e, will they be written to tmp
dir and moved to the index dir when it completes, or they are written to
the index dir directly. In the later case, if one is reading the index dir,
then it can read incomplete index files.

   3. Similar question to index merge. Will the merge process create merged
file in tmp dir and moved to index dir after merge completes? When are
these files merged deleted? Will these merged files be moved to some
archive dir and cleaned up later or deleted right after the merge?

   The final question is about intellij setup for lucene/solr project. I
followed the steps in doc and it seems that the code browsing/build does
not work well for me. Just want to check that these are steps I need to
follow.

   Thanks

    Huaxiang Sun

Re: Some questions about index hard commit and intellij dev setup

[Erick Erickson <er...@gmail.com>]

1> No. There's really no "index queue". Current batches being
processed are finished, then the commit happens.

2,3> The key is the "segments" file in your index, there should only
be one of those. At time T, the segments file "points"
to, say, segments 1, 2, 3. Now segments 4, 5 and 6 are written (but no
commit). If you killed  Solr with, say, a kill -9 or
Solr crashed, you'd only see segments 1, 2, 3.

Contrast that with a commit. The last act after closing all current
segments is to rewrite the segments_n
file to include segments 1, 2, 3, 4, 5, 6.

Or say you had segments 1, 2, 3 as above and segments 2 and 3 were
merged into segment 4. After the merge
was completed, the segments file would point to segments 1 and 4, and
after it was safely written segments 2 and 3
would be deleted.

final question: How is it "not working"? Once you execute the "ant
idea" command successfully, you should just
be able to open the project from IntelliJ. Occasionally I have to run
an "ant clean-idea idea" to start over,
but I do this very often.

In general, when i debug Solr I don't try to run Solr from within
intellij. I build it externally (execute 'ant server'
in the Solr directory then start it up). Then I debug Solr remotely.
If I'm trying to debug something specific,
it's often easiest to do from a unit test. I don't try to build the
entire project from within IntelliJ although I think
others do. Mostly I just don't want to have one extra variable in the
equation...

FWIW,
Erick

On Thu, Feb 7, 2019 at 12:34 PM Huaxiang Sun <hu...@gmail.com> wrote:
>
> Hi Devs,
>
>     I am a newbie to solr/lucene project and have some questions about index hard commit. Excuse me if these have been asked before.
>
>     1. When hard commit happens, will it drain up entries in the index queue?
>
>     2. How exactly is index file written? I.e, will they be written to tmp dir and moved to the index dir when it completes, or they are written to the index dir directly. In the later case, if one is reading the index dir, then it can read incomplete index files.
>
>    3. Similar question to index merge. Will the merge process create merged file in tmp dir and moved to index dir after merge completes? When are these files merged deleted? Will these merged files be moved to some archive dir and cleaned up later or deleted right after the merge?
>
>    The final question is about intellij setup for lucene/solr project. I followed the steps in doc and it seems that the code browsing/build does not work well for me. Just want to check that these are steps I need to follow.
>
>    Thanks
>
>     Huaxiang Sun

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org