You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Petr Pytelka <Pe...@lightcomp.com> on 2006/05/29 14:45:41 UTC
Problem with indexes
Hi all,
I'm testing performance of Jackrabbit and found one issue.
Test case:
1. Insert N items
2. Search each of inserted items (using Query)
3. Drop inserted item
When N is lower then aprox.1000 items every thinks works fine. But when
N is higher then aprox. 1000 items Jackrabbit (Lucene) some how "lost"
indexes and I'm not able to find required item (usualy I'm able to find
first 300 items).
If I delete index directory and re-run search, Jackrabbit build new
indexes during startup and I'm able to find all items.
Lucene: version 1.4.3
DB-backend: Derby
Any idea where can be problem ?
Thanks a lot, Petr Pytelka
Re: Problem with indexes
Posted by Petr Pytelka <Pe...@lightcomp.com>.
Thanks a lot for your answer.
I've created "ProxySearch" engine for Lucene and was looking on
parameters in method "updateNodes" and every thinks looks like ok. Also
for nodes which cannot be find.
Problem is probably somewhere in the Lucene implementation or in Lucene
itself (less probable)....
Petr
Marcel Reutegger wrote:
> Hi Petr,
>
> thanks for the test case. The query does indeed return a wrong result at
> some point. I'll have a look at it asap.
>
> regards
> marcel
>
> Petr Pytelka wrote:
>> Hi Marcel,
>> I've tried to find out more about my issue but didn't succeed.
>>
>> There is small application which demonstrate the problem:
>> http://217.11.254.42/download/index-problem.zip
>>
>> Steps to reproduce:
>>
>> Prerequisites:
>> 1. This test is resource consuming - need computer with at least 512
>> MB, better 1 GB
>> 2. Package contain all needed libs and also source code (10kb) - I was
>> not able to create smaller example - sorry for that.
>>
>> Test can be run using script "run-test" or "run-test.bat" - at the
>> beginning of each file is path to the Java.
>>
>> Whole test is trying to create 10000 nodes
>> - Phase 1: Create 20 nodes
>> - Phase 2: Query for last created 20 nodes
>>
>> Problem is that on faster machines this test fail around 3000-5000th
>> document - it is not possible to locate file. Query in repository is
>> based on the attribute values.
>>
>> Maybe that the problem is in my code.. I'll really appreciate if you
>> can look on it...
>>
>> Thanks, Petr
>>
>
Re: Problem with indexes
Posted by Marcel Reutegger <ma...@gmx.net>.
Hi Petr,
thanks for the test case. The query does indeed return a wrong result at
some point. I'll have a look at it asap.
regards
marcel
Petr Pytelka wrote:
> Hi Marcel,
> I've tried to find out more about my issue but didn't succeed.
>
> There is small application which demonstrate the problem:
> http://217.11.254.42/download/index-problem.zip
>
> Steps to reproduce:
>
> Prerequisites:
> 1. This test is resource consuming - need computer with at least 512 MB,
> better 1 GB
> 2. Package contain all needed libs and also source code (10kb) - I was
> not able to create smaller example - sorry for that.
>
> Test can be run using script "run-test" or "run-test.bat" - at the
> beginning of each file is path to the Java.
>
> Whole test is trying to create 10000 nodes
> - Phase 1: Create 20 nodes
> - Phase 2: Query for last created 20 nodes
>
> Problem is that on faster machines this test fail around 3000-5000th
> document - it is not possible to locate file. Query in repository is
> based on the attribute values.
>
> Maybe that the problem is in my code.. I'll really appreciate if you can
> look on it...
>
> Thanks, Petr
>
Re: Problem with indexes
Posted by Alexandru Popescu <th...@gmail.com>.
Thanks a lot Jukka. I wasn't aware of this feature of JIRA. It is a great one.
./alex
--
.w( the_mindstorm )p.
On 6/1/06, Jukka Zitting <ju...@gmail.com> wrote:
> Hi,
>
> On 5/31/06, Alexandru Popescu <th...@gmail.com> wrote:
> > Marcel, I am wondering if this fix can be easily incorporated in the
> > 1.0.1 release. And I mean, not to change the distribution, but myself
> > to manually patch it.
>
> The fix for JCR-448 is available at:
>
> http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/MultiIndex.java?r1=410647&r2=410646&pathrev=410647&view=patch
>
> or at http://tinyurl.com/p7rjx if the above line gets broken. The
> patch should apply cleanly to 1.0.1.
>
> PS. The above link can be found from Jira by selecting the "Subversion
> commits" tab and following the link to the changed file. The diff page
> contains a "Patch" link that gives you the change in plain text
> unified diff format.
>
> BR,
>
> Jukka Zitting
>
> --
> Yukatan - http://yukatan.fi/ - info@yukatan.fi
> Software craftsmanship, JCR consulting, and Java development
>
Re: Problem with indexes
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On 5/31/06, Alexandru Popescu <th...@gmail.com> wrote:
> Marcel, I am wondering if this fix can be easily incorporated in the
> 1.0.1 release. And I mean, not to change the distribution, but myself
> to manually patch it.
The fix for JCR-448 is available at:
http://svn.apache.org/viewvc/jackrabbit/trunk/jackrabbit/src/main/java/org/apache/jackrabbit/core/query/lucene/MultiIndex.java?r1=410647&r2=410646&pathrev=410647&view=patch
or at http://tinyurl.com/p7rjx if the above line gets broken. The
patch should apply cleanly to 1.0.1.
PS. The above link can be found from Jira by selecting the "Subversion
commits" tab and following the link to the changed file. The diff page
contains a "Patch" link that gives you the change in plain text
unified diff format.
BR,
Jukka Zitting
--
Yukatan - http://yukatan.fi/ - info@yukatan.fi
Software craftsmanship, JCR consulting, and Java development
Re: Problem with indexes
Posted by Alexandru Popescu <th...@gmail.com>.
Marcel, I am wondering if this fix can be easily incorporated in the
1.0.1 release. And I mean, not to change the distribution, but myself
to manually patch it.
tia,
./alex
--
.w( the_mindstorm )p.
On 5/31/06, Marcel Reutegger <ma...@gmx.net> wrote:
> Hi Petr,
>
> There was a problem in the index merger which might be the cause of the
> query issue you discovered. I've created a jira issue:
> http://issues.apache.org/jira/browse/JCR-448
>
> It's already fixed in the jackrabbit trunk. If you want to test if the
> fix works for you, you have to check out the sources and compile them,
> or you can use this link to download a recent build that includes the
> mentioned change: http://service.gmx.net/mc/MVJEVor3l4uE6QCh7huUNpoLzc2xqv
>
> Thanks again for your help
>
> regards
> marcel
>
> Petr Pytelka wrote:
> > Hi Marcel,
> > I've tried to find out more about my issue but didn't succeed.
> >
> > There is small application which demonstrate the problem:
> > http://217.11.254.42/download/index-problem.zip
> >
> > Steps to reproduce:
> >
> > Prerequisites:
> > 1. This test is resource consuming - need computer with at least 512 MB,
> > better 1 GB
> > 2. Package contain all needed libs and also source code (10kb) - I was
> > not able to create smaller example - sorry for that.
> >
> > Test can be run using script "run-test" or "run-test.bat" - at the
> > beginning of each file is path to the Java.
> >
> > Whole test is trying to create 10000 nodes
> > - Phase 1: Create 20 nodes
> > - Phase 2: Query for last created 20 nodes
> >
> > Problem is that on faster machines this test fail around 3000-5000th
> > document - it is not possible to locate file. Query in repository is
> > based on the attribute values.
> >
> > Maybe that the problem is in my code.. I'll really appreciate if you can
> > look on it...
> >
> > Thanks, Petr
>
>
Re: Problem with indexes
Posted by Marcel Reutegger <ma...@gmx.net>.
Hi Petr,
There was a problem in the index merger which might be the cause of the
query issue you discovered. I've created a jira issue:
http://issues.apache.org/jira/browse/JCR-448
It's already fixed in the jackrabbit trunk. If you want to test if the
fix works for you, you have to check out the sources and compile them,
or you can use this link to download a recent build that includes the
mentioned change: http://service.gmx.net/mc/MVJEVor3l4uE6QCh7huUNpoLzc2xqv
Thanks again for your help
regards
marcel
Petr Pytelka wrote:
> Hi Marcel,
> I've tried to find out more about my issue but didn't succeed.
>
> There is small application which demonstrate the problem:
> http://217.11.254.42/download/index-problem.zip
>
> Steps to reproduce:
>
> Prerequisites:
> 1. This test is resource consuming - need computer with at least 512 MB,
> better 1 GB
> 2. Package contain all needed libs and also source code (10kb) - I was
> not able to create smaller example - sorry for that.
>
> Test can be run using script "run-test" or "run-test.bat" - at the
> beginning of each file is path to the Java.
>
> Whole test is trying to create 10000 nodes
> - Phase 1: Create 20 nodes
> - Phase 2: Query for last created 20 nodes
>
> Problem is that on faster machines this test fail around 3000-5000th
> document - it is not possible to locate file. Query in repository is
> based on the attribute values.
>
> Maybe that the problem is in my code.. I'll really appreciate if you can
> look on it...
>
> Thanks, Petr
Re: Problem with indexes
Posted by Alexandru Popescu <th...@gmail.com>.
With 1.5.0_06 and -server the JVM crashes from time to time. If
-server is removed, than it works with 1.5.0_06 oke.
./alex
--
.w( the_mindstorm )p.
On 6/6/06, Marcel Reutegger <ma...@gmx.net> wrote:
> I also used 1.5.0_07 and the test run fine. I actually modified the test
> slightly to create 1/2 million nodes. memory consumption was around 70
> megs at the end of the test.
>
> regards
> marcel
>
> Petr Pytelka wrote:
> > Hi,
> > test passed when I applied JCR-448 and JCR-450 on 1.0.1, using SUN JVM
> > 1.5.0_07.
> >
> > Thanks, Petr
> >
> > Alexandru Popescu wrote:
> >> A quick update (probably I will update it once more later).
> >>
> >> I have run Test3 with JDK1.5.0_06 and JRockit1.5.0_06 and with the
> >> following configs:
> >>
> >> 1/ jackrabbit-1.0: consistently fails between 700 - 1500 documents
> >> 2/ jackrabbit-1.0.1 (with patch JCR-448 included)
> >> 2.1/ with JDK1.5.0_06: results in a JVM crash at about 13000-15000
> >> documents
> >> 2.2/ with JRockit1.5.0_06: the only one that finished the test
> >>
>
>
Re: Problem with indexes
Posted by Marcel Reutegger <ma...@gmx.net>.
I also used 1.5.0_07 and the test run fine. I actually modified the test
slightly to create 1/2 million nodes. memory consumption was around 70
megs at the end of the test.
regards
marcel
Petr Pytelka wrote:
> Hi,
> test passed when I applied JCR-448 and JCR-450 on 1.0.1, using SUN JVM
> 1.5.0_07.
>
> Thanks, Petr
>
> Alexandru Popescu wrote:
>> A quick update (probably I will update it once more later).
>>
>> I have run Test3 with JDK1.5.0_06 and JRockit1.5.0_06 and with the
>> following configs:
>>
>> 1/ jackrabbit-1.0: consistently fails between 700 - 1500 documents
>> 2/ jackrabbit-1.0.1 (with patch JCR-448 included)
>> 2.1/ with JDK1.5.0_06: results in a JVM crash at about 13000-15000
>> documents
>> 2.2/ with JRockit1.5.0_06: the only one that finished the test
>>
Re: Problem with indexes
Posted by Petr Pytelka <Pe...@lightcomp.com>.
Hi,
test passed when I applied JCR-448 and JCR-450 on 1.0.1, using SUN JVM
1.5.0_07.
Thanks, Petr
Alexandru Popescu wrote:
> A quick update (probably I will update it once more later).
>
> I have run Test3 with JDK1.5.0_06 and JRockit1.5.0_06 and with the
> following configs:
>
> 1/ jackrabbit-1.0: consistently fails between 700 - 1500 documents
> 2/ jackrabbit-1.0.1 (with patch JCR-448 included)
> 2.1/ with JDK1.5.0_06: results in a JVM crash at about 13000-15000
> documents
> 2.2/ with JRockit1.5.0_06: the only one that finished the test
>
> Unfortunately, I have run these test last night at about 5am, so I
> will completely redo it today and update this thread. My plan is to
> run the Test3 against jackrabbit-1.0, jackrabbit-1.0.1 and
> jackrabbit-1.0.1a (containing the patches for JCR-448 and JCR-450)
> with SUN JDK. If this will consistently fail (as above), than rerun
> all tests with JRockit 1.5.0_06.
>
> hth,
>
> ./alex
> --
> .w( the_mindstorm )p.
>
>
> On 6/3/06, Alexandru Popescu <th...@gmail.com> wrote:
>> I have checked the jars at the last zip you provided, but the
>> jackrabbit-core seems to not include the patch for JCR-448 (which is
>> supposed to fix the problem).
>>
>> ./alex
>> --
>> .w( the_mindstorm )p.
>>
>>
>> On 6/3/06, Alexandru Popescu <th...@gmail.com> wrote:
>> > Petr is the initial problem still valid if you use the patched
>> > (JCR-448) 1.0.1 release?
>> >
>> > ./alex
>> > --
>> > .w( the_mindstorm )p.
>> >
>> >
>> > On 5/31/06, Petr Pytelka <Pe...@lightcomp.com> wrote:
>> > > Hi,
>> > >
>> > > I've created simplified, faster and less resource consuming
>> example of
>> > > problem.
>> > >
>> > > Only source code:
>> > > http://217.11.254.42/download/Test3.java (7kb)
>> > >
>> > > Package with all libraries:
>> > > http://217.11.254.42/download/index-problem.zip (5MB)
>> > >
>> > > Please could you anybody confirm if it is problem in Jackrabbit or
>> in my
>> > > usage of JackRabbit ?
>> > >
>> > > Thanks a lot.
>> > >
>> > > Petr
>> > >
>> > >
>> > > Petr Pytelka wrote:
>> > > > Hi Marcel,
>> > > > I've tried to find out more about my issue but didn't succeed.
>> > > >
>> > > > There is small application which demonstrate the problem:
>> > > > http://217.11.254.42/download/index-problem.zip
>> > > >
>> > > > Steps to reproduce:
>> > > >
>> > > > Prerequisites:
>> > > > 1. This test is resource consuming - need computer with at least
>> 512 MB,
>> > > > better 1 GB
>> > > > 2. Package contain all needed libs and also source code (10kb) -
>> I was
>> > > > not able to create smaller example - sorry for that.
>> > > >
>> > > > Test can be run using script "run-test" or "run-test.bat" - at the
>> > > > beginning of each file is path to the Java.
>> > > >
>> > > > Whole test is trying to create 10000 nodes
>> > > > - Phase 1: Create 20 nodes
>> > > > - Phase 2: Query for last created 20 nodes
>> > > >
>> > > > Problem is that on faster machines this test fail around
>> 3000-5000th
>> > > > document - it is not possible to locate file. Query in
>> repository is
>> > > > based on the attribute values.
>> > > >
>> > > > Maybe that the problem is in my code.. I'll really appreciate if
>> you can
>> > > > look on it...
>> > > >
>> > > > Thanks, Petr
>> > > >
>> > > >
>> > > > Marcel Reutegger wrote:
>> > > >> Hi Petr,
>> > > >>
>> > > >> can you please provide the source code to your test case that
>> allows
>> > > >> us to reproduce the behaviour? Thanks
>> > > >>
>> > > >> regards
>> > > >> marcel
>> > > >>
>> > > >> Petr Pytelka wrote:
>> > > >>> Hi all,
>> > > >>>
>> > > >>> I'm testing performance of Jackrabbit and found one issue.
>> > > >>>
>> > > >>> Test case:
>> > > >>> 1. Insert N items
>> > > >>> 2. Search each of inserted items (using Query)
>> > > >>> 3. Drop inserted item
>> > > >>>
>> > > >>> When N is lower then aprox.1000 items every thinks works fine.
>> But when
>> > > >>> N is higher then aprox. 1000 items Jackrabbit (Lucene) some
>> how "lost"
>> > > >>> indexes and I'm not able to find required item (usualy I'm
>> able to find
>> > > >>> first 300 items).
>> > > >>> If I delete index directory and re-run search, Jackrabbit
>> build new
>> > > >>> indexes during startup and I'm able to find all items.
>> > > >>>
>> > > >>> Lucene: version 1.4.3
>> > > >>> DB-backend: Derby
>> > > >>>
>> > > >>> Any idea where can be problem ?
>> > > >>>
>> > > >>> Thanks a lot, Petr Pytelka
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>>
>> > >
>> >
>>
Re: Problem with indexes
Posted by Alexandru Popescu <th...@gmail.com>.
A quick update (probably I will update it once more later).
I have run Test3 with JDK1.5.0_06 and JRockit1.5.0_06 and with the
following configs:
1/ jackrabbit-1.0: consistently fails between 700 - 1500 documents
2/ jackrabbit-1.0.1 (with patch JCR-448 included)
2.1/ with JDK1.5.0_06: results in a JVM crash at about 13000-15000 documents
2.2/ with JRockit1.5.0_06: the only one that finished the test
Unfortunately, I have run these test last night at about 5am, so I
will completely redo it today and update this thread. My plan is to
run the Test3 against jackrabbit-1.0, jackrabbit-1.0.1 and
jackrabbit-1.0.1a (containing the patches for JCR-448 and JCR-450)
with SUN JDK. If this will consistently fail (as above), than rerun
all tests with JRockit 1.5.0_06.
hth,
./alex
--
.w( the_mindstorm )p.
On 6/3/06, Alexandru Popescu <th...@gmail.com> wrote:
> I have checked the jars at the last zip you provided, but the
> jackrabbit-core seems to not include the patch for JCR-448 (which is
> supposed to fix the problem).
>
> ./alex
> --
> .w( the_mindstorm )p.
>
>
> On 6/3/06, Alexandru Popescu <th...@gmail.com> wrote:
> > Petr is the initial problem still valid if you use the patched
> > (JCR-448) 1.0.1 release?
> >
> > ./alex
> > --
> > .w( the_mindstorm )p.
> >
> >
> > On 5/31/06, Petr Pytelka <Pe...@lightcomp.com> wrote:
> > > Hi,
> > >
> > > I've created simplified, faster and less resource consuming example of
> > > problem.
> > >
> > > Only source code:
> > > http://217.11.254.42/download/Test3.java (7kb)
> > >
> > > Package with all libraries:
> > > http://217.11.254.42/download/index-problem.zip (5MB)
> > >
> > > Please could you anybody confirm if it is problem in Jackrabbit or in my
> > > usage of JackRabbit ?
> > >
> > > Thanks a lot.
> > >
> > > Petr
> > >
> > >
> > > Petr Pytelka wrote:
> > > > Hi Marcel,
> > > > I've tried to find out more about my issue but didn't succeed.
> > > >
> > > > There is small application which demonstrate the problem:
> > > > http://217.11.254.42/download/index-problem.zip
> > > >
> > > > Steps to reproduce:
> > > >
> > > > Prerequisites:
> > > > 1. This test is resource consuming - need computer with at least 512 MB,
> > > > better 1 GB
> > > > 2. Package contain all needed libs and also source code (10kb) - I was
> > > > not able to create smaller example - sorry for that.
> > > >
> > > > Test can be run using script "run-test" or "run-test.bat" - at the
> > > > beginning of each file is path to the Java.
> > > >
> > > > Whole test is trying to create 10000 nodes
> > > > - Phase 1: Create 20 nodes
> > > > - Phase 2: Query for last created 20 nodes
> > > >
> > > > Problem is that on faster machines this test fail around 3000-5000th
> > > > document - it is not possible to locate file. Query in repository is
> > > > based on the attribute values.
> > > >
> > > > Maybe that the problem is in my code.. I'll really appreciate if you can
> > > > look on it...
> > > >
> > > > Thanks, Petr
> > > >
> > > >
> > > > Marcel Reutegger wrote:
> > > >> Hi Petr,
> > > >>
> > > >> can you please provide the source code to your test case that allows
> > > >> us to reproduce the behaviour? Thanks
> > > >>
> > > >> regards
> > > >> marcel
> > > >>
> > > >> Petr Pytelka wrote:
> > > >>> Hi all,
> > > >>>
> > > >>> I'm testing performance of Jackrabbit and found one issue.
> > > >>>
> > > >>> Test case:
> > > >>> 1. Insert N items
> > > >>> 2. Search each of inserted items (using Query)
> > > >>> 3. Drop inserted item
> > > >>>
> > > >>> When N is lower then aprox.1000 items every thinks works fine. But when
> > > >>> N is higher then aprox. 1000 items Jackrabbit (Lucene) some how "lost"
> > > >>> indexes and I'm not able to find required item (usualy I'm able to find
> > > >>> first 300 items).
> > > >>> If I delete index directory and re-run search, Jackrabbit build new
> > > >>> indexes during startup and I'm able to find all items.
> > > >>>
> > > >>> Lucene: version 1.4.3
> > > >>> DB-backend: Derby
> > > >>>
> > > >>> Any idea where can be problem ?
> > > >>>
> > > >>> Thanks a lot, Petr Pytelka
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > >
> >
>
Re: Problem with indexes
Posted by Alexandru Popescu <th...@gmail.com>.
I have checked the jars at the last zip you provided, but the
jackrabbit-core seems to not include the patch for JCR-448 (which is
supposed to fix the problem).
./alex
--
.w( the_mindstorm )p.
On 6/3/06, Alexandru Popescu <th...@gmail.com> wrote:
> Petr is the initial problem still valid if you use the patched
> (JCR-448) 1.0.1 release?
>
> ./alex
> --
> .w( the_mindstorm )p.
>
>
> On 5/31/06, Petr Pytelka <Pe...@lightcomp.com> wrote:
> > Hi,
> >
> > I've created simplified, faster and less resource consuming example of
> > problem.
> >
> > Only source code:
> > http://217.11.254.42/download/Test3.java (7kb)
> >
> > Package with all libraries:
> > http://217.11.254.42/download/index-problem.zip (5MB)
> >
> > Please could you anybody confirm if it is problem in Jackrabbit or in my
> > usage of JackRabbit ?
> >
> > Thanks a lot.
> >
> > Petr
> >
> >
> > Petr Pytelka wrote:
> > > Hi Marcel,
> > > I've tried to find out more about my issue but didn't succeed.
> > >
> > > There is small application which demonstrate the problem:
> > > http://217.11.254.42/download/index-problem.zip
> > >
> > > Steps to reproduce:
> > >
> > > Prerequisites:
> > > 1. This test is resource consuming - need computer with at least 512 MB,
> > > better 1 GB
> > > 2. Package contain all needed libs and also source code (10kb) - I was
> > > not able to create smaller example - sorry for that.
> > >
> > > Test can be run using script "run-test" or "run-test.bat" - at the
> > > beginning of each file is path to the Java.
> > >
> > > Whole test is trying to create 10000 nodes
> > > - Phase 1: Create 20 nodes
> > > - Phase 2: Query for last created 20 nodes
> > >
> > > Problem is that on faster machines this test fail around 3000-5000th
> > > document - it is not possible to locate file. Query in repository is
> > > based on the attribute values.
> > >
> > > Maybe that the problem is in my code.. I'll really appreciate if you can
> > > look on it...
> > >
> > > Thanks, Petr
> > >
> > >
> > > Marcel Reutegger wrote:
> > >> Hi Petr,
> > >>
> > >> can you please provide the source code to your test case that allows
> > >> us to reproduce the behaviour? Thanks
> > >>
> > >> regards
> > >> marcel
> > >>
> > >> Petr Pytelka wrote:
> > >>> Hi all,
> > >>>
> > >>> I'm testing performance of Jackrabbit and found one issue.
> > >>>
> > >>> Test case:
> > >>> 1. Insert N items
> > >>> 2. Search each of inserted items (using Query)
> > >>> 3. Drop inserted item
> > >>>
> > >>> When N is lower then aprox.1000 items every thinks works fine. But when
> > >>> N is higher then aprox. 1000 items Jackrabbit (Lucene) some how "lost"
> > >>> indexes and I'm not able to find required item (usualy I'm able to find
> > >>> first 300 items).
> > >>> If I delete index directory and re-run search, Jackrabbit build new
> > >>> indexes during startup and I'm able to find all items.
> > >>>
> > >>> Lucene: version 1.4.3
> > >>> DB-backend: Derby
> > >>>
> > >>> Any idea where can be problem ?
> > >>>
> > >>> Thanks a lot, Petr Pytelka
> > >>>
> > >>>
> > >>>
> > >>>
> >
>
Re: Problem with indexes
Posted by Alexandru Popescu <th...@gmail.com>.
Petr is the initial problem still valid if you use the patched
(JCR-448) 1.0.1 release?
./alex
--
.w( the_mindstorm )p.
On 5/31/06, Petr Pytelka <Pe...@lightcomp.com> wrote:
> Hi,
>
> I've created simplified, faster and less resource consuming example of
> problem.
>
> Only source code:
> http://217.11.254.42/download/Test3.java (7kb)
>
> Package with all libraries:
> http://217.11.254.42/download/index-problem.zip (5MB)
>
> Please could you anybody confirm if it is problem in Jackrabbit or in my
> usage of JackRabbit ?
>
> Thanks a lot.
>
> Petr
>
>
> Petr Pytelka wrote:
> > Hi Marcel,
> > I've tried to find out more about my issue but didn't succeed.
> >
> > There is small application which demonstrate the problem:
> > http://217.11.254.42/download/index-problem.zip
> >
> > Steps to reproduce:
> >
> > Prerequisites:
> > 1. This test is resource consuming - need computer with at least 512 MB,
> > better 1 GB
> > 2. Package contain all needed libs and also source code (10kb) - I was
> > not able to create smaller example - sorry for that.
> >
> > Test can be run using script "run-test" or "run-test.bat" - at the
> > beginning of each file is path to the Java.
> >
> > Whole test is trying to create 10000 nodes
> > - Phase 1: Create 20 nodes
> > - Phase 2: Query for last created 20 nodes
> >
> > Problem is that on faster machines this test fail around 3000-5000th
> > document - it is not possible to locate file. Query in repository is
> > based on the attribute values.
> >
> > Maybe that the problem is in my code.. I'll really appreciate if you can
> > look on it...
> >
> > Thanks, Petr
> >
> >
> > Marcel Reutegger wrote:
> >> Hi Petr,
> >>
> >> can you please provide the source code to your test case that allows
> >> us to reproduce the behaviour? Thanks
> >>
> >> regards
> >> marcel
> >>
> >> Petr Pytelka wrote:
> >>> Hi all,
> >>>
> >>> I'm testing performance of Jackrabbit and found one issue.
> >>>
> >>> Test case:
> >>> 1. Insert N items
> >>> 2. Search each of inserted items (using Query)
> >>> 3. Drop inserted item
> >>>
> >>> When N is lower then aprox.1000 items every thinks works fine. But when
> >>> N is higher then aprox. 1000 items Jackrabbit (Lucene) some how "lost"
> >>> indexes and I'm not able to find required item (usualy I'm able to find
> >>> first 300 items).
> >>> If I delete index directory and re-run search, Jackrabbit build new
> >>> indexes during startup and I'm able to find all items.
> >>>
> >>> Lucene: version 1.4.3
> >>> DB-backend: Derby
> >>>
> >>> Any idea where can be problem ?
> >>>
> >>> Thanks a lot, Petr Pytelka
> >>>
> >>>
> >>>
> >>>
>
Re: Problem with indexes
Posted by Petr Pytelka <Pe...@lightcomp.com>.
Hi,
I've created simplified, faster and less resource consuming example of
problem.
Only source code:
http://217.11.254.42/download/Test3.java (7kb)
Package with all libraries:
http://217.11.254.42/download/index-problem.zip (5MB)
Please could you anybody confirm if it is problem in Jackrabbit or in my
usage of JackRabbit ?
Thanks a lot.
Petr
Petr Pytelka wrote:
> Hi Marcel,
> I've tried to find out more about my issue but didn't succeed.
>
> There is small application which demonstrate the problem:
> http://217.11.254.42/download/index-problem.zip
>
> Steps to reproduce:
>
> Prerequisites:
> 1. This test is resource consuming - need computer with at least 512 MB,
> better 1 GB
> 2. Package contain all needed libs and also source code (10kb) - I was
> not able to create smaller example - sorry for that.
>
> Test can be run using script "run-test" or "run-test.bat" - at the
> beginning of each file is path to the Java.
>
> Whole test is trying to create 10000 nodes
> - Phase 1: Create 20 nodes
> - Phase 2: Query for last created 20 nodes
>
> Problem is that on faster machines this test fail around 3000-5000th
> document - it is not possible to locate file. Query in repository is
> based on the attribute values.
>
> Maybe that the problem is in my code.. I'll really appreciate if you can
> look on it...
>
> Thanks, Petr
>
>
> Marcel Reutegger wrote:
>> Hi Petr,
>>
>> can you please provide the source code to your test case that allows
>> us to reproduce the behaviour? Thanks
>>
>> regards
>> marcel
>>
>> Petr Pytelka wrote:
>>> Hi all,
>>>
>>> I'm testing performance of Jackrabbit and found one issue.
>>>
>>> Test case:
>>> 1. Insert N items
>>> 2. Search each of inserted items (using Query)
>>> 3. Drop inserted item
>>>
>>> When N is lower then aprox.1000 items every thinks works fine. But when
>>> N is higher then aprox. 1000 items Jackrabbit (Lucene) some how "lost"
>>> indexes and I'm not able to find required item (usualy I'm able to find
>>> first 300 items).
>>> If I delete index directory and re-run search, Jackrabbit build new
>>> indexes during startup and I'm able to find all items.
>>>
>>> Lucene: version 1.4.3
>>> DB-backend: Derby
>>>
>>> Any idea where can be problem ?
>>>
>>> Thanks a lot, Petr Pytelka
>>>
>>>
>>>
>>>
Re: Problem with indexes
Posted by Petr Pytelka <Pe...@lightcomp.com>.
Hi Marcel,
I've tried to find out more about my issue but didn't succeed.
There is small application which demonstrate the problem:
http://217.11.254.42/download/index-problem.zip
Steps to reproduce:
Prerequisites:
1. This test is resource consuming - need computer with at least 512 MB,
better 1 GB
2. Package contain all needed libs and also source code (10kb) - I was
not able to create smaller example - sorry for that.
Test can be run using script "run-test" or "run-test.bat" - at the
beginning of each file is path to the Java.
Whole test is trying to create 10000 nodes
- Phase 1: Create 20 nodes
- Phase 2: Query for last created 20 nodes
Problem is that on faster machines this test fail around 3000-5000th
document - it is not possible to locate file. Query in repository is
based on the attribute values.
Maybe that the problem is in my code.. I'll really appreciate if you can
look on it...
Thanks, Petr
Marcel Reutegger wrote:
> Hi Petr,
>
> can you please provide the source code to your test case that allows us
> to reproduce the behaviour? Thanks
>
> regards
> marcel
>
> Petr Pytelka wrote:
>> Hi all,
>>
>> I'm testing performance of Jackrabbit and found one issue.
>>
>> Test case:
>> 1. Insert N items
>> 2. Search each of inserted items (using Query)
>> 3. Drop inserted item
>>
>> When N is lower then aprox.1000 items every thinks works fine. But when
>> N is higher then aprox. 1000 items Jackrabbit (Lucene) some how "lost"
>> indexes and I'm not able to find required item (usualy I'm able to find
>> first 300 items).
>> If I delete index directory and re-run search, Jackrabbit build new
>> indexes during startup and I'm able to find all items.
>>
>> Lucene: version 1.4.3
>> DB-backend: Derby
>>
>> Any idea where can be problem ?
>>
>> Thanks a lot, Petr Pytelka
>>
>>
>>
>>
Re: Problem with indexes
Posted by Marcel Reutegger <ma...@gmx.net>.
Hi Petr,
can you please provide the source code to your test case that allows us to
reproduce the behaviour? Thanks
regards
marcel
Petr Pytelka wrote:
> Hi all,
>
> I'm testing performance of Jackrabbit and found one issue.
>
> Test case:
> 1. Insert N items
> 2. Search each of inserted items (using Query)
> 3. Drop inserted item
>
> When N is lower then aprox.1000 items every thinks works fine. But when
> N is higher then aprox. 1000 items Jackrabbit (Lucene) some how "lost"
> indexes and I'm not able to find required item (usualy I'm able to find
> first 300 items).
> If I delete index directory and re-run search, Jackrabbit build new
> indexes during startup and I'm able to find all items.
>
> Lucene: version 1.4.3
> DB-backend: Derby
>
> Any idea where can be problem ?
>
> Thanks a lot, Petr Pytelka
>
>
>
>
Re: Problem with indexes
Posted by Victor Zagrebin <za...@mail.ru>.
I didn't know the JCR-448 influence on the index search. I thought it
influence on keeping a track of nodes that are deleted from the index. Now
all ok. I haven't an indication that this problem still persists.
Thanks a lot.
--
View this message in context: http://www.nabble.com/Problem-with-indexes-tf1699864.html#a5098005
Sent from the Jackrabbit - Users forum at Nabble.com.
Re: Problem with indexes
Posted by Marcel Reutegger <ma...@gmx.net>.
The issue that was reported in this thread has already been fixed. See:
http://issues.apache.org/jira/browse/JCR-448
Do you have an indication that this problem still persists?
Victor Zagrebin wrote:
> I think it's necessary to look at AddIndex class (in MultiIndex class) method
> execute. Why index.indexes.add(idx); is absent?
AddIndex.execute() calls index.getOrCreateIndex() that's the place where
the new index is added to the indexes list.
regards
marcel
Re: Problem with indexes
Posted by Victor Zagrebin <za...@mail.ru>.
I think it's necessary to look at AddIndex class (in MultiIndex class) method
execute. Why index.indexes.add(idx); is absent?
--
View this message in context: http://www.nabble.com/Problem-with-indexes-tf1699864.html#a5067090
Sent from the Jackrabbit - Users forum at Nabble.com.