You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Sridhar Raman <sr...@gmail.com> on 2007/02/02 14:38:49 UTC
NodeIterator very slow
I have a NodeIterator variable iter with 2000 elements. When I do an
iter.hasNext() or an iter.next(), the program takes a very long time to
execute the call.
Is there any solution? I am not using jackrabbit 1.2.1 yet. Should I? Would
it help?
Thanks,
Sridhar
Re: [JCR Browser] NodeIterator very slow
Posted by James Hang <jh...@bea.com>.
To further clarify, although we're only iterating over 100 nodes, each node
has 5-10 child nodes, with 5-10 properties each. On each iteration, we are
traversing the child node hierarchy of the node and all of their properties.
This causes the following CacheManager messages to be printed continuosly:
2007-02-20 09:56:14,330 INFO CacheManager - resizeAll
size=6
2007-02-20 09:56:15,384 INFO CacheManager - resizeAll
size=6
2007-02-20 09:56:16,429 INFO CacheManager - resizeAll
size=6
2007-02-20 09:56:17,616 INFO CacheManager - resizeAll
size=6
..
..
Is there possibly an issue with the CacheManager when trying to access a
large set of items (nodes and properties)?
James
James Hang wrote:
>
> Hi,
>
> Have you been able to find a solution for this problem? I think we are
> having a similar issue on our end and we're actually dealing with a much
> smaller result set - only 100 elements. The query appears to execute
> almost instantly, but it takes a long time to iterate through the results.
>
> We tried setting the "respectDocumentOrder" parameter to "false" in the
> SearchIndex configuration like someone suggested, but it didn't seem to
> speed things up.
>
> Any help would be greatly appreciated!
>
> Thanks,
> James
>
>
>
> Sridhar Raman wrote:
>>
>> I have a NodeIterator variable iter with 2000 elements. When I do an
>> iter.hasNext() or an iter.next(), the program takes a very long time to
>> execute the call.
>>
>> Is there any solution? I am not using jackrabbit 1.2.1 yet. Should I?
>> Would
>> it help?
>>
>> Thanks,
>> Sridhar
>>
>>
>
>
--
View this message in context: http://www.nabble.com/NodeIterator-very-slow-tf3160898.html#a9075695
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
Re: [JCR Browser] NodeIterator very slow
Posted by James Hang <jh...@bea.com>.
Hi,
Have you been able to find a solution for this problem? I think we are
having a similar issue on our end and we're actually dealing with a much
smaller result set - only 100 elements. The query appears to execute almost
instantly, but it takes a long time to iterate through the results.
We tried setting the "respectDocumentOrder" parameter to "false" in the
SearchIndex configuration like someone suggested, but it didn't seem to
speed things up.
Any help would be greatly appreciated!
Thanks,
James
Sridhar Raman wrote:
>
> I have a NodeIterator variable iter with 2000 elements. When I do an
> iter.hasNext() or an iter.next(), the program takes a very long time to
> execute the call.
>
> Is there any solution? I am not using jackrabbit 1.2.1 yet. Should I?
> Would
> it help?
>
> Thanks,
> Sridhar
>
>
--
View this message in context: http://www.nabble.com/NodeIterator-very-slow-tf3160898.html#a9074030
Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
Re: NodeIterator very slow
Posted by Michael Neale <mi...@gmail.com>.
Alex - if you upgrade you can try the option that Marcel is suggesting
(which is a jackrabbit specific thing, presuambly set in the repository.xml?
Marcel ?).
>From the looks of it its 1.1 only and up.
Michael.
On 2/5/07, Marcel Reutegger <ma...@gmx.net> wrote:
>
> Sridhar Raman wrote:
> > When does the query actually get executed? This is my code:
> > try
> > {
> > qr = query.execute();
> > entiter = qr.getNodes();
> > }
> > catch (Exception e)
> > {
> > logger.error("Executing entity query", e);
> > }
> >
> > These two statements are done quite quickly. And when I inspect the
> entiter
> > variable, it shows 2000 nodes.
> >
> > It's when I do either an entiter.hasNext() or an entiter.nextNode() that
> > the
> > problem occurs. Will your suggestions of config tweaks help here?
>
> Then it's probably the document order which is calculated when you access
> the
> first node of the query result:
>
> >> - disable document order on the query result
> >> See: http://issues.apache.org/jira/browse/JCR-145
>
> regards
> marcel
>
Re: NodeIterator very slow
Posted by Marcel Reutegger <ma...@gmx.net>.
Sridhar Raman wrote:
> When does the query actually get executed? This is my code:
> try
> {
> qr = query.execute();
> entiter = qr.getNodes();
> }
> catch (Exception e)
> {
> logger.error("Executing entity query", e);
> }
>
> These two statements are done quite quickly. And when I inspect the entiter
> variable, it shows 2000 nodes.
>
> It's when I do either an entiter.hasNext() or an entiter.nextNode() that
> the
> problem occurs. Will your suggestions of config tweaks help here?
Then it's probably the document order which is calculated when you access the
first node of the query result:
>> - disable document order on the query result
>> See: http://issues.apache.org/jira/browse/JCR-145
regards
marcel
Re: NodeIterator very slow
Posted by Alexandru Popescu <th...@gmail.com>.
I must confess that I hit in the past a similar problem and posted
about it. In my environment (Jackrabbit 1.0, 1-2 hundred nodes)
executing an XPath query vs normal navigation + manual ordering +
filtering is 100 slower (not to speak about the CPU utilization).
Finally I moved away from using queries as we haven't been able to
pinpoint the problem and solve it (I have profiled it and the time was
spend mainly in 2 points: access rights verification and some Lucene
internal call - if my memory is not completely lost something like
Document.next()). The code is much more uglier as we need to filter
each node, traverse them all to order according to a property value,
but the speed plus CPU consumptions have greatly improved.
./alex
--
.w( the_mindstorm )p.
On 2/5/07, Sridhar Raman <sr...@gmail.com> wrote:
> When does the query actually get executed? This is my code:
> try
> {
> qr = query.execute();
> entiter = qr.getNodes();
> }
> catch (Exception e)
> {
> logger.error("Executing entity query", e);
> }
>
> These two statements are done quite quickly. And when I inspect the entiter
> variable, it shows 2000 nodes.
>
> It's when I do either an entiter.hasNext() or an entiter.nextNode() that the
> problem occurs. Will your suggestions of config tweaks help here?
>
> On 2/5/07, Marcel Reutegger <ma...@gmx.net> wrote:
> >
> > Sridhar Raman wrote:
> > > I am executing an XPATH query. And I get the NodeIterator iter from the
> > > QueryResult.getNodes(). This iter is the one causing the problem.
> >
> > Then I'd assume it's the query that takes a long time to execute.
> >
> > You can try the following configuration tweaks:
> >
> > - disable document order on the query result
> > See: http://issues.apache.org/jira/browse/JCR-145
> > - set the paramter 'resultFetchSize' to a lower value if
> > you don't need all 2000 nodes, but are only interested
> > in the first few. please note however, that there are
> > security concerns with this option:
> > http://article.gmane.org/gmane.comp.apache.jackrabbit.devel/10103
> >
> > > Could you tell me what I need to do to get a thread dump from a java
> > > process?
> >
> > on unix:
> >
> > kill -QUIT <java-process-id>
> >
> > on windows:
> >
> > Ctrl-Break in the console where you started the java process
> >
> > The thread dump is sent to the standard out of the java process.
> >
> > regards
> > marcel
> >
>
Re: NodeIterator very slow
Posted by Sridhar Raman <sr...@gmail.com>.
When does the query actually get executed? This is my code:
try
{
qr = query.execute();
entiter = qr.getNodes();
}
catch (Exception e)
{
logger.error("Executing entity query", e);
}
These two statements are done quite quickly. And when I inspect the entiter
variable, it shows 2000 nodes.
It's when I do either an entiter.hasNext() or an entiter.nextNode() that the
problem occurs. Will your suggestions of config tweaks help here?
On 2/5/07, Marcel Reutegger <ma...@gmx.net> wrote:
>
> Sridhar Raman wrote:
> > I am executing an XPATH query. And I get the NodeIterator iter from the
> > QueryResult.getNodes(). This iter is the one causing the problem.
>
> Then I'd assume it's the query that takes a long time to execute.
>
> You can try the following configuration tweaks:
>
> - disable document order on the query result
> See: http://issues.apache.org/jira/browse/JCR-145
> - set the paramter 'resultFetchSize' to a lower value if
> you don't need all 2000 nodes, but are only interested
> in the first few. please note however, that there are
> security concerns with this option:
> http://article.gmane.org/gmane.comp.apache.jackrabbit.devel/10103
>
> > Could you tell me what I need to do to get a thread dump from a java
> > process?
>
> on unix:
>
> kill -QUIT <java-process-id>
>
> on windows:
>
> Ctrl-Break in the console where you started the java process
>
> The thread dump is sent to the standard out of the java process.
>
> regards
> marcel
>
Re: NodeIterator very slow
Posted by Marcel Reutegger <ma...@gmx.net>.
Sridhar Raman wrote:
> I am executing an XPATH query. And I get the NodeIterator iter from the
> QueryResult.getNodes(). This iter is the one causing the problem.
Then I'd assume it's the query that takes a long time to execute.
You can try the following configuration tweaks:
- disable document order on the query result
See: http://issues.apache.org/jira/browse/JCR-145
- set the paramter 'resultFetchSize' to a lower value if
you don't need all 2000 nodes, but are only interested
in the first few. please note however, that there are
security concerns with this option:
http://article.gmane.org/gmane.comp.apache.jackrabbit.devel/10103
> Could you tell me what I need to do to get a thread dump from a java
> process?
on unix:
kill -QUIT <java-process-id>
on windows:
Ctrl-Break in the console where you started the java process
The thread dump is sent to the standard out of the java process.
regards
marcel
Re: NodeIterator very slow
Posted by Sridhar Raman <sr...@gmail.com>.
I am executing an XPATH query. And I get the NodeIterator iter from the
QueryResult.getNodes(). This iter is the one causing the problem.
Could you tell me what I need to do to get a thread dump from a java
process?
On 2/5/07, Marcel Reutegger <ma...@gmx.net> wrote:
>
> Where do you get the NodeIterator from?
>
>
> http://www.day.com/maven/jsr170/javadocs/jcr-1.0/javax/jcr/class-use/NodeIterator.html
>
> You can also try to get a thread dump from the java process while it runs
> through the iterator. that will give you a hint where the time is spent.
>
> regards
> marcel
>
> Sridhar Raman wrote:
> > I have a NodeIterator variable iter with 2000 elements. When I do an
> > iter.hasNext() or an iter.next(), the program takes a very long time to
> > execute the call.
> >
> > Is there any solution? I am not using jackrabbit 1.2.1 yet. Should I?
> Would
> > it help?
> >
> > Thanks,
> > Sridhar
> >
>
Re: NodeIterator very slow
Posted by Marcel Reutegger <ma...@gmx.net>.
Where do you get the NodeIterator from?
http://www.day.com/maven/jsr170/javadocs/jcr-1.0/javax/jcr/class-use/NodeIterator.html
You can also try to get a thread dump from the java process while it runs
through the iterator. that will give you a hint where the time is spent.
regards
marcel
Sridhar Raman wrote:
> I have a NodeIterator variable iter with 2000 elements. When I do an
> iter.hasNext() or an iter.next(), the program takes a very long time to
> execute the call.
>
> Is there any solution? I am not using jackrabbit 1.2.1 yet. Should I? Would
> it help?
>
> Thanks,
> Sridhar
>