You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Uwe Schindler (Jira)" <ji...@apache.org> on 2020/07/16 13:28:00 UTC
[jira] [Comment Edited] (LUCENE-9432) Use LinkedList instead of
manual array re-sizing for better throughput.
[ https://issues.apache.org/jira/browse/LUCENE-9432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159213#comment-17159213 ]
Uwe Schindler edited comment on LUCENE-9432 at 7/16/20, 1:27 PM:
-----------------------------------------------------------------
Hi,
I checked the patch. A linkedlist is in most cases a bad idea, because it does not allow performant index based access - and that is what is used here. I don't fully understand the code, but a linked list is a bad choice for this access pattern, as LinkedList.get() involves linear scan!
The problem you see here is more related to the fact that the array is of zero size initially and Array.oversize() behaves bad there if initial size is too small (many copies of small arrays). I'd siply replace the whole thing by an ArrayList with suitable initial size. If you want a real stack-based behaviour (push/pop), a better replacement would be ArrayDeque implementing the Deque interface.
So in short: Either replace by a Deque (fully using the Deque interface including push/pop and corresponding iterators) or use an ArrayList. A LinkedList is a bad choice as it might save some copy costs, but it brings a huge overhead with linear scans and also lots of garbage on heap.
was (Author: thetaphi):
Hi,
I checked the patch. A linkedlist is in most cases a bad idea, because it does not allow performant index based access - and that is what is used here. I don't fully understand the code, but a linked list is a bad choice for this access pattern, as LinkedList.get() involves linear scan!
The problem you see here is more related to the fact that the array is of zero size initially. I'd siply replace the whole thing by an ArrayList. If you want a real stack-based behaviour (push/pop), a better replacement would be ArrayDeque implementing the Deque interface.
So in short: Either replace by a Deque (fully using the Deque interface including push/pop and corresponding iterators) or use an ArrayList. A LinkedList is a bad choice as it might save some copy costs, but it brings a huge overhead with linear scans and also lots of garbage on heap.
> Use LinkedList instead of manual array re-sizing for better throughput.
> -----------------------------------------------------------------------
>
> Key: LUCENE-9432
> URL: https://issues.apache.org/jira/browse/LUCENE-9432
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
> Reporter: Mohammad Sadiq
> Priority: Minor
> Labels: patch-available, performance, pull-request-available
> Attachments: LUCENE-9432.patch
>
>
> I observed that using {{LinkedList}} instead of manually re-sizing and copying {{SegmentTermEnumFrame}}s improves red-line QPS. Does it make sense to include this?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org