You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "dungba88 (via GitHub)" <gi...@apache.org> on 2023/11/29 11:49:41 UTC

[I] Do we suboptimally call ByteBuffersDataOutput.toDataInput()? [lucene]

dungba88 opened a new issue, #12852:
URL: https://github.com/apache/lucene/issues/12852

   ### Description
   
   See this comment: https://github.com/apache/lucene/pull/12624#issuecomment-1829633268
   
   > I wonder: are there other places in Lucene that might fall prey to this performance trap (calling toDataInput frequently while continuing to append bytes to the DataOutput)? FreqProxTermsWriter seems to use this for each term to re-read the postings (maybe for the static sort use case).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Do we suboptimally call ByteBuffersDataOutput.toDataInput()? [lucene]

Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #12852:
URL: https://github.com/apache/lucene/issues/12852#issuecomment-1840786092

   I don't think it was ever meant to be called repeatedly in fast bursts. It was meant to provide a direct reader for previously written buffers, once the writing is completed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Do we suboptimally call ByteBuffersDataOutput.toDataInput()? [lucene]

Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #12852:
URL: https://github.com/apache/lucene/issues/12852#issuecomment-1840793857

   Also, not sure why it's slow - perhaps because of the wrapping in read-only buffers when toBufferList is called? If so, then this could be internally tweaked by calling toWriteableBufferList inside toDataInput (since we know data input is not going to be messing with the buffers...).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Do we suboptimally call ByteBuffersDataOutput.toDataInput()? [lucene]

Posted by "mikemccand (via GitHub)" <gi...@apache.org>.
mikemccand commented on issue #12852:
URL: https://github.com/apache/lucene/issues/12852#issuecomment-1840648672

   At least we should add a warning to the javadocs here.
   
   And we should audit other places that are frequently calling `.toDataInput()` (e.g. block tree terms dict writing?).
   
   This method is surprisingly slow compared to simply juggling our own `byte[]` under the hood, at least in FST's (well, `Test2BFSTs`) usage.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org