You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Erick Erickson (Commented) (JIRA)" <ji...@apache.org> on 2011/12/08 22:21:40 UTC
[jira] [Commented] (SOLR-2679) Optimize memory usage in
JavaBinCodec.readStr()
[ https://issues.apache.org/jira/browse/SOLR-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165575#comment-13165575 ]
Erick Erickson commented on SOLR-2679:
--------------------------------------
Does anyone have an opinion on this? It seems to me that the tradeoff is that the current form of this code reads the bytes from a local array, where the proposed patch makes a method call for every byte that's read, which seems like it'd be slower since the dis.readFully method does an arrayCopy under the covers.
I'll volunteer to apply or close this, it seems like a time/space tradeoff. And the underlying byte[] grows to a max size and gets re-used.
Since the class that's used to read data is called FastInputStream, my straw-man proposal is that we do NOT apply this patch and I'll close the JIRA.
> Optimize memory usage in JavaBinCodec.readStr()
> -----------------------------------------------
>
> Key: SOLR-2679
> URL: https://issues.apache.org/jira/browse/SOLR-2679
> Project: Solr
> Issue Type: Improvement
> Components: clients - java, Response Writers, update
> Affects Versions: 3.3
> Reporter: Maxim Valyanskiy
> Fix For: 3.6, 4.0
>
> Attachments: SOLR-2679.patch
>
>
> In Solr 3.3, JavaBinCodec.readStr() reads complete string into byte[] and then converts it into String. FastInputStream is already buffered, so this it not necessary and may take a lot of memory if data string is large.
> This patch replaces usage of big byte[] with usage of dis.readUnsignedByte() for getting characters directly from stream.
> Patch reduces memory usage and may slightly improve performance of reading text data in javabin-codec.
> JavaBinCodec.readStr() function already has unit test in TestJavaBinCodec, no additional unit test is required.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org