You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Noble Paul (JIRA)" <ji...@apache.org> on 2019/03/11 02:54:00 UTC
[jira] [Comment Edited] (SOLR-13312) write out responses without
creating SolrDocument objects
[ https://issues.apache.org/jira/browse/SOLR-13312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789077#comment-16789077 ]
Noble Paul edited comment on SOLR-13312 at 3/11/19 2:53 AM:
------------------------------------------------------------
[~dsmiley] SolrDocument is an expensive data structure. Yes, we may need a more efficient data structure to actually accomplish this. HashMaps are extremely memory inefficient.
Skipping transformation is something we can't do now without backward incompatibility. We can probably rewrite Transformers like ChildDocTransformer to adapt to the new format. We may need to create the SolrDocument objects where other transformers are used. But most requests never use any transformers. They are paying a huge price
One problem we have in Solr is we never explore new Data structures suitable for our needs. We just keep using List, Map, Namedlist irrespective of how many objects we create .We don't ever differentiate between a usecase where 10 maps are created Vs. 10 million Maps are created. We never stop to think how this all adds up. Lucene keeps thinking about this all the time and it shows . Lucene is extremely memory efficient while Solr is not
was (Author: noble.paul):
[~dsmiley] SolrDocument is an expensive data structure. Yes, we may need a more efficient data structure to actually accomplish this. HashMaps are extremely memory inefficient.
Skipping transformation is something we can't do now without backward incompatibility. We can probably rewrite Transformers like ChildDocTransformer to adapt to the new format. We may need to create the SolrDocument objects where other transformers are used. But most requests never use any transformers. They are paying a huge price
> write out responses without creating SolrDocument objects
> ---------------------------------------------------------
>
> Key: SOLR-13312
> URL: https://issues.apache.org/jira/browse/SOLR-13312
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Noble Paul
> Priority: Major
>
> Once we get a document from lucene there is no need to create a SolrDocument object to write out the response, if there are no transformers
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org