You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Noble Paul (JIRA)" <ji...@apache.org> on 2018/03/27 23:49:00 UTC
[jira] [Comment Edited] (SOLR-12094) JsonRecordReader ignores root
record fields after the split point
[ https://issues.apache.org/jira/browse/SOLR-12094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416475#comment-16416475 ]
Noble Paul edited comment on SOLR-12094 at 3/27/18 11:48 PM:
-------------------------------------------------------------
before going into the patch, I can see that it is not designed to work like that . The reason is that {{JsonRecordReader}} is a streaming parser. To include the {{'after'}} in the document, It must hold all the data in the {{'exams}} in memory. Think if there are a million docs in {{'exams'}} and keeping all of them in memory before it can read the value of {{'after'}}. So, it is going to seriously affect the performance of the parser for the normal use case.
was (Author: noble.paul):
before going into the patch, I can see that it is not designed to work like that . The reason is that {{JsonRecordReader}} is a streaming parser. To include the {{'after'}} in the document, It must hold all the data in the {{'examsæ}} in memory. So, it is going to seriously affect the performance of the parser for the normal use case.
> JsonRecordReader ignores root record fields after the split point
> -----------------------------------------------------------------
>
> Key: SOLR-12094
> URL: https://issues.apache.org/jira/browse/SOLR-12094
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrJ
> Affects Versions: master (8.0)
> Reporter: Przemysław Szeremiota
> Priority: Major
> Attachments: SOLR-12094.patch, SOLR-12094.patch, json-record-reader-bug.patch
>
>
> JsonRecordReader, when configured with other than top-level split, ignores all top-level JSON nodes after the split ends, for example:
> {code}
> {
> "first": "John",
> "last": "Doe",
> "grade": 8,
> "exams": [
> {
> "subject": "Maths",
> "test": "term1",
> "marks": 90
> },
> {
> "subject": "Biology",
> "test": "term1",
> "marks": 86
> }
> ],
> "after": "456"
> }
> {code}
> Node "after" won't be visible in SolrInputDocument constructed from /update/json/docs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org