You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by biplobbiswas <re...@gmail.com> on 2017/05/19 11:41:05 UTC
Nested Document is flattened even with @Field(child = true)
annotation
I have the following structure for the class used for indexing a solr
document. I am using solrj 5.5.2 (same solr is being used on the cluster
with the collection in solr cloud mode having 3 shards)
I added the @Field(child = true) to the ChnagedAttribute object and even
though my document is indexed, it is flattened, such that, the object is
treated as a different document.
So rather than having 3 documents, I am getting 6 docs from solr when I
query for everything.
Any help regarding this issue is really appreciated.
@Data
public class EventDocument implements Serializable {
private static final long serialVersionUID = 1L;
@Field("id")
private String solrId;
@Field("eventName_t")
private String eventName;
@Field("Message_t")
private String message;
@Field(child = true)
private ChangedAttribute changedAttributes;
}
@Data
class ChangedAttribute implements Serializable {
private static final long serialVersionUID = 1L;
@Field("id")
private String id;
@Field("AttributeName_t")
private String attributeName;
@Field("OldValue_t")
private String oldValue;
@Field("NewValue_t")
private String newValue;
}
--
View this message in context: http://lucene.472066.n3.nabble.com/Nested-Document-is-flattened-even-with-Field-child-true-annotation-tp4335877.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Nested Document is flattened even with @Field(child = true) annotation
Posted by Mikhail Khludnev <mk...@apache.org>.
Hello!
Since you are talking about Banana, you might be interested in faceting.
You probably can have child docs in results and facets them, but this gives
child level counts. If you need to have parent level counts by child fields
you have two ways to do so: see
http://blog-archive.griddynamics.com/2016/03/block-join-faceting-implementation.html
Overall, exposing nested docs in Banana is exciting research. Let us know
how it goes.
On Mon, May 22, 2017 at 4:39 PM, biplobbiswas <revolutionisme+solr@gmail.com
> wrote:
> Rick Leir-2 wrote
> > Yes! And the join queries get complicated. Yonick has some good blogs on
> > this.
> >
> > On May 19, 2017 11:05:52 AM EDT, biplobbiswas <
>
> > revolutionisme+solr@
>
> > > wrote:
> >>Wait, if I understand correctly, the documents would be indexed like
> >>that but
> >>we can get back the document as nested if we perform the
> >>blockjoinqueryparsing?
> >>
> >>So if I query normally with the default parser I would get all
> >>documents
> >>separately?
> >>Did i understand correctly?
> >>
> >>Thanks & regards
> >>Biplob
> >>
> >>
> >>
> >>--
> >>View this message in context:
> >>http://lucene.472066.n3.nabble.com/Nested-Document-is-
> flattened-even-with-Field-child-true-annotation-tp4335877p4335911.html
> >>Sent from the Solr - User mailing list archive at Nabble.com.
> >
> > --
> > Sorry for being brief. Alternate email is rickleir at yahoo dot com
>
>
> Thanks a lot for that reply.
>
> Now I am trying to integrate the results on to a banana dashboard. Is
> there
> any way to visualize the _childDocuments_ as well? Currently its part of a
> the _childDocuments_ and represents itself in a json format, I can't
> generate any statistics on top of the nested documents.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Nested-Document-is-flattened-even-with-Field-
> child-true-annotation-tp4335877p4336296.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Sincerely yours
Mikhail Khludnev
Re: Nested Document is flattened even with @Field(child = true)
annotation
Posted by biplobbiswas <re...@gmail.com>.
Rick Leir-2 wrote
> Yes! And the join queries get complicated. Yonick has some good blogs on
> this.
>
> On May 19, 2017 11:05:52 AM EDT, biplobbiswas <
> revolutionisme+solr@
> > wrote:
>>Wait, if I understand correctly, the documents would be indexed like
>>that but
>>we can get back the document as nested if we perform the
>>blockjoinqueryparsing?
>>
>>So if I query normally with the default parser I would get all
>>documents
>>separately?
>>Did i understand correctly?
>>
>>Thanks & regards
>>Biplob
>>
>>
>>
>>--
>>View this message in context:
>>http://lucene.472066.n3.nabble.com/Nested-Document-is-flattened-even-with-Field-child-true-annotation-tp4335877p4335911.html
>>Sent from the Solr - User mailing list archive at Nabble.com.
>
> --
> Sorry for being brief. Alternate email is rickleir at yahoo dot com
Thanks a lot for that reply.
Now I am trying to integrate the results on to a banana dashboard. Is there
any way to visualize the _childDocuments_ as well? Currently its part of a
the _childDocuments_ and represents itself in a json format, I can't
generate any statistics on top of the nested documents.
--
View this message in context: http://lucene.472066.n3.nabble.com/Nested-Document-is-flattened-even-with-Field-child-true-annotation-tp4335877p4336296.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Nested Document is flattened even with @Field(child = true) annotation
Posted by Rick Leir <rl...@leirtech.com>.
Yes! And the join queries get complicated. Yonick has some good blogs on this.
On May 19, 2017 11:05:52 AM EDT, biplobbiswas <re...@gmail.com> wrote:
>Wait, if I understand correctly, the documents would be indexed like
>that but
>we can get back the document as nested if we perform the
>blockjoinqueryparsing?
>
>So if I query normally with the default parser I would get all
>documents
>separately?
>Did i understand correctly?
>
>Thanks & regards
>Biplob
>
>
>
>--
>View this message in context:
>http://lucene.472066.n3.nabble.com/Nested-Document-is-flattened-even-with-Field-child-true-annotation-tp4335877p4335911.html
>Sent from the Solr - User mailing list archive at Nabble.com.
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
Re: Nested Document is flattened even with @Field(child = true)
annotation
Posted by biplobbiswas <re...@gmail.com>.
Wait, if I understand correctly, the documents would be indexed like that but
we can get back the document as nested if we perform the
blockjoinqueryparsing?
So if I query normally with the default parser I would get all documents
separately?
Did i understand correctly?
Thanks & regards
Biplob
--
View this message in context: http://lucene.472066.n3.nabble.com/Nested-Document-is-flattened-even-with-Field-child-true-annotation-tp4335877p4335911.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Nested Document is flattened even with @Field(child = true)
annotation
Posted by biplobbiswas <re...@gmail.com>.
Hi
Mikhail Khludnev-2 wrote
> Hello,
>
> You need to use
> https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-BlockJoinQueryParsers
> and
> https://cwiki.apache.org/confluence/display/solr/Transforming+Result+Documents#TransformingResultDocuments-[child]-ChildDocTransformerFactory
> to get the nested data back.
>
>
> --
> Sincerely yours
> Mikhail Khludnev
I had already gone through those links you posted and they talk about
retrieving after indexing. My problem is that my documents are not indexed
in a nested structure.
Can you please look at the first comment as well where I posted a sample
code and sample response which i get back.
Because its creating distinct documents for nested structure
--
View this message in context: http://lucene.472066.n3.nabble.com/Nested-Document-is-flattened-even-with-Field-child-true-annotation-tp4335877p4335891.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Nested Document is flattened even with @Field(child = true) annotation
Posted by Mikhail Khludnev <mk...@apache.org>.
Hello,
You need to use
https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-BlockJoinQueryParsers
and
https://cwiki.apache.org/confluence/display/solr/Transforming+Result+Documents#TransformingResultDocuments-[child]-ChildDocTransformerFactory
to get the nested data back.
On Fri, May 19, 2017 at 2:52 PM, biplobbiswas <revolutionisme+solr@gmail.com
> wrote:
> Update,
>
> I checked with the following example as well and this also flattens the
> results.
>
> I took the example from here -
> https://issues.apache.org/jira/browse/SOLR-1945
>
>
> package com.airplus.poc.edl.spark.auditeventindexer;
> import java.io.IOException;
>
> import org.apache.solr.client.solrj.SolrServerException;
> import org.apache.solr.client.solrj.beans.Field;
> import org.apache.solr.client.solrj.impl.CloudSolrClient;
>
> /**
> * @author Biplob Biswas on 19.05.2017.
> */
>
> public class SolrNestedTest {
>
> public static void main(String[] args) throws IOException,
> SolrServerException {
> new SolrNestedTest().test();
> }
>
> public void test() throws IOException, SolrServerException {
>
> String zkHostString = "host:2181/solr";
> CloudSolrClient client = new CloudSolrClient(zkHostString);
>
> Test test = new Test();
> test.setId("2");
> Child c = new Child();
> c.child = true;
> c.id = "1";
> test.setChild(c);
> client.addBean("event_store", test, 10);
>
> client.close();
>
> }
>
> public class Child {
> @Field
> public String id;
> @Field
> public boolean child;
> }
>
> public class Test {
>
> @Field
> private String id;
>
> @Field(child = true)
> private Child child;
>
> public String getId() {
> return id;
> }
>
> public void setId(String id) {
> this.id = id;
> }
>
> public Child getChild() {
> return child;
> }
>
> public void setChild(Child child) {
> this.child = child;
> }
>
> }
> }
>
>
>
> The response back -
>
> {
> "responseHeader": {
> "status": 0,
> "QTime": 8,
> "params": {
> "q": "*:*",
> "indent": "true",
> "wt": "json",
> "_": "1495194572357"
> }
> },
> "response": {
> "numFound": 2,
> "start": 0,
> "maxScore": 1,
> "docs": [
> {
> "id": "1",
> "child": [
> true
> ]
> },
> {
> "id": "2",
> "_version_": 1567825059298410500
> }
> ]
> }
> }
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Nested-Document-is-flattened-even-with-Field-
> child-true-annotation-tp4335877p4335878.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Sincerely yours
Mikhail Khludnev
Re: Nested Document is flattened even with @Field(child = true)
annotation
Posted by biplobbiswas <re...@gmail.com>.
Update,
I checked with the following example as well and this also flattens the
results.
I took the example from here -
https://issues.apache.org/jira/browse/SOLR-1945
package com.airplus.poc.edl.spark.auditeventindexer;
import java.io.IOException;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.beans.Field;
import org.apache.solr.client.solrj.impl.CloudSolrClient;
/**
* @author Biplob Biswas on 19.05.2017.
*/
public class SolrNestedTest {
public static void main(String[] args) throws IOException,
SolrServerException {
new SolrNestedTest().test();
}
public void test() throws IOException, SolrServerException {
String zkHostString = "host:2181/solr";
CloudSolrClient client = new CloudSolrClient(zkHostString);
Test test = new Test();
test.setId("2");
Child c = new Child();
c.child = true;
c.id = "1";
test.setChild(c);
client.addBean("event_store", test, 10);
client.close();
}
public class Child {
@Field
public String id;
@Field
public boolean child;
}
public class Test {
@Field
private String id;
@Field(child = true)
private Child child;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public Child getChild() {
return child;
}
public void setChild(Child child) {
this.child = child;
}
}
}
The response back -
{
"responseHeader": {
"status": 0,
"QTime": 8,
"params": {
"q": "*:*",
"indent": "true",
"wt": "json",
"_": "1495194572357"
}
},
"response": {
"numFound": 2,
"start": 0,
"maxScore": 1,
"docs": [
{
"id": "1",
"child": [
true
]
},
{
"id": "2",
"_version_": 1567825059298410500
}
]
}
}
--
View this message in context: http://lucene.472066.n3.nabble.com/Nested-Document-is-flattened-even-with-Field-child-true-annotation-tp4335877p4335878.html
Sent from the Solr - User mailing list archive at Nabble.com.