You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by A Geek <dw...@live.com> on 2013/09/06 13:10:20 UTC
Store 2 dimensional array( of int values) in solr 4.0
hi All, I'm trying to store a 2 dimensional array in SOLR [version 4.0]. Basically I've the following data:
[[20121108, 1],[20121110, 7],[20121111, 2],[20121112, 2]] ...
The inner array being used to keep some count say X for that particular day. Currently, I'm using the following field to store this data:
<field name="dataX" type="string" indexed="true" stored="true" multiValued="true"/>
and I'm using python library pySolr to store the data. Currently the data that gets stored looks like this(its array of strings)
<arr name="dataX"><str>[20121108, 1]</str><str>[20121110, 7]</str><str>[20121111, 2]</str><str>[20121112, 2]</str><str>[20121113, 2]</str><str>[20121116, 1]</str></arr>
Is there a way, i can store the 2 dimensional array and the inner array can contain int values, like the one shown in the beginning example, such that the the final/stored data in SOLR looks something like: <arr name="dataX">
<arr name=index><int>20121108</int> <int> 7 </int> </arr>
<arr name=index><int> 20121110</int><int> 12 </int></arr>
<arr name=index><int> 20121110</int><int> 12 </int></arr>
</arr>
Just a guess, I think for this case, we need to add one more field[the index for instance], for each inner array which will again be multivalued (which will store int values only)? How do I add the actual 2 dimensional array, how to pass the inner arrays and how to store the full doc that contains this 2 dimensional array. Please help me out sort this issue.
Please share your views and point me in the right direction. Any help would be highly appreciated.
I found similar things on the web, but not the one I'm looking for: http://lucene.472066.n3.nabble.com/Two-dimensional-array-in-Solr-schema-td4003309.html
Thanks
Re: Store 2 dimensional array( of int values) in solr 4.0
Posted by Jack Krupansky <ja...@basetechnology.com>.
You still haven't supplied any queries.
If all you really need is the JSON as a blob, simply store it as a string
and parse the JSON in your application layer.
-- Jack Krupansky
-----Original Message-----
From: A Geek
Sent: Friday, September 06, 2013 10:30 AM
To: solr user
Subject: RE: Store 2 dimensional array( of int values) in solr 4.0
Hi,Thanks for the quick reply. Sure, please find below the details as per
your query.
Essentially, I want to retrieve the doc through JSON [using JSON format as
SOLR result output]and want JSON to pick the the data from the dataX field
as a two dimensional array of ints. When I store the data as show below, it
shows up in JSON array of strings where the internal array is basically
shown as strings (because thats how the field is configured and I'm storing,
not finding any other option). Following is the current JSON output that I'm
able to fetch:
"dataX":["[20130614, 2]","[20130615, 11]","[20130616, 1]","[20130617,
1]","[20130619, 8]","[20130620, 5]","[20130623, 5]"]
whereas I want to fetch the dataX as something like:
"dataX":[[20130614, 2],[20130615, 11],[20130616, 1],[20130617, 1],[20130619,
8],[20130620, 5],[20130623, 5]]
as can be seen, the dataX is essentially a 2D array where the internal array
is of two ints, one being date and other being the count.
Please point me in the right direction. Appreciate your time.
Thanks.
> From: jack@basetechnology.com
> To: solr-user@lucene.apache.org
> Subject: Re: Store 2 dimensional array( of int values) in solr 4.0
> Date: Fri, 6 Sep 2013 08:44:06 -0400
>
> First you need to tell us how you wish to use and query the data. That
> will
> largely determine how the data must be stored. Give us a few example
> queries
> of how you would like your application to be able to access the data.
>
> Note that Lucene has only simple multivalued fields - no structure or
> nesting within a single field other that a list of scalar values.
>
> But you can always store a complex structure as a BSON blob or JSON string
> if all you want is to store and retrieve it in its entirety without
> querying
> its internal structure. And note that Lucene queries are field level -
> does
> a field contain or match a scalar value.
>
> -- Jack Krupansky
>
> -----Original Message-----
> From: A Geek
> Sent: Friday, September 06, 2013 7:10 AM
> To: solr user
> Subject: Store 2 dimensional array( of int values) in solr 4.0
>
> hi All, I'm trying to store a 2 dimensional array in SOLR [version 4.0].
> Basically I've the following data:
> [[20121108, 1],[20121110, 7],[20121111, 2],[20121112, 2]] ...
>
> The inner array being used to keep some count say X for that particular
> day.
> Currently, I'm using the following field to store this data:
> <field name="dataX" type="string" indexed="true" stored="true"
> multiValued="true"/>
> and I'm using python library pySolr to store the data. Currently the data
> that gets stored looks like this(its array of strings)
> <arr name="dataX"><str>[20121108, 1]</str><str>[20121110,
> 7]</str><str>[20121111, 2]</str><str>[20121112, 2]</str><str>[20121113,
> 2]</str><str>[20121116, 1]</str></arr>
> Is there a way, i can store the 2 dimensional array and the inner array
> can
> contain int values, like the one shown in the beginning example, such that
> the the final/stored data in SOLR looks something like: <arr name="dataX">
> <arr name=index><int>20121108</int> <int> 7 </int> </arr>
> <arr name=index><int> 20121110</int><int> 12 </int></arr>
> <arr name=index><int> 20121110</int><int> 12 </int></arr>
> </arr>
> Just a guess, I think for this case, we need to add one more field[the
> index
> for instance], for each inner array which will again be multivalued (which
> will store int values only)? How do I add the actual 2 dimensional array,
> how to pass the inner arrays and how to store the full doc that contains
> this 2 dimensional array. Please help me out sort this issue.
> Please share your views and point me in the right direction. Any help
> would
> be highly appreciated.
> I found similar things on the web, but not the one I'm looking for:
> http://lucene.472066.n3.nabble.com/Two-dimensional-array-in-Solr-schema-td4003309.html
> Thanks
>
RE: Store 2 dimensional array( of int values) in solr 4.0
Posted by A Geek <dw...@live.com>.
Hi,Thanks for the quick reply. Sure, please find below the details as per your query.
Essentially, I want to retrieve the doc through JSON [using JSON format as SOLR result output]and want JSON to pick the the data from the dataX field as a two dimensional array of ints. When I store the data as show below, it shows up in JSON array of strings where the internal array is basically shown as strings (because thats how the field is configured and I'm storing, not finding any other option). Following is the current JSON output that I'm able to fetch:
"dataX":["[20130614, 2]","[20130615, 11]","[20130616, 1]","[20130617, 1]","[20130619, 8]","[20130620, 5]","[20130623, 5]"]
whereas I want to fetch the dataX as something like:
"dataX":[[20130614, 2],[20130615, 11],[20130616, 1],[20130617, 1],[20130619, 8],[20130620, 5],[20130623, 5]]
as can be seen, the dataX is essentially a 2D array where the internal array is of two ints, one being date and other being the count.
Please point me in the right direction. Appreciate your time.
Thanks.
> From: jack@basetechnology.com
> To: solr-user@lucene.apache.org
> Subject: Re: Store 2 dimensional array( of int values) in solr 4.0
> Date: Fri, 6 Sep 2013 08:44:06 -0400
>
> First you need to tell us how you wish to use and query the data. That will
> largely determine how the data must be stored. Give us a few example queries
> of how you would like your application to be able to access the data.
>
> Note that Lucene has only simple multivalued fields - no structure or
> nesting within a single field other that a list of scalar values.
>
> But you can always store a complex structure as a BSON blob or JSON string
> if all you want is to store and retrieve it in its entirety without querying
> its internal structure. And note that Lucene queries are field level - does
> a field contain or match a scalar value.
>
> -- Jack Krupansky
>
> -----Original Message-----
> From: A Geek
> Sent: Friday, September 06, 2013 7:10 AM
> To: solr user
> Subject: Store 2 dimensional array( of int values) in solr 4.0
>
> hi All, I'm trying to store a 2 dimensional array in SOLR [version 4.0].
> Basically I've the following data:
> [[20121108, 1],[20121110, 7],[20121111, 2],[20121112, 2]] ...
>
> The inner array being used to keep some count say X for that particular day.
> Currently, I'm using the following field to store this data:
> <field name="dataX" type="string" indexed="true" stored="true"
> multiValued="true"/>
> and I'm using python library pySolr to store the data. Currently the data
> that gets stored looks like this(its array of strings)
> <arr name="dataX"><str>[20121108, 1]</str><str>[20121110,
> 7]</str><str>[20121111, 2]</str><str>[20121112, 2]</str><str>[20121113,
> 2]</str><str>[20121116, 1]</str></arr>
> Is there a way, i can store the 2 dimensional array and the inner array can
> contain int values, like the one shown in the beginning example, such that
> the the final/stored data in SOLR looks something like: <arr name="dataX">
> <arr name=index><int>20121108</int> <int> 7 </int> </arr>
> <arr name=index><int> 20121110</int><int> 12 </int></arr>
> <arr name=index><int> 20121110</int><int> 12 </int></arr>
> </arr>
> Just a guess, I think for this case, we need to add one more field[the index
> for instance], for each inner array which will again be multivalued (which
> will store int values only)? How do I add the actual 2 dimensional array,
> how to pass the inner arrays and how to store the full doc that contains
> this 2 dimensional array. Please help me out sort this issue.
> Please share your views and point me in the right direction. Any help would
> be highly appreciated.
> I found similar things on the web, but not the one I'm looking for:
> http://lucene.472066.n3.nabble.com/Two-dimensional-array-in-Solr-schema-td4003309.html
> Thanks
>
Re: Store 2 dimensional array( of int values) in solr 4.0
Posted by Jack Krupansky <ja...@basetechnology.com>.
First you need to tell us how you wish to use and query the data. That will
largely determine how the data must be stored. Give us a few example queries
of how you would like your application to be able to access the data.
Note that Lucene has only simple multivalued fields - no structure or
nesting within a single field other that a list of scalar values.
But you can always store a complex structure as a BSON blob or JSON string
if all you want is to store and retrieve it in its entirety without querying
its internal structure. And note that Lucene queries are field level - does
a field contain or match a scalar value.
-- Jack Krupansky
-----Original Message-----
From: A Geek
Sent: Friday, September 06, 2013 7:10 AM
To: solr user
Subject: Store 2 dimensional array( of int values) in solr 4.0
hi All, I'm trying to store a 2 dimensional array in SOLR [version 4.0].
Basically I've the following data:
[[20121108, 1],[20121110, 7],[20121111, 2],[20121112, 2]] ...
The inner array being used to keep some count say X for that particular day.
Currently, I'm using the following field to store this data:
<field name="dataX" type="string" indexed="true" stored="true"
multiValued="true"/>
and I'm using python library pySolr to store the data. Currently the data
that gets stored looks like this(its array of strings)
<arr name="dataX"><str>[20121108, 1]</str><str>[20121110,
7]</str><str>[20121111, 2]</str><str>[20121112, 2]</str><str>[20121113,
2]</str><str>[20121116, 1]</str></arr>
Is there a way, i can store the 2 dimensional array and the inner array can
contain int values, like the one shown in the beginning example, such that
the the final/stored data in SOLR looks something like: <arr name="dataX">
<arr name=index><int>20121108</int> <int> 7 </int> </arr>
<arr name=index><int> 20121110</int><int> 12 </int></arr>
<arr name=index><int> 20121110</int><int> 12 </int></arr>
</arr>
Just a guess, I think for this case, we need to add one more field[the index
for instance], for each inner array which will again be multivalued (which
will store int values only)? How do I add the actual 2 dimensional array,
how to pass the inner arrays and how to store the full doc that contains
this 2 dimensional array. Please help me out sort this issue.
Please share your views and point me in the right direction. Any help would
be highly appreciated.
I found similar things on the web, but not the one I'm looking for:
http://lucene.472066.n3.nabble.com/Two-dimensional-array-in-Solr-schema-td4003309.html
Thanks