You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Raymond Xie <xi...@gmail.com> on 2018/04/28 02:45:33 UTC
How to create a solr collection providing as much searching
flexibility as possible?
I have huge amount of data in FIX format (
https://en.wikipedia.org/wiki/Financial_Information_eXchange)
I want to give the data users the most flexibility to do their search,
usually like trading date range, order id or type, amount, ....
Can anyone share any experience on that?
Thanks.
*------------------------------------------------*
*Sincerely yours,*
*Raymond*
Re: How to create a solr collection providing as much searching
flexibility as possible?
Posted by Alessandro Benedetti <a....@sease.io>.
Hi Raymond,
as Charlie correctly stated, the input format is not that important, what is
important is to focus on your requirements and properly design a
configuration and data model to solve them.
Extracting the information for such a data format is not going to be
particularly challenging ( as i assume you know the semantic of such
structure).
You need to properly build your Solr document accordingly to the set of
features you want to expose.
Designing fields and field types will be fundamental to reach the search
flexibility you are looking for.
e.g.
*Feature*: expose a fast range search on a numerical field (Int)
*Implementation* :
<Index time>[1]
IntPointField
Integer field (32-bit signed integer). This class encodes int values using a
"Dimensional Points" based data structure that allows for very efficient
searches for specific values, or ranges of values. For single valued fields,
docValues="true" must be used to enable sorting.
<Query time>[2]
Regards
[1]
https://lucene.apache.org/solr/guide/7_3/field-types-included-with-solr.html
[2]
https://lucene.apache.org/solr/guide/7_3/the-standard-query-parser.html#range-searches
-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: How to create a solr collection providing as much searching
flexibility as possible?
Posted by Charlie Hull <ch...@flax.co.uk>.
On 29/04/2018 22:25, Raymond Xie wrote:
> Thank you Alessandro,
>
> It looks like my requirement is vague, but indeed I already indicated my
> data is in FIX format, which is a <tag:value> format, here is an example in
> the Wiki link in my original question:
>
> 8=FIX.4.2 | 9=178 | 35=8 | 49=PHLX | 56=PERS |
> 52=20071123-05:30:00.000 | 11=ATOMNOCCC9990900 | 20=3 | 150=E | 39=E |
> 55=MSFT | 167=CS | 54=1 | 38=15 | 40=2 | 44=15 | 58=PHLX EQUITY
> TESTING | 59=0 | 47=C | 32=0 | 31=0 | 151=15 | 14=0 | 6=0 | 10=128 |
>
> As the data format is quite special, and commonly used in Financial area
> (especially for trading data), I believe there must have been lots of
> studies already made. That's why I want to find out.
Hi,
Start with the search functionality you want to provide: which fields
should be covered by a standard search box; which fields should the user
be able to facet on; which should they be able to sort on. From these
requirements you should be able to work backwards and decide how to
index the data appropriately.
Cheers
Charlie
>
> Thank you.
>
>
>
>
> *------------------------------------------------*
> *Sincerely yours,*
>
>
> *Raymond*
>
> On Sat, Apr 28, 2018 at 11:32 AM, Alessandro Benedetti <a.benedetti@sease.io
>> wrote:
>
>> Hi Raymond,
>> your requirements are quite vague, Solr offers you those capabilities but
>> you need to model your configuration and data accordingly.
>>
>> https://lucene.apache.org/solr/guide/7_3/solr-tutorial.html
>> is a good starting point.
>> After that you can study your requirements and design the search solution
>> accordingly.
>>
>> Cheers
>>
>>
>>
>> -----
>> ---------------
>> Alessandro Benedetti
>> Search Consultant, R&D Software Engineer, Director
>> Sease Ltd. - www.sease.io
>> --
>> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>>
>
--
Charlie Hull
Flax - Open Source Enterprise Search
tel/fax: +44 (0)8700 118334
mobile: +44 (0)7767 825828
web: www.flax.co.uk
Re: How to create a solr collection providing as much searching
flexibility as possible?
Posted by Raymond Xie <xi...@gmail.com>.
Thank you Alessandro,
It looks like my requirement is vague, but indeed I already indicated my
data is in FIX format, which is a <tag:value> format, here is an example in
the Wiki link in my original question:
8=FIX.4.2 | 9=178 | 35=8 | 49=PHLX | 56=PERS |
52=20071123-05:30:00.000 | 11=ATOMNOCCC9990900 | 20=3 | 150=E | 39=E |
55=MSFT | 167=CS | 54=1 | 38=15 | 40=2 | 44=15 | 58=PHLX EQUITY
TESTING | 59=0 | 47=C | 32=0 | 31=0 | 151=15 | 14=0 | 6=0 | 10=128 |
As the data format is quite special, and commonly used in Financial area
(especially for trading data), I believe there must have been lots of
studies already made. That's why I want to find out.
Thank you.
*------------------------------------------------*
*Sincerely yours,*
*Raymond*
On Sat, Apr 28, 2018 at 11:32 AM, Alessandro Benedetti <a.benedetti@sease.io
> wrote:
> Hi Raymond,
> your requirements are quite vague, Solr offers you those capabilities but
> you need to model your configuration and data accordingly.
>
> https://lucene.apache.org/solr/guide/7_3/solr-tutorial.html
> is a good starting point.
> After that you can study your requirements and design the search solution
> accordingly.
>
> Cheers
>
>
>
> -----
> ---------------
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
Re: How to create a solr collection providing as much searching
flexibility as possible?
Posted by Alessandro Benedetti <a....@sease.io>.
Hi Raymond,
your requirements are quite vague, Solr offers you those capabilities but
you need to model your configuration and data accordingly.
https://lucene.apache.org/solr/guide/7_3/solr-tutorial.html
is a good starting point.
After that you can study your requirements and design the search solution
accordingly.
Cheers
-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html