You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Raymond Xie <xi...@gmail.com> on 2018/04/05 10:56:37 UTC
How to create my schema and add document, thank you
I have the data ready for index now, it is a json file:
{"122": "20180320-08:08:35.038", "49": "VIPER", "382": "0", "151": "1.0",
"9": "653", "10071": "20180320-08:08:35.088", "15": "JPY", "56": "XSVC",
"54": "1", "10202": "APMKTMAKING", "10537": "XOSE", "10217": "Y", "48":
"179492540", "201": "1", "40": "2", "8": "FIX.4.4", "167": "OPT", "421":
"JPN", "10292": "115", "10184": "337912000000002", "456": "101", "11210":
"337912000000002", "1133": "G", "10515": "178", "10": "200", "11032": "-1",
"10436": "20180320-08:08:35.038", "10518": "178", "11":
"337912000000002", "75":
"20180320", "10005": "178", "10104": "Y", "35": "RIO", "10208":
"APAC.VIPER.OOE", "59": "0", "60": "20180320-08:08:35.088", "528": "P",
"581": "13", "1": "TEST", "202": "25375.0", "455": "179492540", "55":
"JNI253D8.OS", "100": "XOSE", "52": "20180320-08:08:35.088", "10241":
"viperooe", "150": "A", "10039": "viperooe", "39": "A", "10438": "RIO.4.5",
"38": "1", "37": "337912000000002", "372": "D", "660": "102", "44": "2.0",
"10066": "20180320-08:08:35.038", "29": "4", "50": "JPNIK01", "22": "101"}
You can inspect the json here: https://jsonformatter.org/
I need to create index and enable searching on tags: 37, 75 and 10242
(where available, this sample message doesn't have it)
My understanding is I need to create the file managed-schema, I added two
fields as below:
<field name="order_id" type="text_general" indexed="true" stored="false"
multiValued="true"/>
<field name="trd_date" type="text_general" indexed="true"
stored="false" multiValued="true"/>
Then I go back to Solr Admin, I don't see the two new fields in Schema
section
Anything I am missing here? and once the two fields are put in the
managed-schema, can I add the json file through upload in Solr Admin?
Thank you very much.
*------------------------------------------------*
*Sincerely yours,*
*Raymond*
Re: How to create my schema and add document, thank you
Posted by Raymond Xie <xi...@gmail.com>.
Thanks.
I have moved this to the next stage:
1. 3 fields are to be extracted from the raw files, the location of the raw
file is the fourth field, all the 4 fields become a document
2. Only the 3 fields will be index'ed?
3. The search result should be kind of re-formatted to include the 3 fields
and the content by parsing the location and retrieve the original raw
data's content.
Current challenge is: the raw data is zipped, I am not sure what can I do
to process the zipped file, can Solr handle that?
My plan is below:
AJava (?) based website will be created, a GUI is created for accepting
user's keyword, the keyword will be used to form a POST url, the url will
be used to GET response from Solr, the response will contain the
correspondent message location, the java program will fetch the zipped file
and unzip it, as one zip file could contain multiple messages, the java
program will need to parse out the matched message(s), the parsed matched
message(s) will be shown to the end user together with the other meta data
(the three index'ed fields).
Is this a feasible plan? is there better solution?
Thank you very much.
*------------------------------------------------*
*Sincerely yours,*
*Raymond*
On Thu, Apr 5, 2018 at 7:11 AM, Adhyan Arizki <a....@gmail.com> wrote:
> Raymond,
>
> 1. Please ensure your Solr instance does indeed load up the correct
> managed-schema file. You do not need to create the file, it should have
> been created automatically in the newer version of Solr out of the box. you
> just need to edit it
> 2. Have you reload your instance after you made the modification?
>
> On Thu, Apr 5, 2018 at 6:56 PM, Raymond Xie <xi...@gmail.com> wrote:
>
> > I have the data ready for index now, it is a json file:
> >
> > {"122": "20180320-08:08:35.038", "49": "VIPER", "382": "0", "151": "1.0",
> > "9": "653", "10071": "20180320-08:08:35.088", "15": "JPY", "56": "XSVC",
> > "54": "1", "10202": "APMKTMAKING", "10537": "XOSE", "10217": "Y", "48":
> > "179492540", "201": "1", "40": "2", "8": "FIX.4.4", "167": "OPT", "421":
> > "JPN", "10292": "115", "10184": "337912000000002", "456": "101", "11210":
> > "337912000000002", "1133": "G", "10515": "178", "10": "200", "11032":
> "-1",
> > "10436": "20180320-08:08:35.038", "10518": "178", "11":
> > "337912000000002", "75":
> > "20180320", "10005": "178", "10104": "Y", "35": "RIO", "10208":
> > "APAC.VIPER.OOE", "59": "0", "60": "20180320-08:08:35.088", "528": "P",
> > "581": "13", "1": "TEST", "202": "25375.0", "455": "179492540", "55":
> > "JNI253D8.OS", "100": "XOSE", "52": "20180320-08:08:35.088", "10241":
> > "viperooe", "150": "A", "10039": "viperooe", "39": "A", "10438":
> "RIO.4.5",
> > "38": "1", "37": "337912000000002", "372": "D", "660": "102", "44":
> "2.0",
> > "10066": "20180320-08:08:35.038", "29": "4", "50": "JPNIK01", "22":
> "101"}
> >
> > You can inspect the json here: https://jsonformatter.org/
> >
> > I need to create index and enable searching on tags: 37, 75 and 10242
> > (where available, this sample message doesn't have it)
> >
> > My understanding is I need to create the file managed-schema, I added two
> > fields as below:
> >
> > <field name="order_id" type="text_general" indexed="true" stored="false"
> > multiValued="true"/>
> > <field name="trd_date" type="text_general" indexed="true"
> > stored="false" multiValued="true"/>
> >
> > Then I go back to Solr Admin, I don't see the two new fields in Schema
> > section
> >
> > Anything I am missing here? and once the two fields are put in the
> > managed-schema, can I add the json file through upload in Solr Admin?
> >
> > Thank you very much.
> >
> >
> > *------------------------------------------------*
> > *Sincerely yours,*
> >
> >
> > *Raymond*
> >
>
>
>
> --
>
> Best regards,
> Adhyan Arizki
>
Re: How to create my schema and add document, thank you
Posted by Adhyan Arizki <a....@gmail.com>.
Raymond,
1. Please ensure your Solr instance does indeed load up the correct
managed-schema file. You do not need to create the file, it should have
been created automatically in the newer version of Solr out of the box. you
just need to edit it
2. Have you reload your instance after you made the modification?
On Thu, Apr 5, 2018 at 6:56 PM, Raymond Xie <xi...@gmail.com> wrote:
> I have the data ready for index now, it is a json file:
>
> {"122": "20180320-08:08:35.038", "49": "VIPER", "382": "0", "151": "1.0",
> "9": "653", "10071": "20180320-08:08:35.088", "15": "JPY", "56": "XSVC",
> "54": "1", "10202": "APMKTMAKING", "10537": "XOSE", "10217": "Y", "48":
> "179492540", "201": "1", "40": "2", "8": "FIX.4.4", "167": "OPT", "421":
> "JPN", "10292": "115", "10184": "337912000000002", "456": "101", "11210":
> "337912000000002", "1133": "G", "10515": "178", "10": "200", "11032": "-1",
> "10436": "20180320-08:08:35.038", "10518": "178", "11":
> "337912000000002", "75":
> "20180320", "10005": "178", "10104": "Y", "35": "RIO", "10208":
> "APAC.VIPER.OOE", "59": "0", "60": "20180320-08:08:35.088", "528": "P",
> "581": "13", "1": "TEST", "202": "25375.0", "455": "179492540", "55":
> "JNI253D8.OS", "100": "XOSE", "52": "20180320-08:08:35.088", "10241":
> "viperooe", "150": "A", "10039": "viperooe", "39": "A", "10438": "RIO.4.5",
> "38": "1", "37": "337912000000002", "372": "D", "660": "102", "44": "2.0",
> "10066": "20180320-08:08:35.038", "29": "4", "50": "JPNIK01", "22": "101"}
>
> You can inspect the json here: https://jsonformatter.org/
>
> I need to create index and enable searching on tags: 37, 75 and 10242
> (where available, this sample message doesn't have it)
>
> My understanding is I need to create the file managed-schema, I added two
> fields as below:
>
> <field name="order_id" type="text_general" indexed="true" stored="false"
> multiValued="true"/>
> <field name="trd_date" type="text_general" indexed="true"
> stored="false" multiValued="true"/>
>
> Then I go back to Solr Admin, I don't see the two new fields in Schema
> section
>
> Anything I am missing here? and once the two fields are put in the
> managed-schema, can I add the json file through upload in Solr Admin?
>
> Thank you very much.
>
>
> *------------------------------------------------*
> *Sincerely yours,*
>
>
> *Raymond*
>
--
Best regards,
Adhyan Arizki