You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Baskar Sikkayan <ba...@gmail.com> on 2013/09/15 02:08:42 UTC
Solr Java Client
Hi,
I am new to Solr and trying to use Solr java client instead of using the
Data handler.
Is there any configuration i need to do for this?
I got the following sample code.
SolrInputDocument doc = new SolrInputDocument();
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
server.commit(); // periodically flush
I am confused here. I am going to index 3 different tables for 3 different
kind of searches. Here i dont have any option to differentiate 3 kind of
indexes.
Am i missing anything here. Could anyone please shed some light here?
Thanks,
Baskar.S
Re: Solr Java Client
Posted by Gora Mohanty <go...@mimirtech.com>.
Dear Baskar,
It seems that you are trying to jump into the
middle, and have a complete understanding
of everything before trying things. This makes
things difficult both for yourself, and for people
trying to help you. Solr can be a little complex
for beginners, so it is best to approach things
a step at a time.
I would suggest:
* Learn how to index from a single database
table into Solr. The Wiki links that I referred
you to should help. It might be easiest to get
started with the DataImportHandler using the
configuration in example/example-DIH/solr/db/
in the Solr source code.
* Familiarise yourself with how queries work
in Solr, in particular how to use the default
search field, and specific fields.
* Set up SolrJ to index the same table as with
DIH
* Move on to multiple, possibly related tables
only once you understand the above.
Regards,
Gora
Re: Solr Java Client
Posted by Baskar Sikkayan <ba...@gmail.com>.
Hi Gora,
Thanks a lot for your reply.
"As people have already advised you, the best way to decide
how to organise your data in the Solr index depends on the
searches that you want to make. This is not entirely clear
from your description above. The flattening sample that you
show above would be suitable if the user is to search by
'child' attributes, but can be simplified otherwise."
*Yes, the search is based on the child attributes.*
"How would you know that the user name has been changed?
Is there a modification date for that table. If so, it would make
sense to check that against the last time indexing to Solr was
done. A DIH delta-import makes this straightforward."
*As of now, there is no special column to know if the username has been
changed.
*
*But, whenever the user update his name, i can track that in my java code
and send the update to Solr.
*
*Here, I am planning to use Solr java client.
*
*But, all the above things are possible with Java client and also with
delta-import.
*
*I am looking for changing the solr data whenever there is a change in the
database.*
*Even there is a small delay i am fine with that.
*
*Which one you will suggest?
*
*Solr Java client or DIH delta-import????
*
*My application running on server A, database on server B and solr will be
on server C.
*
*If i am supposed to use, Solr Java client, i may need to hit the database
sometimes to get some parent data and then need to send the same to Solr.
*
*Guess, its a unnecessary trip.
*
*So confused here, if i need to go with Java client or DIH delta import.
*
Thanks,
Baskar.S
On Mon, Sep 16, 2013 at 9:23 AM, Gora Mohanty <go...@mimirtech.com> wrote:
> On 16 September 2013 02:47, Baskar Sikkayan <ba...@gmail.com> wrote:
> [...]
> > Have a question now.
> >
> > I know in solr its flat file system and the data will be in denormalized
> > form.
> >
> > My question :
> >
> > Have 3 tables,
> >
> > 1) user (userid, firstname, lastname, ...)
> > 2) master (masterid, skills, ...)
> > 3) child (childid, masterid, userid, ...)
> >
> > In solr, i have added all these field for each document.
> >
> > Example,
> >
> > childid,masterid,userid,skills,firstname,lastname
> >
> > Real Data Example,
> >
> > 1(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
> > 2(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
> > 3(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
>
> As people have already advised you, the best way to decide
> how to organise your data in the Solr index depends on the
> searches that you want to make. This is not entirely clear
> from your description above. The flattening sample that you
> show above would be suitable if the user is to search by
> 'child' attributes, but can be simplified otherwise.
>
> > The above data sample is from solr document.
> > In my search result, i will have to show all these fields.
> >
> > User may change the name at any time.The same has to be updated in solr.
> >
> > In this case, i need to find all the child id that belongs to the user
> and
> > update the username with those child ids.
> >
> > Please tell me if there is any other better approach than this.
>
> How would you know that the user name has been changed?
> Is there a modification date for that table. If so, it would make
> sense to check that against the last time indexing to Solr was
> done. A DIH delta-import makes this straightforward.
>
> Updates as you suggest above would be the normal way to handle
> things. You should batch your updates, say by running an update
> script at periodic intervals.
>
> Regards,
> Gora
>
Re: Solr Java Client
Posted by Gora Mohanty <go...@mimirtech.com>.
On 16 September 2013 02:47, Baskar Sikkayan <ba...@gmail.com> wrote:
[...]
> Have a question now.
>
> I know in solr its flat file system and the data will be in denormalized
> form.
>
> My question :
>
> Have 3 tables,
>
> 1) user (userid, firstname, lastname, ...)
> 2) master (masterid, skills, ...)
> 3) child (childid, masterid, userid, ...)
>
> In solr, i have added all these field for each document.
>
> Example,
>
> childid,masterid,userid,skills,firstname,lastname
>
> Real Data Example,
>
> 1(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
> 2(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
> 3(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
As people have already advised you, the best way to decide
how to organise your data in the Solr index depends on the
searches that you want to make. This is not entirely clear
from your description above. The flattening sample that you
show above would be suitable if the user is to search by
'child' attributes, but can be simplified otherwise.
> The above data sample is from solr document.
> In my search result, i will have to show all these fields.
>
> User may change the name at any time.The same has to be updated in solr.
>
> In this case, i need to find all the child id that belongs to the user and
> update the username with those child ids.
>
> Please tell me if there is any other better approach than this.
How would you know that the user name has been changed?
Is there a modification date for that table. If so, it would make
sense to check that against the last time indexing to Solr was
done. A DIH delta-import makes this straightforward.
Updates as you suggest above would be the normal way to handle
things. You should batch your updates, say by running an update
script at periodic intervals.
Regards,
Gora
Re: Solr Java Client
Posted by Baskar Sikkayan <ba...@gmail.com>.
Hi,
I read few chapters of solr cookbook and hope, I have good basic
understanding of Apache solr and Datahandler.
I was able to modify the field and all. I am able to understand now, what i
was advised to do.
Have a question now.
I know in solr its flat file system and the data will be in denormalized
form.
My question :
Have 3 tables,
1) user (userid, firstname, lastname, ...)
2) master (masterid, skills, ...)
3) child (childid, masterid, userid, ...)
In solr, i have added all these field for each document.
Example,
childid,masterid,userid,skills,firstname,lastname
Real Data Example,
1(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
2(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
3(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
The above data sample is from solr document.
In my search result, i will have to show all these fields.
User may change the name at any time.The same has to be updated in solr.
In this case, i need to find all the child id that belongs to the user and
update the username with those child ids.
Please tell me if there is any other better approach than this.
Thanks,
Baskar.S
On Sun, Sep 15, 2013 at 9:42 PM, Gora Mohanty <go...@mimirtech.com> wrote:
> On 15 September 2013 21:36, Baskar Sikkayan <ba...@gmail.com> wrote:
> >
> > Hi,
> > Thanks for the reply. I got it now. Is there XSD for schema.xml and
> > solrconfig.xml?
> > Also, if i have two different fields with different filed type, how can
> > differentiate it in SolrJava client.
> >
> > SolrInputDocument doc = new SolrInputDocument();
> >
> > doc.addField("cat", "book");
> > doc.addField("id", "book-" + i);
> > doc.addField("name", "The Legend of the Hobbit part " + i);
> > server.add(doc);
> >
> > Is there any option here to put the field type?
> [...]
>
> Field types are defined in schema.xml.
>
> I would again suggest approaching things systematically,
> and understanding how the various pieces work.
>
> Regards,
> Gora
>
Re: Solr Java Client
Posted by Gora Mohanty <go...@mimirtech.com>.
On 15 September 2013 21:36, Baskar Sikkayan <ba...@gmail.com> wrote:
>
> Hi,
> Thanks for the reply. I got it now. Is there XSD for schema.xml and
> solrconfig.xml?
> Also, if i have two different fields with different filed type, how can
> differentiate it in SolrJava client.
>
> SolrInputDocument doc = new SolrInputDocument();
>
> doc.addField("cat", "book");
> doc.addField("id", "book-" + i);
> doc.addField("name", "The Legend of the Hobbit part " + i);
> server.add(doc);
>
> Is there any option here to put the field type?
[...]
Field types are defined in schema.xml.
I would again suggest approaching things systematically,
and understanding how the various pieces work.
Regards,
Gora
Re: Solr Java Client
Posted by Baskar Sikkayan <ba...@gmail.com>.
Hi,
Thanks for the reply. I got it now. Is there XSD for schema.xml and
solrconfig.xml?
Also, if i have two different fields with different filed type, how can
differentiate it in SolrJava client.
SolrInputDocument doc = new SolrInputDocument();
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
Is there any option here to put the field type?
Thanks,
Baskar.S
On Sun, Sep 15, 2013 at 9:20 PM, Erick Erickson <er...@gmail.com>wrote:
> Put the data in two different fields. You can also include
> a type field and use the fq=type:type1 or fq=type:type2
> if you need to distinguish them separately.
>
> Best,
> Erick
>
>
> On Sun, Sep 15, 2013 at 11:00 AM, Baskar Sikkayan <baskar.sks@gmail.com
> >wrote:
>
> > Hi Erick,
> > Thanks a lot for your reply. I am still not clear.
> > I will have 2 different searches.
> > So, there will be 2 different kind of documents with different fields.
> >
> > But the example below, gives me a impression that
> >
> > SolrInputDocument doc = new SolrInputDocument();even there are 2 diff
> > searches, the data is getting stored in the same
> >
> > doc.addField("cat", "book");
> > doc.addField("id", "book-" + i);
> > doc.addField("name", "The Legend of the Hobbit part " + i);
> > server.add(doc);
> >
> > If i use the same doc, how to diff data for 2 different searches.
> > Could you explain me clear on this??????//
> >
> >
> > Thanks,
> > Baskar.S
> >
> >
> >
> > On Sun, Sep 15, 2013 at 7:57 PM, Erick Erickson <erickerickson@gmail.com
> > >wrote:
> >
> > > Listen to Wunder. You really have to switch
> > > gears and think in terms of searching rather
> > > than database tables.
> > >
> > > First, there's no requirement that all documents
> > > have the same fields. There's very little penalty
> > > for this.
> > >
> > > Second, you have to get over normalized data.
> > > It's unclear how many employees/records you have,
> > > but unless it's a really big set of records (i.e multiple
> > > tens of millions) you don't have to worry much
> > > about saving space.
> > >
> > > BTW, your java example works, but you do NOT
> > > want to commit after every document. Configure
> > > your auto soft and hard commits to do this on a
> > > time-based basis.
> > >
> > > So I'd recommend you approach it differently.
> > > Don't get stuck on your current data model. Instead
> > > define the searches you want to make and _then_
> > > decide the appropriate fields in the docs.
> > >
> > > But to your question, you can certainly create
> > > two cores. That's a good strategy if there is
> > > little data shared between the two types of users.
> > > It's probably conceptually cleaner as well.
> > >
> > > Best,
> > > Erick
> > >
> > >
> > > On Sun, Sep 15, 2013 at 8:18 AM, Baskar Sikkayan <baskar.sks@gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > > Thank you very much for your reply.
> > > > Let me clearly explain my requirement.
> > > >
> > > > Its a kind of job site.
> > > >
> > > > I have 2 tables that need to be added in solr.
> > > >
> > > > 1) employee ( id, name, skills, location, .....)
> > > > 2) job_postings ( id, name, job title, description, salary,
> > date_posted,
> > > > ....)
> > > >
> > > >
> > > > Here, there are 2 different kind of searches in my application.
> > > >
> > > > 1) employee search by employer
> > > > 2) job postings search by employee
> > > >
> > > > New row can be added at any time. Also the existing row can be
> altered
> > at
> > > > any time. For example, employee mobile number can be changed at any
> > time
> > > > and the solr data has to be updated.
> > > >
> > > > Here there are 2 different searches. So i need to maintain two
> > different
> > > > kind of indexes in solr.
> > > >
> > > > The java example says, to add a document, below is the sample code.
> My
> > > > doubt is, how can i add two different kind of search data here.
> > > > Should i create a 2 different core for this 2 different searches?
> > > >
> > > > SolrInputDocument doc = new SolrInputDocument();
> > > >
> > > > doc.addField("cat", "book");
> > > > doc.addField("id", "book-" + i);
> > > > doc.addField("name", "The Legend of the Hobbit part " +
> > i);
> > > > server.add(doc);
> > > > server.commit();
> > > >
> > > > Thanks,
> > > > Baskar.S
> > > >
> > > >
> > > > On Sun, Sep 15, 2013 at 9:14 AM, Walter Underwood <
> > wunder@wunderwood.org
> > > > >wrote:
> > > >
> > > > > Solr does not have tables, and you can't add an index.
> > > > >
> > > > > Solr's data model is flat, like a single table with lots of
> columns.
> > > > Think
> > > > > about creating a view where each row has all the information for
> one
> > > > search
> > > > > result. It includes everything that is searched (indexed=true in
> the
> > > > > schema) and returned (stored=true in the schema).
> > > > >
> > > > > wunder
> > > > >
> > > > > On Sep 14, 2013, at 7:32 PM, Amit Jha wrote:
> > > > >
> > > > > > Question is not clear to me. Please be more elaborative in your
> > > query.
> > > > > Why do u want to store index to DB tables?
> > > > > >
> > > > > > Rgds
> > > > > > AJ
> > > > > >
> > > > > > On 15-Sep-2013, at 7:20, Baskar Sikkayan <ba...@gmail.com>
> > > wrote:
> > > > > >
> > > > > >> How to add index to 3 diff tables from java ...
> > > > > >>
> > > > > >>
> > > > > >> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha <shanuu.jha@gmail.com
> >
> > > > wrote:
> > > > > >>
> > > > > >>> Add a field called "source" in schema.xml and value would be
> your
> > > > table
> > > > > >>> names.
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>> Rgds
> > > > > >>> AJ
> > > > > >>>
> > > > > >>> On 15-Sep-2013, at 5:38, Baskar Sikkayan <baskar.sks@gmail.com
> >
> > > > wrote:
> > > > > >>>
> > > > > >>>> Hi,
> > > > > >>>> I am new to Solr and trying to use Solr java client instead of
> > > using
> > > > > the
> > > > > >>>> Data handler.
> > > > > >>>> Is there any configuration i need to do for this?
> > > > > >>>>
> > > > > >>>> I got the following sample code.
> > > > > >>>>
> > > > > >>>> SolrInputDocument doc = new SolrInputDocument();
> > > > > >>>>
> > > > > >>>> doc.addField("cat", "book");
> > > > > >>>> doc.addField("id", "book-" + i);
> > > > > >>>> doc.addField("name", "The Legend of the Hobbit part
> > " +
> > > > i);
> > > > > >>>> server.add(doc);
> > > > > >>>> server.commit(); // periodically flush
> > > > > >>>>
> > > > > >>>> I am confused here. I am going to index 3 different tables
> for 3
> > > > > >>> different
> > > > > >>>> kind of searches. Here i dont have any option to
> differentiate 3
> > > > kind
> > > > > of
> > > > > >>>> indexes.
> > > > > >>>> Am i missing anything here. Could anyone please shed some
> light
> > > > here?
> > > > > >>>>
> > > > > >>>> Thanks,
> > > > > >>>> Baskar.S
> > > > > >>>
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Solr Java Client
Posted by Erick Erickson <er...@gmail.com>.
Put the data in two different fields. You can also include
a type field and use the fq=type:type1 or fq=type:type2
if you need to distinguish them separately.
Best,
Erick
On Sun, Sep 15, 2013 at 11:00 AM, Baskar Sikkayan <ba...@gmail.com>wrote:
> Hi Erick,
> Thanks a lot for your reply. I am still not clear.
> I will have 2 different searches.
> So, there will be 2 different kind of documents with different fields.
>
> But the example below, gives me a impression that
>
> SolrInputDocument doc = new SolrInputDocument();even there are 2 diff
> searches, the data is getting stored in the same
>
> doc.addField("cat", "book");
> doc.addField("id", "book-" + i);
> doc.addField("name", "The Legend of the Hobbit part " + i);
> server.add(doc);
>
> If i use the same doc, how to diff data for 2 different searches.
> Could you explain me clear on this??????//
>
>
> Thanks,
> Baskar.S
>
>
>
> On Sun, Sep 15, 2013 at 7:57 PM, Erick Erickson <erickerickson@gmail.com
> >wrote:
>
> > Listen to Wunder. You really have to switch
> > gears and think in terms of searching rather
> > than database tables.
> >
> > First, there's no requirement that all documents
> > have the same fields. There's very little penalty
> > for this.
> >
> > Second, you have to get over normalized data.
> > It's unclear how many employees/records you have,
> > but unless it's a really big set of records (i.e multiple
> > tens of millions) you don't have to worry much
> > about saving space.
> >
> > BTW, your java example works, but you do NOT
> > want to commit after every document. Configure
> > your auto soft and hard commits to do this on a
> > time-based basis.
> >
> > So I'd recommend you approach it differently.
> > Don't get stuck on your current data model. Instead
> > define the searches you want to make and _then_
> > decide the appropriate fields in the docs.
> >
> > But to your question, you can certainly create
> > two cores. That's a good strategy if there is
> > little data shared between the two types of users.
> > It's probably conceptually cleaner as well.
> >
> > Best,
> > Erick
> >
> >
> > On Sun, Sep 15, 2013 at 8:18 AM, Baskar Sikkayan <baskar.sks@gmail.com
> > >wrote:
> >
> > > Hi,
> > > Thank you very much for your reply.
> > > Let me clearly explain my requirement.
> > >
> > > Its a kind of job site.
> > >
> > > I have 2 tables that need to be added in solr.
> > >
> > > 1) employee ( id, name, skills, location, .....)
> > > 2) job_postings ( id, name, job title, description, salary,
> date_posted,
> > > ....)
> > >
> > >
> > > Here, there are 2 different kind of searches in my application.
> > >
> > > 1) employee search by employer
> > > 2) job postings search by employee
> > >
> > > New row can be added at any time. Also the existing row can be altered
> at
> > > any time. For example, employee mobile number can be changed at any
> time
> > > and the solr data has to be updated.
> > >
> > > Here there are 2 different searches. So i need to maintain two
> different
> > > kind of indexes in solr.
> > >
> > > The java example says, to add a document, below is the sample code. My
> > > doubt is, how can i add two different kind of search data here.
> > > Should i create a 2 different core for this 2 different searches?
> > >
> > > SolrInputDocument doc = new SolrInputDocument();
> > >
> > > doc.addField("cat", "book");
> > > doc.addField("id", "book-" + i);
> > > doc.addField("name", "The Legend of the Hobbit part " +
> i);
> > > server.add(doc);
> > > server.commit();
> > >
> > > Thanks,
> > > Baskar.S
> > >
> > >
> > > On Sun, Sep 15, 2013 at 9:14 AM, Walter Underwood <
> wunder@wunderwood.org
> > > >wrote:
> > >
> > > > Solr does not have tables, and you can't add an index.
> > > >
> > > > Solr's data model is flat, like a single table with lots of columns.
> > > Think
> > > > about creating a view where each row has all the information for one
> > > search
> > > > result. It includes everything that is searched (indexed=true in the
> > > > schema) and returned (stored=true in the schema).
> > > >
> > > > wunder
> > > >
> > > > On Sep 14, 2013, at 7:32 PM, Amit Jha wrote:
> > > >
> > > > > Question is not clear to me. Please be more elaborative in your
> > query.
> > > > Why do u want to store index to DB tables?
> > > > >
> > > > > Rgds
> > > > > AJ
> > > > >
> > > > > On 15-Sep-2013, at 7:20, Baskar Sikkayan <ba...@gmail.com>
> > wrote:
> > > > >
> > > > >> How to add index to 3 diff tables from java ...
> > > > >>
> > > > >>
> > > > >> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha <sh...@gmail.com>
> > > wrote:
> > > > >>
> > > > >>> Add a field called "source" in schema.xml and value would be your
> > > table
> > > > >>> names.
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> Rgds
> > > > >>> AJ
> > > > >>>
> > > > >>> On 15-Sep-2013, at 5:38, Baskar Sikkayan <ba...@gmail.com>
> > > wrote:
> > > > >>>
> > > > >>>> Hi,
> > > > >>>> I am new to Solr and trying to use Solr java client instead of
> > using
> > > > the
> > > > >>>> Data handler.
> > > > >>>> Is there any configuration i need to do for this?
> > > > >>>>
> > > > >>>> I got the following sample code.
> > > > >>>>
> > > > >>>> SolrInputDocument doc = new SolrInputDocument();
> > > > >>>>
> > > > >>>> doc.addField("cat", "book");
> > > > >>>> doc.addField("id", "book-" + i);
> > > > >>>> doc.addField("name", "The Legend of the Hobbit part
> " +
> > > i);
> > > > >>>> server.add(doc);
> > > > >>>> server.commit(); // periodically flush
> > > > >>>>
> > > > >>>> I am confused here. I am going to index 3 different tables for 3
> > > > >>> different
> > > > >>>> kind of searches. Here i dont have any option to differentiate 3
> > > kind
> > > > of
> > > > >>>> indexes.
> > > > >>>> Am i missing anything here. Could anyone please shed some light
> > > here?
> > > > >>>>
> > > > >>>> Thanks,
> > > > >>>> Baskar.S
> > > > >>>
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
>
Re: Solr Java Client
Posted by Baskar Sikkayan <ba...@gmail.com>.
Hi Erick,
Thanks a lot for your reply. I am still not clear.
I will have 2 different searches.
So, there will be 2 different kind of documents with different fields.
But the example below, gives me a impression that
SolrInputDocument doc = new SolrInputDocument();even there are 2 diff
searches, the data is getting stored in the same
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
If i use the same doc, how to diff data for 2 different searches.
Could you explain me clear on this??????//
Thanks,
Baskar.S
On Sun, Sep 15, 2013 at 7:57 PM, Erick Erickson <er...@gmail.com>wrote:
> Listen to Wunder. You really have to switch
> gears and think in terms of searching rather
> than database tables.
>
> First, there's no requirement that all documents
> have the same fields. There's very little penalty
> for this.
>
> Second, you have to get over normalized data.
> It's unclear how many employees/records you have,
> but unless it's a really big set of records (i.e multiple
> tens of millions) you don't have to worry much
> about saving space.
>
> BTW, your java example works, but you do NOT
> want to commit after every document. Configure
> your auto soft and hard commits to do this on a
> time-based basis.
>
> So I'd recommend you approach it differently.
> Don't get stuck on your current data model. Instead
> define the searches you want to make and _then_
> decide the appropriate fields in the docs.
>
> But to your question, you can certainly create
> two cores. That's a good strategy if there is
> little data shared between the two types of users.
> It's probably conceptually cleaner as well.
>
> Best,
> Erick
>
>
> On Sun, Sep 15, 2013 at 8:18 AM, Baskar Sikkayan <baskar.sks@gmail.com
> >wrote:
>
> > Hi,
> > Thank you very much for your reply.
> > Let me clearly explain my requirement.
> >
> > Its a kind of job site.
> >
> > I have 2 tables that need to be added in solr.
> >
> > 1) employee ( id, name, skills, location, .....)
> > 2) job_postings ( id, name, job title, description, salary, date_posted,
> > ....)
> >
> >
> > Here, there are 2 different kind of searches in my application.
> >
> > 1) employee search by employer
> > 2) job postings search by employee
> >
> > New row can be added at any time. Also the existing row can be altered at
> > any time. For example, employee mobile number can be changed at any time
> > and the solr data has to be updated.
> >
> > Here there are 2 different searches. So i need to maintain two different
> > kind of indexes in solr.
> >
> > The java example says, to add a document, below is the sample code. My
> > doubt is, how can i add two different kind of search data here.
> > Should i create a 2 different core for this 2 different searches?
> >
> > SolrInputDocument doc = new SolrInputDocument();
> >
> > doc.addField("cat", "book");
> > doc.addField("id", "book-" + i);
> > doc.addField("name", "The Legend of the Hobbit part " + i);
> > server.add(doc);
> > server.commit();
> >
> > Thanks,
> > Baskar.S
> >
> >
> > On Sun, Sep 15, 2013 at 9:14 AM, Walter Underwood <wunder@wunderwood.org
> > >wrote:
> >
> > > Solr does not have tables, and you can't add an index.
> > >
> > > Solr's data model is flat, like a single table with lots of columns.
> > Think
> > > about creating a view where each row has all the information for one
> > search
> > > result. It includes everything that is searched (indexed=true in the
> > > schema) and returned (stored=true in the schema).
> > >
> > > wunder
> > >
> > > On Sep 14, 2013, at 7:32 PM, Amit Jha wrote:
> > >
> > > > Question is not clear to me. Please be more elaborative in your
> query.
> > > Why do u want to store index to DB tables?
> > > >
> > > > Rgds
> > > > AJ
> > > >
> > > > On 15-Sep-2013, at 7:20, Baskar Sikkayan <ba...@gmail.com>
> wrote:
> > > >
> > > >> How to add index to 3 diff tables from java ...
> > > >>
> > > >>
> > > >> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha <sh...@gmail.com>
> > wrote:
> > > >>
> > > >>> Add a field called "source" in schema.xml and value would be your
> > table
> > > >>> names.
> > > >>>
> > > >>>
> > > >>>
> > > >>> Rgds
> > > >>> AJ
> > > >>>
> > > >>> On 15-Sep-2013, at 5:38, Baskar Sikkayan <ba...@gmail.com>
> > wrote:
> > > >>>
> > > >>>> Hi,
> > > >>>> I am new to Solr and trying to use Solr java client instead of
> using
> > > the
> > > >>>> Data handler.
> > > >>>> Is there any configuration i need to do for this?
> > > >>>>
> > > >>>> I got the following sample code.
> > > >>>>
> > > >>>> SolrInputDocument doc = new SolrInputDocument();
> > > >>>>
> > > >>>> doc.addField("cat", "book");
> > > >>>> doc.addField("id", "book-" + i);
> > > >>>> doc.addField("name", "The Legend of the Hobbit part " +
> > i);
> > > >>>> server.add(doc);
> > > >>>> server.commit(); // periodically flush
> > > >>>>
> > > >>>> I am confused here. I am going to index 3 different tables for 3
> > > >>> different
> > > >>>> kind of searches. Here i dont have any option to differentiate 3
> > kind
> > > of
> > > >>>> indexes.
> > > >>>> Am i missing anything here. Could anyone please shed some light
> > here?
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Baskar.S
> > > >>>
> > >
> > >
> > >
> > >
> > >
> > >
> >
>
Re: Solr Java Client
Posted by Erick Erickson <er...@gmail.com>.
Listen to Wunder. You really have to switch
gears and think in terms of searching rather
than database tables.
First, there's no requirement that all documents
have the same fields. There's very little penalty
for this.
Second, you have to get over normalized data.
It's unclear how many employees/records you have,
but unless it's a really big set of records (i.e multiple
tens of millions) you don't have to worry much
about saving space.
BTW, your java example works, but you do NOT
want to commit after every document. Configure
your auto soft and hard commits to do this on a
time-based basis.
So I'd recommend you approach it differently.
Don't get stuck on your current data model. Instead
define the searches you want to make and _then_
decide the appropriate fields in the docs.
But to your question, you can certainly create
two cores. That's a good strategy if there is
little data shared between the two types of users.
It's probably conceptually cleaner as well.
Best,
Erick
On Sun, Sep 15, 2013 at 8:18 AM, Baskar Sikkayan <ba...@gmail.com>wrote:
> Hi,
> Thank you very much for your reply.
> Let me clearly explain my requirement.
>
> Its a kind of job site.
>
> I have 2 tables that need to be added in solr.
>
> 1) employee ( id, name, skills, location, .....)
> 2) job_postings ( id, name, job title, description, salary, date_posted,
> ....)
>
>
> Here, there are 2 different kind of searches in my application.
>
> 1) employee search by employer
> 2) job postings search by employee
>
> New row can be added at any time. Also the existing row can be altered at
> any time. For example, employee mobile number can be changed at any time
> and the solr data has to be updated.
>
> Here there are 2 different searches. So i need to maintain two different
> kind of indexes in solr.
>
> The java example says, to add a document, below is the sample code. My
> doubt is, how can i add two different kind of search data here.
> Should i create a 2 different core for this 2 different searches?
>
> SolrInputDocument doc = new SolrInputDocument();
>
> doc.addField("cat", "book");
> doc.addField("id", "book-" + i);
> doc.addField("name", "The Legend of the Hobbit part " + i);
> server.add(doc);
> server.commit();
>
> Thanks,
> Baskar.S
>
>
> On Sun, Sep 15, 2013 at 9:14 AM, Walter Underwood <wunder@wunderwood.org
> >wrote:
>
> > Solr does not have tables, and you can't add an index.
> >
> > Solr's data model is flat, like a single table with lots of columns.
> Think
> > about creating a view where each row has all the information for one
> search
> > result. It includes everything that is searched (indexed=true in the
> > schema) and returned (stored=true in the schema).
> >
> > wunder
> >
> > On Sep 14, 2013, at 7:32 PM, Amit Jha wrote:
> >
> > > Question is not clear to me. Please be more elaborative in your query.
> > Why do u want to store index to DB tables?
> > >
> > > Rgds
> > > AJ
> > >
> > > On 15-Sep-2013, at 7:20, Baskar Sikkayan <ba...@gmail.com> wrote:
> > >
> > >> How to add index to 3 diff tables from java ...
> > >>
> > >>
> > >> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha <sh...@gmail.com>
> wrote:
> > >>
> > >>> Add a field called "source" in schema.xml and value would be your
> table
> > >>> names.
> > >>>
> > >>>
> > >>>
> > >>> Rgds
> > >>> AJ
> > >>>
> > >>> On 15-Sep-2013, at 5:38, Baskar Sikkayan <ba...@gmail.com>
> wrote:
> > >>>
> > >>>> Hi,
> > >>>> I am new to Solr and trying to use Solr java client instead of using
> > the
> > >>>> Data handler.
> > >>>> Is there any configuration i need to do for this?
> > >>>>
> > >>>> I got the following sample code.
> > >>>>
> > >>>> SolrInputDocument doc = new SolrInputDocument();
> > >>>>
> > >>>> doc.addField("cat", "book");
> > >>>> doc.addField("id", "book-" + i);
> > >>>> doc.addField("name", "The Legend of the Hobbit part " +
> i);
> > >>>> server.add(doc);
> > >>>> server.commit(); // periodically flush
> > >>>>
> > >>>> I am confused here. I am going to index 3 different tables for 3
> > >>> different
> > >>>> kind of searches. Here i dont have any option to differentiate 3
> kind
> > of
> > >>>> indexes.
> > >>>> Am i missing anything here. Could anyone please shed some light
> here?
> > >>>>
> > >>>> Thanks,
> > >>>> Baskar.S
> > >>>
> >
> >
> >
> >
> >
> >
>
Re: Solr Java Client
Posted by Baskar Sikkayan <ba...@gmail.com>.
Hi,
Thank you very much for your reply.
Let me clearly explain my requirement.
Its a kind of job site.
I have 2 tables that need to be added in solr.
1) employee ( id, name, skills, location, .....)
2) job_postings ( id, name, job title, description, salary, date_posted,
....)
Here, there are 2 different kind of searches in my application.
1) employee search by employer
2) job postings search by employee
New row can be added at any time. Also the existing row can be altered at
any time. For example, employee mobile number can be changed at any time
and the solr data has to be updated.
Here there are 2 different searches. So i need to maintain two different
kind of indexes in solr.
The java example says, to add a document, below is the sample code. My
doubt is, how can i add two different kind of search data here.
Should i create a 2 different core for this 2 different searches?
SolrInputDocument doc = new SolrInputDocument();
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
server.commit();
Thanks,
Baskar.S
On Sun, Sep 15, 2013 at 9:14 AM, Walter Underwood <wu...@wunderwood.org>wrote:
> Solr does not have tables, and you can't add an index.
>
> Solr's data model is flat, like a single table with lots of columns. Think
> about creating a view where each row has all the information for one search
> result. It includes everything that is searched (indexed=true in the
> schema) and returned (stored=true in the schema).
>
> wunder
>
> On Sep 14, 2013, at 7:32 PM, Amit Jha wrote:
>
> > Question is not clear to me. Please be more elaborative in your query.
> Why do u want to store index to DB tables?
> >
> > Rgds
> > AJ
> >
> > On 15-Sep-2013, at 7:20, Baskar Sikkayan <ba...@gmail.com> wrote:
> >
> >> How to add index to 3 diff tables from java ...
> >>
> >>
> >> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha <sh...@gmail.com> wrote:
> >>
> >>> Add a field called "source" in schema.xml and value would be your table
> >>> names.
> >>>
> >>>
> >>>
> >>> Rgds
> >>> AJ
> >>>
> >>> On 15-Sep-2013, at 5:38, Baskar Sikkayan <ba...@gmail.com> wrote:
> >>>
> >>>> Hi,
> >>>> I am new to Solr and trying to use Solr java client instead of using
> the
> >>>> Data handler.
> >>>> Is there any configuration i need to do for this?
> >>>>
> >>>> I got the following sample code.
> >>>>
> >>>> SolrInputDocument doc = new SolrInputDocument();
> >>>>
> >>>> doc.addField("cat", "book");
> >>>> doc.addField("id", "book-" + i);
> >>>> doc.addField("name", "The Legend of the Hobbit part " + i);
> >>>> server.add(doc);
> >>>> server.commit(); // periodically flush
> >>>>
> >>>> I am confused here. I am going to index 3 different tables for 3
> >>> different
> >>>> kind of searches. Here i dont have any option to differentiate 3 kind
> of
> >>>> indexes.
> >>>> Am i missing anything here. Could anyone please shed some light here?
> >>>>
> >>>> Thanks,
> >>>> Baskar.S
> >>>
>
>
>
>
>
>
Re: Solr Java Client
Posted by Walter Underwood <wu...@wunderwood.org>.
Solr does not have tables, and you can't add an index.
Solr's data model is flat, like a single table with lots of columns. Think about creating a view where each row has all the information for one search result. It includes everything that is searched (indexed=true in the schema) and returned (stored=true in the schema).
wunder
On Sep 14, 2013, at 7:32 PM, Amit Jha wrote:
> Question is not clear to me. Please be more elaborative in your query. Why do u want to store index to DB tables?
>
> Rgds
> AJ
>
> On 15-Sep-2013, at 7:20, Baskar Sikkayan <ba...@gmail.com> wrote:
>
>> How to add index to 3 diff tables from java ...
>>
>>
>> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha <sh...@gmail.com> wrote:
>>
>>> Add a field called "source" in schema.xml and value would be your table
>>> names.
>>>
>>>
>>>
>>> Rgds
>>> AJ
>>>
>>> On 15-Sep-2013, at 5:38, Baskar Sikkayan <ba...@gmail.com> wrote:
>>>
>>>> Hi,
>>>> I am new to Solr and trying to use Solr java client instead of using the
>>>> Data handler.
>>>> Is there any configuration i need to do for this?
>>>>
>>>> I got the following sample code.
>>>>
>>>> SolrInputDocument doc = new SolrInputDocument();
>>>>
>>>> doc.addField("cat", "book");
>>>> doc.addField("id", "book-" + i);
>>>> doc.addField("name", "The Legend of the Hobbit part " + i);
>>>> server.add(doc);
>>>> server.commit(); // periodically flush
>>>>
>>>> I am confused here. I am going to index 3 different tables for 3
>>> different
>>>> kind of searches. Here i dont have any option to differentiate 3 kind of
>>>> indexes.
>>>> Am i missing anything here. Could anyone please shed some light here?
>>>>
>>>> Thanks,
>>>> Baskar.S
>>>
Re: Solr Java Client
Posted by Amit Jha <sh...@gmail.com>.
Question is not clear to me. Please be more elaborative in your query. Why do u want to store index to DB tables?
Rgds
AJ
On 15-Sep-2013, at 7:20, Baskar Sikkayan <ba...@gmail.com> wrote:
> How to add index to 3 diff tables from java ...
>
>
> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha <sh...@gmail.com> wrote:
>
>> Add a field called "source" in schema.xml and value would be your table
>> names.
>>
>>
>>
>> Rgds
>> AJ
>>
>> On 15-Sep-2013, at 5:38, Baskar Sikkayan <ba...@gmail.com> wrote:
>>
>>> Hi,
>>> I am new to Solr and trying to use Solr java client instead of using the
>>> Data handler.
>>> Is there any configuration i need to do for this?
>>>
>>> I got the following sample code.
>>>
>>> SolrInputDocument doc = new SolrInputDocument();
>>>
>>> doc.addField("cat", "book");
>>> doc.addField("id", "book-" + i);
>>> doc.addField("name", "The Legend of the Hobbit part " + i);
>>> server.add(doc);
>>> server.commit(); // periodically flush
>>>
>>> I am confused here. I am going to index 3 different tables for 3
>> different
>>> kind of searches. Here i dont have any option to differentiate 3 kind of
>>> indexes.
>>> Am i missing anything here. Could anyone please shed some light here?
>>>
>>> Thanks,
>>> Baskar.S
>>
Re: Solr Java Client
Posted by Baskar Sikkayan <ba...@gmail.com>.
How to add index to 3 diff tables from java ...
On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha <sh...@gmail.com> wrote:
> Add a field called "source" in schema.xml and value would be your table
> names.
>
>
>
> Rgds
> AJ
>
> On 15-Sep-2013, at 5:38, Baskar Sikkayan <ba...@gmail.com> wrote:
>
> > Hi,
> > I am new to Solr and trying to use Solr java client instead of using the
> > Data handler.
> > Is there any configuration i need to do for this?
> >
> > I got the following sample code.
> >
> > SolrInputDocument doc = new SolrInputDocument();
> >
> > doc.addField("cat", "book");
> > doc.addField("id", "book-" + i);
> > doc.addField("name", "The Legend of the Hobbit part " + i);
> > server.add(doc);
> > server.commit(); // periodically flush
> >
> > I am confused here. I am going to index 3 different tables for 3
> different
> > kind of searches. Here i dont have any option to differentiate 3 kind of
> > indexes.
> > Am i missing anything here. Could anyone please shed some light here?
> >
> > Thanks,
> > Baskar.S
>
Re: Solr Java Client
Posted by Amit Jha <sh...@gmail.com>.
Add a field called "source" in schema.xml and value would be your table names.
Rgds
AJ
On 15-Sep-2013, at 5:38, Baskar Sikkayan <ba...@gmail.com> wrote:
> Hi,
> I am new to Solr and trying to use Solr java client instead of using the
> Data handler.
> Is there any configuration i need to do for this?
>
> I got the following sample code.
>
> SolrInputDocument doc = new SolrInputDocument();
>
> doc.addField("cat", "book");
> doc.addField("id", "book-" + i);
> doc.addField("name", "The Legend of the Hobbit part " + i);
> server.add(doc);
> server.commit(); // periodically flush
>
> I am confused here. I am going to index 3 different tables for 3 different
> kind of searches. Here i dont have any option to differentiate 3 kind of
> indexes.
> Am i missing anything here. Could anyone please shed some light here?
>
> Thanks,
> Baskar.S