You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Raghunandan Rao <Ra...@newScale.com> on 2008/10/30 13:04:36 UTC
Using Solrj
Hi,
I am trying to use Solrj for my web application. I am indexing a table
using the @Field annotation tag. Now I need to index or query multiple
tables. Like, get all the employees who are managers in Finance
department (interacting with 3 entities). How do I do that?
Does anyone have any idea?
Thanks
Re: Using Solrj
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
not really. you can explain your usecase and it will be more clear
On Thu, Oct 30, 2008 at 6:20 PM, Raghunandan Rao
<Ra...@newscale.com> wrote:
> Thanks Noble.
>
> So you mean to say that I need to create a view according to my query and then index on the view and fetch?
>
> -----Original Message-----
> From: Noble Paul നോബിള് नोब्ळ् [mailto:noble.paul@gmail.com]
> Sent: Thursday, October 30, 2008 6:16 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Using Solrj
>
> hi ,
> There are two sides to this .
> 1. indexing (getting data into Solr) SolrJ or DataImportHandler can be
> used for this
> 2.querying . getting data out of solr. Here you do not have the choice
> of joining multiple tables. There only one index for Solr
>
>
>
> On Thu, Oct 30, 2008 at 5:34 PM, Raghunandan Rao
> <Ra...@newscale.com> wrote:
>> Hi,
>>
>> I am trying to use Solrj for my web application. I am indexing a table
>> using the @Field annotation tag. Now I need to index or query multiple
>> tables. Like, get all the employees who are managers in Finance
>> department (interacting with 3 entities). How do I do that?
>>
>>
>>
>> Does anyone have any idea?
>>
>>
>>
>> Thanks
>>
>>
>
>
>
> --
> --Noble Paul
>
--
--Noble Paul
Re: Using Solrj
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Fri, Oct 31, 2008 at 5:21 PM, Raghunandan Rao <
Raghunandan.Rao@newscale.com> wrote:
> Thank you.
> I was talking about DataImportHandler API.
>
>
Most likely, you will not need to use the API. DataImportHandler will let
you index your database without writing code -- you just need an XML
configuration file. Only if you need to do some custom tasks, will you need
to touch the API.
The API is marked as experimental just because it is new and we're not sure
of the use-cases that might come up in the near future and therefore we'd
like the freedom of modifying it. As always, we strive to maintain
backwards-compatibility as much as possible. I know that DataImportHandler
is being used in production in a few high traffic websites.
--
Regards,
Shalin Shekhar Mangar.
RE: Using Solrj
Posted by Raghunandan Rao <Ra...@newScale.com>.
Thank you.
I was talking about DataImportHandler API.
-----Original Message-----
From: Shalin Shekhar Mangar [mailto:shalinmangar@gmail.com]
Sent: Friday, October 31, 2008 5:19 PM
To: solr-user@lucene.apache.org
Subject: Re: Using Solrj
On Fri, Oct 31, 2008 at 4:32 PM, Raghunandan Rao <
Raghunandan.Rao@newscale.com> wrote:
> I am doing that but the API is in experimental stage. Not sure to use
it or
> not. BTW can you also let me know how clustering works on Windows OS
cos I
> saw clustering scripts for Unix OS bundled out with Solr release.
>
>
Which API is in experimental stage?
By clustering, I think you mean replication. Until the last release
(1.3),
we had support only for *nix platforms for replication. In the next
release
we have a Java based replication coming which works for Windows also. If
you
want, you can try it with one of the nightly (un-released) builds.
http://wiki.apache.org/solr/SolrReplication
--
Regards,
Shalin Shekhar Mangar.
Re: Using Solrj
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Fri, Oct 31, 2008 at 4:32 PM, Raghunandan Rao <
Raghunandan.Rao@newscale.com> wrote:
> I am doing that but the API is in experimental stage. Not sure to use it or
> not. BTW can you also let me know how clustering works on Windows OS cos I
> saw clustering scripts for Unix OS bundled out with Solr release.
>
>
Which API is in experimental stage?
By clustering, I think you mean replication. Until the last release (1.3),
we had support only for *nix platforms for replication. In the next release
we have a Java based replication coming which works for Windows also. If you
want, you can try it with one of the nightly (un-released) builds.
http://wiki.apache.org/solr/SolrReplication
--
Regards,
Shalin Shekhar Mangar.
RE: Using Solrj
Posted by Raghunandan Rao <Ra...@newScale.com>.
I am doing that but the API is in experimental stage. Not sure to use it or not. BTW can you also let me know how clustering works on Windows OS cos I saw clustering scripts for Unix OS bundled out with Solr release.
-----Original Message-----
From: Noble Paul നോബിള് नोब्ळ् [mailto:noble.paul@gmail.com]
Sent: Friday, October 31, 2008 11:37 AM
To: solr-user@lucene.apache.org
Subject: Re: Using Solrj
First of all you need to index your data in Solr. I suggest
DataImportHandler because it can help you join multiple tables and
index data
On Fri, Oct 31, 2008 at 10:20 AM, Raghunandan Rao
<Ra...@newscale.com> wrote:
> Thank you so much.
>
> Here goes my Use case:
>
> I need to search the database for collection of input parameters which touches 'n' number of tables. The data is very huge. The search query itself is so dynamic. I use lot of views for same search. How do I make use of Solr in this case?
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Thursday, October 30, 2008 7:01 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Using Solrj
>
> Generally, you need to get your head out of the database world and into
> the search world to be successful with Lucene. For instance, one
> of the cardinal tenets of database design is to normalize your
> data. It goes against every instinct to *denormalize* your data when
> creating an Lucene index explicitly so you do NOT have to think
> in terms of joins or sub-queries. Whenever I start thinking this
> way, I try to back up and think again.
>
> Both your posts indicate to me that you're thinking in database
> terms. There are no views in Lucene, for instance. You refer
> to tables. There are no tables in Lucene, there are only documents
> with various numbers of fields. You could conceivable make your index
> look like a database by creatively naming your document fields. But
> that doesn't play to the strengths of Lucene *or* the database.
>
> In fact, there is NO requirement that documents have the *same* fields.
> Which is really difficult to get into when thinking like a DBA.
>
> Lucene is designed to search text. Fast and well. It is NOT intended to
> efficiently manipulate relationships *between* documents. There
> are various hybrid solutions that people have used. That is, put the
> data you really need to do text searching on in a Lucene index,
> along with enough data to be able to get the *rest* of what you need
> from your database. But it all depends upon the problem you're trying to
> solve.
>
> But as Noble says, all this is too general to be really useful, you need
> to provide quite more detail about the problem you're trying to
> solve to get useful recommendations.
>
> Best
> Erick
>
> On Thu, Oct 30, 2008 at 8:50 AM, Raghunandan Rao <
> Raghunandan.Rao@newscale.com> wrote:
>
>> Thanks Noble.
>>
>> So you mean to say that I need to create a view according to my query and
>> then index on the view and fetch?
>>
>> -----Original Message-----
>> From: Noble Paul നോബിള് नोब्ळ् [mailto:noble.paul@gmail.com]
>> Sent: Thursday, October 30, 2008 6:16 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Using Solrj
>>
>> hi ,
>> There are two sides to this .
>> 1. indexing (getting data into Solr) SolrJ or DataImportHandler can be
>> used for this
>> 2.querying . getting data out of solr. Here you do not have the choice
>> of joining multiple tables. There only one index for Solr
>>
>>
>>
>> On Thu, Oct 30, 2008 at 5:34 PM, Raghunandan Rao
>> <Ra...@newscale.com> wrote:
>> > Hi,
>> >
>> > I am trying to use Solrj for my web application. I am indexing a table
>> > using the @Field annotation tag. Now I need to index or query multiple
>> > tables. Like, get all the employees who are managers in Finance
>> > department (interacting with 3 entities). How do I do that?
>> >
>> >
>> >
>> > Does anyone have any idea?
>> >
>> >
>> >
>> > Thanks
>> >
>> >
>>
>>
>>
>> --
>> --Noble Paul
>>
>
--
--Noble Paul
Re: Using Solrj
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
First of all you need to index your data in Solr. I suggest
DataImportHandler because it can help you join multiple tables and
index data
On Fri, Oct 31, 2008 at 10:20 AM, Raghunandan Rao
<Ra...@newscale.com> wrote:
> Thank you so much.
>
> Here goes my Use case:
>
> I need to search the database for collection of input parameters which touches 'n' number of tables. The data is very huge. The search query itself is so dynamic. I use lot of views for same search. How do I make use of Solr in this case?
>
> -----Original Message-----
> From: Erick Erickson [mailto:erickerickson@gmail.com]
> Sent: Thursday, October 30, 2008 7:01 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Using Solrj
>
> Generally, you need to get your head out of the database world and into
> the search world to be successful with Lucene. For instance, one
> of the cardinal tenets of database design is to normalize your
> data. It goes against every instinct to *denormalize* your data when
> creating an Lucene index explicitly so you do NOT have to think
> in terms of joins or sub-queries. Whenever I start thinking this
> way, I try to back up and think again.
>
> Both your posts indicate to me that you're thinking in database
> terms. There are no views in Lucene, for instance. You refer
> to tables. There are no tables in Lucene, there are only documents
> with various numbers of fields. You could conceivable make your index
> look like a database by creatively naming your document fields. But
> that doesn't play to the strengths of Lucene *or* the database.
>
> In fact, there is NO requirement that documents have the *same* fields.
> Which is really difficult to get into when thinking like a DBA.
>
> Lucene is designed to search text. Fast and well. It is NOT intended to
> efficiently manipulate relationships *between* documents. There
> are various hybrid solutions that people have used. That is, put the
> data you really need to do text searching on in a Lucene index,
> along with enough data to be able to get the *rest* of what you need
> from your database. But it all depends upon the problem you're trying to
> solve.
>
> But as Noble says, all this is too general to be really useful, you need
> to provide quite more detail about the problem you're trying to
> solve to get useful recommendations.
>
> Best
> Erick
>
> On Thu, Oct 30, 2008 at 8:50 AM, Raghunandan Rao <
> Raghunandan.Rao@newscale.com> wrote:
>
>> Thanks Noble.
>>
>> So you mean to say that I need to create a view according to my query and
>> then index on the view and fetch?
>>
>> -----Original Message-----
>> From: Noble Paul നോബിള് नोब्ळ् [mailto:noble.paul@gmail.com]
>> Sent: Thursday, October 30, 2008 6:16 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Using Solrj
>>
>> hi ,
>> There are two sides to this .
>> 1. indexing (getting data into Solr) SolrJ or DataImportHandler can be
>> used for this
>> 2.querying . getting data out of solr. Here you do not have the choice
>> of joining multiple tables. There only one index for Solr
>>
>>
>>
>> On Thu, Oct 30, 2008 at 5:34 PM, Raghunandan Rao
>> <Ra...@newscale.com> wrote:
>> > Hi,
>> >
>> > I am trying to use Solrj for my web application. I am indexing a table
>> > using the @Field annotation tag. Now I need to index or query multiple
>> > tables. Like, get all the employees who are managers in Finance
>> > department (interacting with 3 entities). How do I do that?
>> >
>> >
>> >
>> > Does anyone have any idea?
>> >
>> >
>> >
>> > Thanks
>> >
>> >
>>
>>
>>
>> --
>> --Noble Paul
>>
>
--
--Noble Paul
RE: Using Solrj
Posted by Raghunandan Rao <Ra...@newScale.com>.
Thank you so much.
Here goes my Use case:
I need to search the database for collection of input parameters which touches 'n' number of tables. The data is very huge. The search query itself is so dynamic. I use lot of views for same search. How do I make use of Solr in this case?
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Thursday, October 30, 2008 7:01 PM
To: solr-user@lucene.apache.org
Subject: Re: Using Solrj
Generally, you need to get your head out of the database world and into
the search world to be successful with Lucene. For instance, one
of the cardinal tenets of database design is to normalize your
data. It goes against every instinct to *denormalize* your data when
creating an Lucene index explicitly so you do NOT have to think
in terms of joins or sub-queries. Whenever I start thinking this
way, I try to back up and think again.
Both your posts indicate to me that you're thinking in database
terms. There are no views in Lucene, for instance. You refer
to tables. There are no tables in Lucene, there are only documents
with various numbers of fields. You could conceivable make your index
look like a database by creatively naming your document fields. But
that doesn't play to the strengths of Lucene *or* the database.
In fact, there is NO requirement that documents have the *same* fields.
Which is really difficult to get into when thinking like a DBA.
Lucene is designed to search text. Fast and well. It is NOT intended to
efficiently manipulate relationships *between* documents. There
are various hybrid solutions that people have used. That is, put the
data you really need to do text searching on in a Lucene index,
along with enough data to be able to get the *rest* of what you need
from your database. But it all depends upon the problem you're trying to
solve.
But as Noble says, all this is too general to be really useful, you need
to provide quite more detail about the problem you're trying to
solve to get useful recommendations.
Best
Erick
On Thu, Oct 30, 2008 at 8:50 AM, Raghunandan Rao <
Raghunandan.Rao@newscale.com> wrote:
> Thanks Noble.
>
> So you mean to say that I need to create a view according to my query and
> then index on the view and fetch?
>
> -----Original Message-----
> From: Noble Paul നോബിള് नोब्ळ् [mailto:noble.paul@gmail.com]
> Sent: Thursday, October 30, 2008 6:16 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Using Solrj
>
> hi ,
> There are two sides to this .
> 1. indexing (getting data into Solr) SolrJ or DataImportHandler can be
> used for this
> 2.querying . getting data out of solr. Here you do not have the choice
> of joining multiple tables. There only one index for Solr
>
>
>
> On Thu, Oct 30, 2008 at 5:34 PM, Raghunandan Rao
> <Ra...@newscale.com> wrote:
> > Hi,
> >
> > I am trying to use Solrj for my web application. I am indexing a table
> > using the @Field annotation tag. Now I need to index or query multiple
> > tables. Like, get all the employees who are managers in Finance
> > department (interacting with 3 entities). How do I do that?
> >
> >
> >
> > Does anyone have any idea?
> >
> >
> >
> > Thanks
> >
> >
>
>
>
> --
> --Noble Paul
>
Re: Using Solrj
Posted by Erick Erickson <er...@gmail.com>.
Generally, you need to get your head out of the database world and into
the search world to be successful with Lucene. For instance, one
of the cardinal tenets of database design is to normalize your
data. It goes against every instinct to *denormalize* your data when
creating an Lucene index explicitly so you do NOT have to think
in terms of joins or sub-queries. Whenever I start thinking this
way, I try to back up and think again.
Both your posts indicate to me that you're thinking in database
terms. There are no views in Lucene, for instance. You refer
to tables. There are no tables in Lucene, there are only documents
with various numbers of fields. You could conceivable make your index
look like a database by creatively naming your document fields. But
that doesn't play to the strengths of Lucene *or* the database.
In fact, there is NO requirement that documents have the *same* fields.
Which is really difficult to get into when thinking like a DBA.
Lucene is designed to search text. Fast and well. It is NOT intended to
efficiently manipulate relationships *between* documents. There
are various hybrid solutions that people have used. That is, put the
data you really need to do text searching on in a Lucene index,
along with enough data to be able to get the *rest* of what you need
from your database. But it all depends upon the problem you're trying to
solve.
But as Noble says, all this is too general to be really useful, you need
to provide quite more detail about the problem you're trying to
solve to get useful recommendations.
Best
Erick
On Thu, Oct 30, 2008 at 8:50 AM, Raghunandan Rao <
Raghunandan.Rao@newscale.com> wrote:
> Thanks Noble.
>
> So you mean to say that I need to create a view according to my query and
> then index on the view and fetch?
>
> -----Original Message-----
> From: Noble Paul നോബിള് नोब्ळ् [mailto:noble.paul@gmail.com]
> Sent: Thursday, October 30, 2008 6:16 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Using Solrj
>
> hi ,
> There are two sides to this .
> 1. indexing (getting data into Solr) SolrJ or DataImportHandler can be
> used for this
> 2.querying . getting data out of solr. Here you do not have the choice
> of joining multiple tables. There only one index for Solr
>
>
>
> On Thu, Oct 30, 2008 at 5:34 PM, Raghunandan Rao
> <Ra...@newscale.com> wrote:
> > Hi,
> >
> > I am trying to use Solrj for my web application. I am indexing a table
> > using the @Field annotation tag. Now I need to index or query multiple
> > tables. Like, get all the employees who are managers in Finance
> > department (interacting with 3 entities). How do I do that?
> >
> >
> >
> > Does anyone have any idea?
> >
> >
> >
> > Thanks
> >
> >
>
>
>
> --
> --Noble Paul
>
RE: Using Solrj
Posted by Raghunandan Rao <Ra...@newScale.com>.
Thanks Noble.
So you mean to say that I need to create a view according to my query and then index on the view and fetch?
-----Original Message-----
From: Noble Paul നോബിള് नोब्ळ् [mailto:noble.paul@gmail.com]
Sent: Thursday, October 30, 2008 6:16 PM
To: solr-user@lucene.apache.org
Subject: Re: Using Solrj
hi ,
There are two sides to this .
1. indexing (getting data into Solr) SolrJ or DataImportHandler can be
used for this
2.querying . getting data out of solr. Here you do not have the choice
of joining multiple tables. There only one index for Solr
On Thu, Oct 30, 2008 at 5:34 PM, Raghunandan Rao
<Ra...@newscale.com> wrote:
> Hi,
>
> I am trying to use Solrj for my web application. I am indexing a table
> using the @Field annotation tag. Now I need to index or query multiple
> tables. Like, get all the employees who are managers in Finance
> department (interacting with 3 entities). How do I do that?
>
>
>
> Does anyone have any idea?
>
>
>
> Thanks
>
>
--
--Noble Paul
Re: Using Solrj
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
hi ,
There are two sides to this .
1. indexing (getting data into Solr) SolrJ or DataImportHandler can be
used for this
2.querying . getting data out of solr. Here you do not have the choice
of joining multiple tables. There only one index for Solr
On Thu, Oct 30, 2008 at 5:34 PM, Raghunandan Rao
<Ra...@newscale.com> wrote:
> Hi,
>
> I am trying to use Solrj for my web application. I am indexing a table
> using the @Field annotation tag. Now I need to index or query multiple
> tables. Like, get all the employees who are managers in Finance
> department (interacting with 3 entities). How do I do that?
>
>
>
> Does anyone have any idea?
>
>
>
> Thanks
>
>
--
--Noble Paul