You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ryan McKinley <ry...@gmail.com> on 2007/07/27 02:56:39 UTC
solr + carrot2
Has anyone looked into using carrot2 clustering with solr?
I know this is integrated with nutch:
http://lucene.apache.org/nutch/apidocs/org/apache/nutch/clustering/carrot2/Clusterer.html
It looks like carrot has support to read results from a solr index:
http://demo.carrot2.org/head/api/org/carrot2/input/solr/package-summary.html
But I'm hoping for something that returns clustered results from solr.
Carrot also has something to read lucene indexes:
http://demo.carrot2.org/head/api/org/carrot2/input/lucene/package-summary.html
Any pointers or experience before I (may) delve into this?
ryan
RE: solr + carrot2
Posted by Lance Norskog <go...@gmail.com>.
Thanks very much! It came up and worked on our Solr.
I found three quirks:
1) Clicking the URL sends something not quite right to IE. I don't know what
it is.
For some links for file types like .MOV (quicktime) it gives the 'download
this file' popup.
For other links it gives something about JavaScript executing too much.
If it works for you, I'm sure that's fine.
2) The Swing UI does not accept Chinese character input. It displays the
standard square box and does not search correctly.
I know this involves programming in Swing. I doubt it is possible to type in
%xxxx characters in the search box. These would get escaped out in the URL.
3) Setting changes did not get used until I restarted Carrot2.
Thanks again,
Lance
-----Original Message-----
From: stachoo@gmail.com [mailto:stachoo@gmail.com] On Behalf Of Stanislaw
Osinski
Sent: Monday, August 27, 2007 12:26 AM
To: solr-user@lucene.apache.org
Subject: Re: solr + carrot2
Hi Lance and all,
I've just implemented a configuration UI for Solr, similar to the one we
have for Lucene. The new UI is available in the HEAD version of the browser:
http://demo.carrot2.org/head/dist/carrot2-demo-browser-head.zip
or through WebStart:
http://demo.carrot2.org/head/webstart/
Please let us know if the new UI works for you.
Thanks,
Staszek
On 20/08/07, Lance Norskog <go...@gmail.com> wrote:
>
> Exactly! The Lucene version requires direct access to the file. Our
> indexes are on servers which do not have graphics (VNC) configured.
>
> A generic Solr access UI would be great.
>
> Lance
>
> -----Original Message-----
> From: stachoo@gmail.com [mailto:stachoo@gmail.com] On Behalf Of
> Stanislaw Osinski
> Sent: Saturday, August 18, 2007 2:23 AM
> To: solr-user@lucene.apache.org
> Subject: Re: solr + carrot2
>
> Hi Lance,
>
> The Lucene interface is cool, but not many people put their indexes on
> > machines with Swing access.
> >
> > I just did a Solr integration by copying the eTools.ch implementation.
> > This
> > took several edits. As long as we're making requests, please do a
> > general-pupose implementation by cloning the Lucene implementation.
>
>
> I'm not sure if I'm getting you right here... By "implementation" do
> you mean adding to the Swing application an option for pulling data
> from Solr (with a configuration dialog for Solr URL etc.)?
>
> Thanks,
>
> Stanislaw
>
>
Re: solr + carrot2
Posted by Stanislaw Osinski <st...@osinski.name>.
Hi Lance and all,
I've just implemented a configuration UI for Solr, similar to the one we
have for Lucene. The new UI is available in the HEAD version of the browser:
http://demo.carrot2.org/head/dist/carrot2-demo-browser-head.zip
or through WebStart:
http://demo.carrot2.org/head/webstart/
Please let us know if the new UI works for you.
Thanks,
Staszek
On 20/08/07, Lance Norskog <go...@gmail.com> wrote:
>
> Exactly! The Lucene version requires direct access to the file. Our
> indexes
> are on servers which do not have graphics (VNC) configured.
>
> A generic Solr access UI would be great.
>
> Lance
>
> -----Original Message-----
> From: stachoo@gmail.com [mailto:stachoo@gmail.com] On Behalf Of Stanislaw
> Osinski
> Sent: Saturday, August 18, 2007 2:23 AM
> To: solr-user@lucene.apache.org
> Subject: Re: solr + carrot2
>
> Hi Lance,
>
> The Lucene interface is cool, but not many people put their indexes on
> > machines with Swing access.
> >
> > I just did a Solr integration by copying the eTools.ch implementation.
> > This
> > took several edits. As long as we're making requests, please do a
> > general-pupose implementation by cloning the Lucene implementation.
>
>
> I'm not sure if I'm getting you right here... By "implementation" do you
> mean adding to the Swing application an option for pulling data from Solr
> (with a configuration dialog for Solr URL etc.)?
>
> Thanks,
>
> Stanislaw
>
>
RE: solr + carrot2
Posted by Lance Norskog <go...@gmail.com>.
No, this about the Carrot2 clustering tool, specifically the Swing
application.
To make this app use a Solr service you have to code a custom searcher for
your Solr.
I'm requesting a generic UI for Carrot2 that works against any Solr.
-----Original Message-----
From: Mike Klaas [mailto:mike.klaas@gmail.com]
Sent: Monday, August 20, 2007 12:03 PM
To: solr-user@lucene.apache.org
Subject: Re: solr + carrot2
On 20-Aug-07, at 11:24 AM, Lance Norskog wrote:
> Exactly! The Lucene version requires direct access to the file. Our
> indexes are on servers which do not have graphics (VNC) configured.
>
> A generic Solr access UI would be great.
A generic Solr access UI? Is this different from the existing admin ui that
ships with Solr?
-Mike
Re: solr + carrot2
Posted by Mike Klaas <mi...@gmail.com>.
On 20-Aug-07, at 11:24 AM, Lance Norskog wrote:
> Exactly! The Lucene version requires direct access to the file. Our
> indexes
> are on servers which do not have graphics (VNC) configured.
>
> A generic Solr access UI would be great.
A generic Solr access UI? Is this different from the existing admin
ui that ships with Solr?
-Mike
RE: solr + carrot2
Posted by Lance Norskog <go...@gmail.com>.
Exactly! The Lucene version requires direct access to the file. Our indexes
are on servers which do not have graphics (VNC) configured.
A generic Solr access UI would be great.
Lance
-----Original Message-----
From: stachoo@gmail.com [mailto:stachoo@gmail.com] On Behalf Of Stanislaw
Osinski
Sent: Saturday, August 18, 2007 2:23 AM
To: solr-user@lucene.apache.org
Subject: Re: solr + carrot2
Hi Lance,
The Lucene interface is cool, but not many people put their indexes on
> machines with Swing access.
>
> I just did a Solr integration by copying the eTools.ch implementation.
> This
> took several edits. As long as we're making requests, please do a
> general-pupose implementation by cloning the Lucene implementation.
I'm not sure if I'm getting you right here... By "implementation" do you
mean adding to the Swing application an option for pulling data from Solr
(with a configuration dialog for Solr URL etc.)?
Thanks,
Stanislaw
Re: solr + carrot2
Posted by Stanislaw Osinski <st...@osinski.name>.
Hi Lance,
The Lucene interface is cool, but not many people put their indexes on
> machines with Swing access.
>
> I just did a Solr integration by copying the eTools.ch implementation.
> This
> took several edits. As long as we're making requests, please do a
> general-pupose implementation by cloning the Lucene implementation.
I'm not sure if I'm getting you right here... By "implementation" do you
mean adding to the Swing application an option for pulling data from Solr
(with a configuration dialog for Solr URL etc.)?
Thanks,
Stanislaw
RE: solr + carrot2
Posted by Lance Norskog <go...@gmail.com>.
Hello-
The Lucene interface is cool, but not many people put their indexes on
machines with Swing access.
I just did a Solr integration by copying the eTools.ch implementation. This
took several edits. As long as we're making requests, please do a
general-pupose implementation by cloning the Lucene implementation.
Lance
-----Original Message-----
From: stachoo@gmail.com [mailto:stachoo@gmail.com] On Behalf Of Stanislaw
Osinski
Sent: Friday, August 17, 2007 1:35 AM
To: solr-user@lucene.apache.org
Subject: Re: solr + carrot2
Hi All,
I've just filed an issue for us related to this:
http://issues.carrot2.org/browse/CARROT-106
I'll try to find some spare cycles to look into it, hopefully in some not
too distant future. Meanwhile, feel free to post your thoughts and concerns
on this either here or on our JIRA.
Thanks,
Stanislaw
--
Stanislaw Osinski, stanislaw.osinski@carrot-search.com
http://www.carrot-search.com
On 17/08/07, Pieter Berkel <pi...@gmail.com> wrote:
>
> Any updates on this? It certainly would be quite interesting to see
> how well carrot2 clustering can be integrated with solr, I suppose
> it's a fairly similar concept to simple faceting (maybe another
> candidate for SOLR-281 component?).
>
> One concern I have is that the additional processing required at query
> time would make the whole operation significant slower (which is
> something I'd like to avoid). I've been wondering if it might be
> possible to calculate (and store) clustering information at index time
> however since carrot2 seems to use the query term & result set to
> create clustering info this doesn't appear to be a practical approach.
>
> In a similar vein, I'm also looking at methods of term extraction and
> automatic keyword generation from indexed documents. I've been
> experimenting with MoreLikeThis and values returned by the "
> mlt.interestingTerms" parameter, which has potential but needs a bit
> of refinement before it can be truely useful. Has anybody else
> discovered clever or useful methods of term extraction using solr?
>
> Piete
>
>
>
> On 02/08/07, Burkamp, Christian <C....@ceyoniq.com> wrote:
> >
> > Hi,
> >
> > In my opinion the results from carrot2 clustering could be used in
> > the same way that facet results are used.
> > That's the way I'm planning to use them.
> > The user of the search application can narrow the search by
> > selecting
> one
> > of the facets presented in the search result presentation. These
> > facets could come from metadata (classic facets) or from dynamically
> > computed categories which are results from carrot2.
> >
> > From this point of view it would be most convenient to have the
> > integration for carrot2 directly in the StandardRequestHandler. This
> leaves
> > questions open like "how should filters for categories from carrot2
> > be formulated".
> >
> > Is anybody already using carrot2 with solr?
> >
> > -- Christian
> >
> > -----Ursprüngliche Nachricht-----
> > Von: stachoo@gmail.com [mailto: stachoo@gmail.com] Im Auftrag von
> > Stanislaw Osinski
> > Gesendet: Mittwoch, 1. August 2007 14:01
> > An: solr-user@lucene.apache.org
> > Betreff: Re: solr + carrot2
> >
> >
> > >
> > > Has anyone looked into using carrot2 clustering with solr?
> > >
> > > I know this is integrated with nutch:
> > >
> > > http://lucene.apache.org/nutch/apidocs/org/apache/nutch/clustering
> > > /car
> > > rot2/Clusterer.html
> > >
> > > It looks like carrot has support to read results from a solr index:
> > >
> > > http://demo.carrot2.org/head/api/org/carrot2/input/solr/package-su
> > > mmar
> > > y.html
> > >
> > > But I'm hoping for something that returns clustered results from solr.
> > >
> > > Carrot also has something to read lucene indexes:
> > >
> > > http://demo.carrot2.org/head/api/org/carrot2/input/lucene/package-
> > > summ
> > > ary.html
> > >
> > > Any pointers or experience before I (may) delve into this?
> > >
> >
> > First of all, apologies for a delayed response. I'm one of Carrot2
> > developers and indeed we did some Solr integration, but from
> > Carrot2's perspective, which I guess will not be directly useful in
> > this case. If
> you
> > have any ideas for integration, questions or requests for
> changes/patches,
> > feel free to post on Carrot2 mailing list or file an issue for us.
> >
> > Thanks,
> >
> > Staszek
> >
>
Re: solr + carrot2
Posted by Stanislaw Osinski <st...@osinski.name>.
Hi All,
I've just filed an issue for us related to this:
http://issues.carrot2.org/browse/CARROT-106
I'll try to find some spare cycles to look into it, hopefully in some not
too distant future. Meanwhile, feel free to post your thoughts and concerns
on this either here or on our JIRA.
Thanks,
Stanislaw
--
Stanislaw Osinski, stanislaw.osinski@carrot-search.com
http://www.carrot-search.com
On 17/08/07, Pieter Berkel <pi...@gmail.com> wrote:
>
> Any updates on this? It certainly would be quite interesting to see how
> well carrot2 clustering can be integrated with solr, I suppose it's a
> fairly
> similar concept to simple faceting (maybe another candidate for SOLR-281
> component?).
>
> One concern I have is that the additional processing required at query
> time
> would make the whole operation significant slower (which is something I'd
> like to avoid). I've been wondering if it might be possible to calculate
> (and store) clustering information at index time
> however since carrot2 seems to use the query term & result set to create
> clustering info this doesn't appear to be a practical approach.
>
> In a similar vein, I'm also looking at methods of term extraction and
> automatic keyword generation from indexed documents. I've been
> experimenting with MoreLikeThis and values returned by the "
> mlt.interestingTerms" parameter, which has potential but needs a bit of
> refinement before it can be truely useful. Has anybody else discovered
> clever or useful methods of term extraction using solr?
>
> Piete
>
>
>
> On 02/08/07, Burkamp, Christian <C....@ceyoniq.com> wrote:
> >
> > Hi,
> >
> > In my opinion the results from carrot2 clustering could be used in the
> > same way that facet results are used.
> > That's the way I'm planning to use them.
> > The user of the search application can narrow the search by selecting
> one
> > of the facets presented in the search result presentation. These facets
> > could come from metadata (classic facets) or from dynamically computed
> > categories which are results from carrot2.
> >
> > From this point of view it would be most convenient to have the
> > integration for carrot2 directly in the StandardRequestHandler. This
> leaves
> > questions open like "how should filters for categories from carrot2 be
> > formulated".
> >
> > Is anybody already using carrot2 with solr?
> >
> > -- Christian
> >
> > -----Ursprüngliche Nachricht-----
> > Von: stachoo@gmail.com [mailto: stachoo@gmail.com] Im Auftrag von
> > Stanislaw Osinski
> > Gesendet: Mittwoch, 1. August 2007 14:01
> > An: solr-user@lucene.apache.org
> > Betreff: Re: solr + carrot2
> >
> >
> > >
> > > Has anyone looked into using carrot2 clustering with solr?
> > >
> > > I know this is integrated with nutch:
> > >
> > > http://lucene.apache.org/nutch/apidocs/org/apache/nutch/clustering/car
> > > rot2/Clusterer.html
> > >
> > > It looks like carrot has support to read results from a solr index:
> > >
> > > http://demo.carrot2.org/head/api/org/carrot2/input/solr/package-summar
> > > y.html
> > >
> > > But I'm hoping for something that returns clustered results from solr.
> > >
> > > Carrot also has something to read lucene indexes:
> > >
> > > http://demo.carrot2.org/head/api/org/carrot2/input/lucene/package-summ
> > > ary.html
> > >
> > > Any pointers or experience before I (may) delve into this?
> > >
> >
> > First of all, apologies for a delayed response. I'm one of Carrot2
> > developers and indeed we did some Solr integration, but from Carrot2's
> > perspective, which I guess will not be directly useful in this case. If
> you
> > have any ideas for integration, questions or requests for
> changes/patches,
> > feel free to post on Carrot2 mailing list or file an issue for us.
> >
> > Thanks,
> >
> > Staszek
> >
>
Re: solr + carrot2
Posted by Alf Eaton <li...@hubmed.org>.
Pieter Berkel wrote:
> In a similar vein, I'm also looking at methods of term extraction and
> automatic keyword generation from indexed documents. I've been
> experimenting with MoreLikeThis and values returned by the "
> mlt.interestingTerms" parameter, which has potential but needs a bit of
> refinement before it can be truely useful. Has anybody else discovered
> clever or useful methods of term extraction using solr?
I've been using mlt.interestingTerms and it works well. For example,
fetching the mlt.interestingTerms for each of the top 20 search results,
combining the scores and using the top words to suggest additional
search terms.
alf.
Re: solr + carrot2
Posted by Pieter Berkel <pi...@gmail.com>.
Any updates on this? It certainly would be quite interesting to see how
well carrot2 clustering can be integrated with solr, I suppose it's a fairly
similar concept to simple faceting (maybe another candidate for SOLR-281
component?).
One concern I have is that the additional processing required at query time
would make the whole operation significant slower (which is something I'd
like to avoid). I've been wondering if it might be possible to calculate
(and store) clustering information at index time
however since carrot2 seems to use the query term & result set to create
clustering info this doesn't appear to be a practical approach.
In a similar vein, I'm also looking at methods of term extraction and
automatic keyword generation from indexed documents. I've been
experimenting with MoreLikeThis and values returned by the "
mlt.interestingTerms" parameter, which has potential but needs a bit of
refinement before it can be truely useful. Has anybody else discovered
clever or useful methods of term extraction using solr?
Piete
On 02/08/07, Burkamp, Christian <C....@ceyoniq.com> wrote:
>
> Hi,
>
> In my opinion the results from carrot2 clustering could be used in the
> same way that facet results are used.
> That's the way I'm planning to use them.
> The user of the search application can narrow the search by selecting one
> of the facets presented in the search result presentation. These facets
> could come from metadata (classic facets) or from dynamically computed
> categories which are results from carrot2.
>
> From this point of view it would be most convenient to have the
> integration for carrot2 directly in the StandardRequestHandler. This leaves
> questions open like "how should filters for categories from carrot2 be
> formulated".
>
> Is anybody already using carrot2 with solr?
>
> -- Christian
>
> -----Ursprüngliche Nachricht-----
> Von: stachoo@gmail.com [mailto: stachoo@gmail.com] Im Auftrag von
> Stanislaw Osinski
> Gesendet: Mittwoch, 1. August 2007 14:01
> An: solr-user@lucene.apache.org
> Betreff: Re: solr + carrot2
>
>
> >
> > Has anyone looked into using carrot2 clustering with solr?
> >
> > I know this is integrated with nutch:
> >
> > http://lucene.apache.org/nutch/apidocs/org/apache/nutch/clustering/car
> > rot2/Clusterer.html
> >
> > It looks like carrot has support to read results from a solr index:
> >
> > http://demo.carrot2.org/head/api/org/carrot2/input/solr/package-summar
> > y.html
> >
> > But I'm hoping for something that returns clustered results from solr.
> >
> > Carrot also has something to read lucene indexes:
> >
> > http://demo.carrot2.org/head/api/org/carrot2/input/lucene/package-summ
> > ary.html
> >
> > Any pointers or experience before I (may) delve into this?
> >
>
> First of all, apologies for a delayed response. I'm one of Carrot2
> developers and indeed we did some Solr integration, but from Carrot2's
> perspective, which I guess will not be directly useful in this case. If you
> have any ideas for integration, questions or requests for changes/patches,
> feel free to post on Carrot2 mailing list or file an issue for us.
>
> Thanks,
>
> Staszek
>
AW: solr + carrot2
Posted by "Burkamp, Christian" <C....@Ceyoniq.com>.
Hi,
In my opinion the results from carrot2 clustering could be used in the same way that facet results are used.
That's the way I'm planning to use them.
The user of the search application can narrow the search by selecting one of the facets presented in the search result presentation. These facets could come from metadata (classic facets) or from dynamically computed categories which are results from carrot2.
>From this point of view it would be most convenient to have the integration for carrot2 directly in the StandardRequestHandler. This leaves questions open like "how should filters for categories from carrot2 be formulated".
Is anybody already using carrot2 with solr?
-- Christian
-----Ursprüngliche Nachricht-----
Von: stachoo@gmail.com [mailto:stachoo@gmail.com] Im Auftrag von Stanislaw Osinski
Gesendet: Mittwoch, 1. August 2007 14:01
An: solr-user@lucene.apache.org
Betreff: Re: solr + carrot2
>
> Has anyone looked into using carrot2 clustering with solr?
>
> I know this is integrated with nutch:
>
> http://lucene.apache.org/nutch/apidocs/org/apache/nutch/clustering/car
> rot2/Clusterer.html
>
> It looks like carrot has support to read results from a solr index:
>
> http://demo.carrot2.org/head/api/org/carrot2/input/solr/package-summar
> y.html
>
> But I'm hoping for something that returns clustered results from solr.
>
> Carrot also has something to read lucene indexes:
>
> http://demo.carrot2.org/head/api/org/carrot2/input/lucene/package-summ
> ary.html
>
> Any pointers or experience before I (may) delve into this?
>
First of all, apologies for a delayed response. I'm one of Carrot2 developers and indeed we did some Solr integration, but from Carrot2's perspective, which I guess will not be directly useful in this case. If you have any ideas for integration, questions or requests for changes/patches, feel free to post on Carrot2 mailing list or file an issue for us.
Thanks,
Staszek
Re: solr + carrot2
Posted by Stanislaw Osinski <st...@man.poznan.pl>.
>
> Has anyone looked into using carrot2 clustering with solr?
>
> I know this is integrated with nutch:
>
> http://lucene.apache.org/nutch/apidocs/org/apache/nutch/clustering/carrot2/Clusterer.html
>
> It looks like carrot has support to read results from a solr index:
>
> http://demo.carrot2.org/head/api/org/carrot2/input/solr/package-summary.html
>
> But I'm hoping for something that returns clustered results from solr.
>
> Carrot also has something to read lucene indexes:
>
> http://demo.carrot2.org/head/api/org/carrot2/input/lucene/package-summary.html
>
> Any pointers or experience before I (may) delve into this?
>
First of all, apologies for a delayed response. I'm one of Carrot2
developers and indeed we did some Solr integration, but from Carrot2's
perspective, which I guess will not be directly useful in this case. If you
have any ideas for integration, questions or requests for changes/patches,
feel free to post on Carrot2 mailing list or file an issue for us.
Thanks,
Staszek