You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Cass Costello <ca...@gmail.com> on 2007/03/20 20:58:25 UTC
Thank you...
...to everyone who helps make Lucene and Solr such fantastic tools.
I'm the Platform Architect for a leading online event ticket
after-marketplace (think eBay for tickets), and we've just completed a 12
month project to rewrite the Browse and Search components of our
customer-facing site. Both rely heavily on Solr and a smattering of custom
Lucene API calls throughout the code base. As of this morning (2 weeks
after our final-stage push to production), we've reduced our hardware needs
over 50% and have improved page response times by 30-60% as compared to the
old architecture (a mix of very old CF code for the browse pieces, and a
couple GSAs for the seach pages). More importantly, we've established new
patterns and flex points for future growth, and I have utmost confidence
that we'll be leveraging the Lucene/Solr combo for years to come as we grow.
The devs are proud, the execs are happy, our customers have noticed the
speed improvements, and we all learned many positive lessons along the way.
These forums and various online resources (the Solr wiki first and foremost)
have been tremendously helpful. And so, thank you, all, for pointing the
way to a simpler, faster, better solution for our needs.
Cass Costello
--
A complex system that works is invariably found to have evolved from a
simple system that works.
- John Gaule
Re: Thank you...
Posted by Chris Hostetter <ho...@fucit.org>.
: Care to write up a Use Case when you have a few spare cycles? http://
: wiki.apache.org/lucene-java/UseCases
Oh, Oh OH! ... competing requests for wiki submisions: Can you add some of
the info about the performance numbers you are seeing to...
http://wiki.apache.org/solr/SolrPerformanceData
...I've taken the liberty of adding stubhub to the "Powered By" Solr
page...
http://wiki.apache.org/solr/PublicServers
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Thank you...
Posted by Grant Ingersoll <gs...@apache.org>.
Hi Cass,
Care to write up a Use Case when you have a few spare cycles? http://
wiki.apache.org/lucene-java/UseCases
-Grant
On Mar 20, 2007, at 4:49 PM, Cass Costello wrote:
> Heh - it used to be in my sig ... my bad.
>
> Thanks, all. :)
>
> http://www.stubhub.com
>
>
> On 3/20/07, bruce <be...@earthlink.net> wrote:
>>
>> hey cass...
>>
>> anyway you could let us know the site/app that we're powering!!!
>>
>> always good to see what's going on in the world!
>>
>> thanks
>>
>>
>> -----Original Message-----
>> From: Cass Costello [mailto:casspc@gmail.com]
>> Sent: Tuesday, March 20, 2007 12:58 PM
>> To: solr-user@lucene.apache.org; java-user@lucene.apache.org
>> Subject: Thank you...
>>
>>
>> ...to everyone who helps make Lucene and Solr such fantastic tools.
>>
>> I'm the Platform Architect for a leading online event ticket
>> after-marketplace (think eBay for tickets), and we've just
>> completed a 12
>> month project to rewrite the Browse and Search components of our
>> customer-facing site. Both rely heavily on Solr and a smattering of
>> custom
>> Lucene API calls throughout the code base. As of this morning (2
>> weeks
>> after our final-stage push to production), we've reduced our hardware
>> needs
>> over 50% and have improved page response times by 30-60% as
>> compared to
>> the
>> old architecture (a mix of very old CF code for the browse pieces,
>> and a
>> couple GSAs for the seach pages). More importantly, we've
>> established new
>> patterns and flex points for future growth, and I have utmost
>> confidence
>> that we'll be leveraging the Lucene/Solr combo for years to come
>> as we
>> grow.
>> The devs are proud, the execs are happy, our customers have
>> noticed the
>> speed improvements, and we all learned many positive lessons along
>> the
>> way.
>>
>> These forums and various online resources (the Solr wiki first and
>> foremost)
>> have been tremendously helpful. And so, thank you, all, for
>> pointing the
>> way to a simpler, faster, better solution for our needs.
>>
>> Cass Costello
>>
>> --
>> A complex system that works is invariably found to have evolved
>> from a
>> simple system that works.
>> - John Gaule
>>
>>
>
>
> --
> A complex system that works is invariably found to have evolved from a
> simple system that works.
> - John Gaule
--------------------------
Grant Ingersoll
Center for Natural Language Processing
http://www.cnlp.org
Read the Lucene Java FAQ at http://wiki.apache.org/jakarta-lucene/
LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Thank you...
Posted by Cass Costello <ca...@gmail.com>.
You all rock. I'm clearing the semi-official legal hurdle with my CTO and
our head counsel to full (or something close to full) disclosure of some of
the architectural details, so stay tuned for as much as I'm allowed to share
(and btw, for any of you that live/work/vacation in the SF Bay area, I'd be
happy to discuss things over a beer :)
In the meantime, yes, much of the faceting you see on the Browse and Search
pages is direct Solr faceting. We made *very* liberal use of
faceting-specific fields in our event and ontology documents.
Similarly, we rely on the standard query handler for almost all the Solr
calls. Our business logic is modelled in a CoR pattern, and we did write
one command to encapsulate SolrCore/Request/Response use for our highest
throughput needs - the win there was direct building into our predominant
xml object model (we use XOM) via a QueryResponseXOMBuilder.build(
solrRequest, solrResponse ) call.
I definitely need clearance before sharing hard perf numbers (anyone working
for TicketMaster on these lists? - funny, I used to use "eBay" in lines like
that - see below) but I will share the fact that our response times and q/s
numbers were very good out-of-the-box. We've an Ehcache-based distributed
caching framework that we threw in front of the Solr servers, and our final
total throughput for the Solr queries utilized to support Browse runs into
the 100's of requests per sec, per Solr server. Our Search (true free-text
searches) numbers are lower, but not by that much.
This might be the coolest (for me, anyway) result of the entire endeavor...
StubHub was recently acquired by eBay. I've been working closely with a
couple of their architects over the past few weeks, and we've done a bit of
theoretical, comparative analysis between our search/browse platform and
theirs (which is completely home grown), and things stack up favorably for
the Solr/Lucene approach, esp. when considering the *vast* difference in
hardware allocation per-request. Again, if I'm allowed, more details to
follow.
>>...I've taken the liberty of adding stubhub to the "Powered By" Solr
page...
Awesome. I can't say it enough... thank you all.
On 3/21/07, Yonik Seeley <yo...@apache.org> wrote:
>
> On 3/20/07, Cass Costello <ca...@gmail.com> wrote:
> > Heh - it used to be in my sig ... my bad.
> >
> > Thanks, all. :)
> >
> > http://www.stubhub.com
>
> I tried it out... looks great!
> Is the faceting stuff done with Solr too?
> If so is it the Solr built-in faceting, or a custom query handler? If
> the latter, can you think of ways to improve the built in faceting
> that would have made things easier?
>
> -Yonik
>
--
A complex system that works is invariably found to have evolved from a
simple system that works.
- John Gaule
--
A complex system that works is invariably found to have evolved from a
simple system that works.
- John Gaule
Re: Thank you...
Posted by Cass Costello <ca...@gmail.com>.
You all rock. I'm clearing the semi-official legal hurdle with my CTO and
our head counsel to full (or something close to full) disclosure of some of
the architectural details, so stay tuned for as much as I'm allowed to share
(and btw, for any of you that live/work/vacation in the SF Bay area, I'd be
happy to discuss things over a beer :)
In the meantime, yes, much of the faceting you see on the Browse and Search
pages is direct Solr faceting. We made *very* liberal use of
faceting-specific fields in our event and ontology documents.
Similarly, we rely on the standard query handler for almost all the Solr
calls. Our business logic is modelled in a CoR pattern, and we did write
one command to encapsulate SolrCore/Request/Response use for our highest
throughput needs - the win there was direct building into our predominant
xml object model (we use XOM) via a QueryResponseXOMBuilder.build(
solrRequest, solrResponse ) call.
I definitely need clearance before sharing hard perf numbers (anyone working
for TicketMaster on these lists? - funny, I used to use "eBay" in lines like
that - see below) but I will share the fact that our response times and q/s
numbers were very good out-of-the-box. We've an Ehcache-based distributed
caching framework that we threw in front of the Solr servers, and our final
total throughput for the Solr queries utilized to support Browse runs into
the 100's of requests per sec, per Solr server. Our Search (true free-text
searches) numbers are lower, but not by that much.
This might be the coolest (for me, anyway) result of the entire endeavor...
StubHub was recently acquired by eBay. I've been working closely with a
couple of their architects over the past few weeks, and we've done a bit of
theoretical, comparative analysis between our search/browse platform and
theirs (which is completely home grown), and things stack up favorably for
the Solr/Lucene approach, esp. when considering the *vast* difference in
hardware allocation per-request. Again, if I'm allowed, more details to
follow.
>>...I've taken the liberty of adding stubhub to the "Powered By" Solr
page...
Awesome. I can't say it enough... thank you all.
On 3/21/07, Yonik Seeley <yo...@apache.org> wrote:
>
> On 3/20/07, Cass Costello <ca...@gmail.com> wrote:
> > Heh - it used to be in my sig ... my bad.
> >
> > Thanks, all. :)
> >
> > http://www.stubhub.com
>
> I tried it out... looks great!
> Is the faceting stuff done with Solr too?
> If so is it the Solr built-in faceting, or a custom query handler? If
> the latter, can you think of ways to improve the built in faceting
> that would have made things easier?
>
> -Yonik
>
--
A complex system that works is invariably found to have evolved from a
simple system that works.
- John Gaule
Re: Thank you...
Posted by Yonik Seeley <yo...@apache.org>.
On 3/20/07, Cass Costello <ca...@gmail.com> wrote:
> Heh - it used to be in my sig ... my bad.
>
> Thanks, all. :)
>
> http://www.stubhub.com
I tried it out... looks great!
Is the faceting stuff done with Solr too?
If so is it the Solr built-in faceting, or a custom query handler? If
the latter, can you think of ways to improve the built in faceting
that would have made things easier?
-Yonik
Re: Thank you...
Posted by Cass Costello <ca...@gmail.com>.
Heh - it used to be in my sig ... my bad.
Thanks, all. :)
http://www.stubhub.com
On 3/20/07, bruce <be...@earthlink.net> wrote:
>
> hey cass...
>
> anyway you could let us know the site/app that we're powering!!!
>
> always good to see what's going on in the world!
>
> thanks
>
>
> -----Original Message-----
> From: Cass Costello [mailto:casspc@gmail.com]
> Sent: Tuesday, March 20, 2007 12:58 PM
> To: solr-user@lucene.apache.org; java-user@lucene.apache.org
> Subject: Thank you...
>
>
> ...to everyone who helps make Lucene and Solr such fantastic tools.
>
> I'm the Platform Architect for a leading online event ticket
> after-marketplace (think eBay for tickets), and we've just completed a 12
> month project to rewrite the Browse and Search components of our
> customer-facing site. Both rely heavily on Solr and a smattering of
> custom
> Lucene API calls throughout the code base. As of this morning (2 weeks
> after our final-stage push to production), we've reduced our hardware
> needs
> over 50% and have improved page response times by 30-60% as compared to
> the
> old architecture (a mix of very old CF code for the browse pieces, and a
> couple GSAs for the seach pages). More importantly, we've established new
> patterns and flex points for future growth, and I have utmost confidence
> that we'll be leveraging the Lucene/Solr combo for years to come as we
> grow.
> The devs are proud, the execs are happy, our customers have noticed the
> speed improvements, and we all learned many positive lessons along the
> way.
>
> These forums and various online resources (the Solr wiki first and
> foremost)
> have been tremendously helpful. And so, thank you, all, for pointing the
> way to a simpler, faster, better solution for our needs.
>
> Cass Costello
>
> --
> A complex system that works is invariably found to have evolved from a
> simple system that works.
> - John Gaule
>
>
--
A complex system that works is invariably found to have evolved from a
simple system that works.
- John Gaule
Re: Thank you...
Posted by Cass Costello <ca...@gmail.com>.
Heh - it used to be in my sig ... my bad.
Thanks, all. :)
http://www.stubhub.com
On 3/20/07, bruce <be...@earthlink.net> wrote:
>
> hey cass...
>
> anyway you could let us know the site/app that we're powering!!!
>
> always good to see what's going on in the world!
>
> thanks
>
>
> -----Original Message-----
> From: Cass Costello [mailto:casspc@gmail.com]
> Sent: Tuesday, March 20, 2007 12:58 PM
> To: solr-user@lucene.apache.org; java-user@lucene.apache.org
> Subject: Thank you...
>
>
> ...to everyone who helps make Lucene and Solr such fantastic tools.
>
> I'm the Platform Architect for a leading online event ticket
> after-marketplace (think eBay for tickets), and we've just completed a 12
> month project to rewrite the Browse and Search components of our
> customer-facing site. Both rely heavily on Solr and a smattering of
> custom
> Lucene API calls throughout the code base. As of this morning (2 weeks
> after our final-stage push to production), we've reduced our hardware
> needs
> over 50% and have improved page response times by 30-60% as compared to
> the
> old architecture (a mix of very old CF code for the browse pieces, and a
> couple GSAs for the seach pages). More importantly, we've established new
> patterns and flex points for future growth, and I have utmost confidence
> that we'll be leveraging the Lucene/Solr combo for years to come as we
> grow.
> The devs are proud, the execs are happy, our customers have noticed the
> speed improvements, and we all learned many positive lessons along the
> way.
>
> These forums and various online resources (the Solr wiki first and
> foremost)
> have been tremendously helpful. And so, thank you, all, for pointing the
> way to a simpler, faster, better solution for our needs.
>
> Cass Costello
>
> --
> A complex system that works is invariably found to have evolved from a
> simple system that works.
> - John Gaule
>
>
--
A complex system that works is invariably found to have evolved from a
simple system that works.
- John Gaule
RE: Thank you...
Posted by bruce <be...@earthlink.net>.
hey cass...
anyway you could let us know the site/app that we're powering!!!
always good to see what's going on in the world!
thanks
-----Original Message-----
From: Cass Costello [mailto:casspc@gmail.com]
Sent: Tuesday, March 20, 2007 12:58 PM
To: solr-user@lucene.apache.org; java-user@lucene.apache.org
Subject: Thank you...
...to everyone who helps make Lucene and Solr such fantastic tools.
I'm the Platform Architect for a leading online event ticket
after-marketplace (think eBay for tickets), and we've just completed a 12
month project to rewrite the Browse and Search components of our
customer-facing site. Both rely heavily on Solr and a smattering of custom
Lucene API calls throughout the code base. As of this morning (2 weeks
after our final-stage push to production), we've reduced our hardware needs
over 50% and have improved page response times by 30-60% as compared to the
old architecture (a mix of very old CF code for the browse pieces, and a
couple GSAs for the seach pages). More importantly, we've established new
patterns and flex points for future growth, and I have utmost confidence
that we'll be leveraging the Lucene/Solr combo for years to come as we grow.
The devs are proud, the execs are happy, our customers have noticed the
speed improvements, and we all learned many positive lessons along the way.
These forums and various online resources (the Solr wiki first and foremost)
have been tremendously helpful. And so, thank you, all, for pointing the
way to a simpler, faster, better solution for our needs.
Cass Costello
--
A complex system that works is invariably found to have evolved from a
simple system that works.
- John Gaule
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org