Re: Zappos's new Solr Site

[Alok Dhir <ad...@symplicity.com>]

Hey Matt - congratulations on your new site -- it looks great.

I'm curious, after a few weeks of having run this way, what your  
findings are regarding running the shared index on NFS.  Any problems  
as of yet?

I assume you're indexing from one machine and calling 'commit' on the  
others on some schedule to get them to 'see' changes.

How is that working out for you?

---
Alok K. Dhir
adhir@symplicity.com
Symplicity Corporation
1 703 351 0200 x 8080
www.symplicity.com

On Apr 11, 2008, at 1:35 PM, Matthew Runo wrote:

> Hello folks!
>
> First, the link: https://zeta.zappos.com (it's a very early open  
> beta... we're just very proud of everyone's work and wanted to share  
> it with you all)
>
> We've been working on a new site here at Zappos for about the last 7  
> months, with planning going back almost two years. We looked at  
> Endeca, we looked at Fast, we looked at sooooo many commercial  
> search engine technologies in that time that I can't even remember  
> them all. We ended up choosing Solr, and not just because it's free.  
> Solr has a truly wonderful group of users here who respond to  
> support questions far faster than most paid support contracts. I've  
> never had a question that I couldn't get answered on this list, no  
> matter how stupid it's been (sorry Hoss!) =p
>
> Zappos has a long history of using open source technologies to drive  
> their business, and have used Apache 1.3 + Perl 5 for the past 8  
> years. Our new site is written in Java, and is really built around  
> our Solr index. Solr powers all the navigation and facets, as well  
> as the brand list and brand pages. One of the issues with our old  
> site was how database heavy it was, with some pages generating 100s  
> of queries. Zeta is much better in this regard, and we really think  
> Solr is going to serve us very well.
>
> Here's some stats on our Solr index...  158,821 documents in about 2  
> gigs of disk space, running in Tomcat 6 with 10 gigs of ram set  
> aside. We have 5 servers clustered together, and each runs an  
> instance of zeta.zappos.com and a local copy of solr. For now, each  
> of these servers reads from a single Solr index stored on NFS -  
> we'll see how this works out, and are prepared to store a local copy  
> of the index on each server.
>
> Thanks, and we'd love any feedback on the new site (keep in mind,  
> some parts of it aren't quite done).
>
> Matthew Runo
> Software Developer
> Zappos.com
> 702.943.7833
>

Re: Zappos's new Solr Site

[Matthew Runo <mr...@zappos.com>]

We have a dedicated server set up as the "master", with it's own local  
index. We have an NFS mount (read-only) on each of the other machines  
which the master copies it's index to every 20 minutes. We run a  
commit on each "slave" then to force them to open new readers. So far,  
it's worked fine. I would suggest having the reading and writing done  
to different indexes though, it makes it easier when you can have a  
read-only NFS mounted index (no chance of another server updating it  
at all).

Thanks!

Matthew Runo
Software Developer
Zappos.com
702.943.7833

On May 2, 2008, at 6:41 AM, Alok Dhir wrote:
> Hey Matt - congratulations on your new site -- it looks great.
>
> I'm curious, after a few weeks of having run this way, what your  
> findings are regarding running the shared index on NFS.  Any  
> problems as of yet?
>
> I assume you're indexing from one machine and calling 'commit' on  
> the others on some schedule to get them to 'see' changes.
>
> How is that working out for you?
>
> ---
> Alok K. Dhir
> adhir@symplicity.com
> Symplicity Corporation
> 1 703 351 0200 x 8080
> www.symplicity.com
>
> On Apr 11, 2008, at 1:35 PM, Matthew Runo wrote:
>
>> Hello folks!
>>
>> First, the link: https://zeta.zappos.com (it's a very early open  
>> beta... we're just very proud of everyone's work and wanted to  
>> share it with you all)
>>
>> We've been working on a new site here at Zappos for about the last  
>> 7 months, with planning going back almost two years. We looked at  
>> Endeca, we looked at Fast, we looked at sooooo many commercial  
>> search engine technologies in that time that I can't even remember  
>> them all. We ended up choosing Solr, and not just because it's  
>> free. Solr has a truly wonderful group of users here who respond to  
>> support questions far faster than most paid support contracts. I've  
>> never had a question that I couldn't get answered on this list, no  
>> matter how stupid it's been (sorry Hoss!) =p
>>
>> Zappos has a long history of using open source technologies to  
>> drive their business, and have used Apache 1.3 + Perl 5 for the  
>> past 8 years. Our new site is written in Java, and is really built  
>> around our Solr index. Solr powers all the navigation and facets,  
>> as well as the brand list and brand pages. One of the issues with  
>> our old site was how database heavy it was, with some pages  
>> generating 100s of queries. Zeta is much better in this regard, and  
>> we really think Solr is going to serve us very well.
>>
>> Here's some stats on our Solr index...  158,821 documents in about  
>> 2 gigs of disk space, running in Tomcat 6 with 10 gigs of ram set  
>> aside. We have 5 servers clustered together, and each runs an  
>> instance of zeta.zappos.com and a local copy of solr. For now, each  
>> of these servers reads from a single Solr index stored on NFS -  
>> we'll see how this works out, and are prepared to store a local  
>> copy of the index on each server.
>>
>> Thanks, and we'd love any feedback on the new site (keep in mind,  
>> some parts of it aren't quite done).
>>
>> Matthew Runo
>> Software Developer
>> Zappos.com
>> 702.943.7833
>>
>