You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Chris Hostetter <ho...@fucit.org> on 2025/09/01 19:39:27 UTC
Tue, 30 Aug 2005 19:25:14 -0700 (PDT)
If you will indulge me a moment of (slightly delayed) nostalgia...
This past Saturday (I was traveling) marked the 20th anniversary of the
first public acknowledgement of the existence of Solr -- albeit, not by
name...
https://lists.apache.org/thread/g8t11tf2gs5x2wtdj09o15toxo1q91o7
Subject: Announcement: Lucene powering CNET.com Product Category Listings
>> At a high level there are four major pieces...
>>
>> 1) A Servlet which abstracts away most of the Lucene index modification
>> APIs into an HTTP/XML based "web service" by accepting POSTed XML
>> documents to add/update in the index. It also replies to GET search
>> requests using query plugins that have access to an IndexReader.
>>
>> ...
IIRC...
At this point in time, Yonik had been working for CNET (on the east coast)
for about a year focused on prototyping a "Lucene Server" -- initially as
a skunkworks project. I had been working (on the west coast) on
"Dynamicly Generated Facet Pages" (secretly) using Lucene since roughly
the same time as Yonik joined the company -- but we didn't know each
other.
When I was "caught" using Lucene ~ Dec 2004, and told to use the approved
database features instead, the performance numbers I showed in my "Either
replace me or leave me the fuck alone" email gave our CTO something to
chew on, and the excuse Yonik's boss was looking for to unveil their
skunkworks project.
So Yonik & I were finally introduced ~ Jan 2005 when our projects were
blessed & merged. Yonik focused on the lower level "engine" that
interacted with Lucene, while I focused on implementing the complex biz
requirements for the facets. We both quietly discussed how to architect
everything so the biz logic specific code could live in "plugins" in the
hope of eventually convincing the CTO to let us contribute the main
"server" to Apache.
Open Source "participation" was a completely new concept to the company at
that point, and it took a lot of approval from multiple tiers of tech
managers for me to even be able to send that email -- but it helped that
my Biz unit site owner (Mark) was really happy with with the engagement
numbers on these new facet pages, and how responsive & interactive they
were (compared with what we had before), so he was happy to let me brag a
little bit.
Mark's boss was pressing us to try and implement "faceted product catalog
search" (something that would have never been possible with the commercial
search engine we were using at the time) in time for the holiday shopping
season. Which meant I was looking into keyword relevancy tuning (facets
were cool, but we had to match documents at least as well as our previous
engine) while Yonik was putting a lot of effort into performance
improvements to handle the additional traffic -- which included upgrading
from Lucene 1.4 -> 1.9.
The speed at which the community was moving forward with improvements;
combined with our ability to iterate on new feature & performance
improvements quickly (due to both the open source nature of Lucene, as
well as the community advice to help solve problems we encountered on the
way); combined with the enthusiasm of the biz teams; were a huge factor in
convincing our previously hostile CTO to go "all in" on contributing
"Solr" to the ASF -- less then a year after I got reprimanding for using
Lucene at all.
I'm skeptical that I'll still be an active member of the Solr community in
another 20 years -- but I am pretty confident that there will still be an
active community, ... and hopefully they will still be sharing their cool
use cases & accomplishments on the mailing list.
-Hoss
http://www.lucidworks.com/
Re: Tue, 30 Aug 2005 19:25:14 -0700 (PDT)
Posted by Tim Casey <tc...@gmail.com>.
In 2004 I stared working for a company doing unstructured analysis for
automated user driven categorization. We used lucene as the backend for
representing the documents.
We did many many projects around this low level thing, which we would just
jam in all of the documents. And then run multi-page queries through it as
a way to show categorization to the users. We built our own query parser
to build lucene queries in java.
Over time, the backdrop of lucene led to us building our own query engine.
Lucene, Solr and the eco system around it has been a good backdrop of what
it means to do search, what it means to have meaning within a query and
ultimately what it means to go from 'finding' to semantically understanding
a query space and returning related documents in that specific space.
If there were a computer science hall of fame, it should include this work.
20 years in software is like 10 life times.
Tim
On Mon, Sep 1, 2025 at 2:28 PM Walter Underwood <wu...@wunderwood.org>
wrote:
> I was at Netflix and was about to write a server around Lucene. Then Solr
> was released, so I didn’t have to do that. I was a user since version 1.2.
> Continued using it at Chegg and Lexis/Nexis. (300+ node clusters).
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
> > On Sep 1, 2025, at 12:39 PM, Chris Hostetter <ho...@fucit.org>
> wrote:
> >
> >
> > If you will indulge me a moment of (slightly delayed) nostalgia...
> >
> > This past Saturday (I was traveling) marked the 20th anniversary of the
> first public acknowledgement of the existence of Solr -- albeit, not by
> name...
> >
> > https://lists.apache.org/thread/g8t11tf2gs5x2wtdj09o15toxo1q91o7
> > Subject: Announcement: Lucene powering CNET.com Product Category Listings
> >
> >>> At a high level there are four major pieces...
> >>> 1) A Servlet which abstracts away most of the Lucene index modification
> >>> APIs into an HTTP/XML based "web service" by accepting POSTed XML
> >>> documents to add/update in the index. It also replies to GET search
> >>> requests using query plugins that have access to an IndexReader.
> >>>
> >>> ...
> >
> > IIRC...
> >
> > At this point in time, Yonik had been working for CNET (on the east
> coast) for about a year focused on prototyping a "Lucene Server" --
> initially as a skunkworks project. I had been working (on the west coast)
> on "Dynamicly Generated Facet Pages" (secretly) using Lucene since roughly
> the same time as Yonik joined the company -- but we didn't know each other.
> >
> > When I was "caught" using Lucene ~ Dec 2004, and told to use the
> approved database features instead, the performance numbers I showed in my
> "Either replace me or leave me the fuck alone" email gave our CTO something
> to chew on, and the excuse Yonik's boss was looking for to unveil their
> skunkworks project.
> >
> > So Yonik & I were finally introduced ~ Jan 2005 when our projects were
> blessed & merged. Yonik focused on the lower level "engine" that
> interacted with Lucene, while I focused on implementing the complex biz
> requirements for the facets. We both quietly discussed how to architect
> everything so the biz logic specific code could live in "plugins" in the
> hope of eventually convincing the CTO to let us contribute the main
> "server" to Apache.
> >
> > Open Source "participation" was a completely new concept to the company
> at that point, and it took a lot of approval from multiple tiers of tech
> managers for me to even be able to send that email -- but it helped that my
> Biz unit site owner (Mark) was really happy with with the engagement
> numbers on these new facet pages, and how responsive & interactive they
> were (compared with what we had before), so he was happy to let me brag a
> little bit.
> >
> > Mark's boss was pressing us to try and implement "faceted product
> catalog search" (something that would have never been possible with the
> commercial search engine we were using at the time) in time for the holiday
> shopping season. Which meant I was looking into keyword relevancy tuning
> (facets were cool, but we had to match documents at least as well as our
> previous engine) while Yonik was putting a lot of effort into performance
> improvements to handle the additional traffic -- which included upgrading
> from Lucene 1.4 -> 1.9.
> >
> > The speed at which the community was moving forward with improvements;
> combined with our ability to iterate on new feature & performance
> improvements quickly (due to both the open source nature of Lucene, as well
> as the community advice to help solve problems we encountered on the way);
> combined with the enthusiasm of the biz teams; were a huge factor in
> convincing our previously hostile CTO to go "all in" on contributing "Solr"
> to the ASF -- less then a year after I got reprimanding for using Lucene at
> all.
> >
> >
> > I'm skeptical that I'll still be an active member of the Solr community
> in another 20 years -- but I am pretty confident that there will still be
> an active community, ... and hopefully they will still be sharing their
> cool use cases & accomplishments on the mailing list.
> >
> >
> >
> >
> > -Hoss
> > http://www.lucidworks.com/
>
>
Re: Tue, 30 Aug 2005 19:25:14 -0700 (PDT)
Posted by Walter Underwood <wu...@wunderwood.org>.
I was at Netflix and was about to write a server around Lucene. Then Solr was released, so I didn’t have to do that. I was a user since version 1.2. Continued using it at Chegg and Lexis/Nexis. (300+ node clusters).
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Sep 1, 2025, at 12:39 PM, Chris Hostetter <ho...@fucit.org> wrote:
>
>
> If you will indulge me a moment of (slightly delayed) nostalgia...
>
> This past Saturday (I was traveling) marked the 20th anniversary of the first public acknowledgement of the existence of Solr -- albeit, not by name...
>
> https://lists.apache.org/thread/g8t11tf2gs5x2wtdj09o15toxo1q91o7
> Subject: Announcement: Lucene powering CNET.com Product Category Listings
>
>>> At a high level there are four major pieces...
>>> 1) A Servlet which abstracts away most of the Lucene index modification
>>> APIs into an HTTP/XML based "web service" by accepting POSTed XML
>>> documents to add/update in the index. It also replies to GET search
>>> requests using query plugins that have access to an IndexReader.
>>>
>>> ...
>
> IIRC...
>
> At this point in time, Yonik had been working for CNET (on the east coast) for about a year focused on prototyping a "Lucene Server" -- initially as a skunkworks project. I had been working (on the west coast) on "Dynamicly Generated Facet Pages" (secretly) using Lucene since roughly the same time as Yonik joined the company -- but we didn't know each other.
>
> When I was "caught" using Lucene ~ Dec 2004, and told to use the approved database features instead, the performance numbers I showed in my "Either replace me or leave me the fuck alone" email gave our CTO something to chew on, and the excuse Yonik's boss was looking for to unveil their skunkworks project.
>
> So Yonik & I were finally introduced ~ Jan 2005 when our projects were blessed & merged. Yonik focused on the lower level "engine" that interacted with Lucene, while I focused on implementing the complex biz requirements for the facets. We both quietly discussed how to architect everything so the biz logic specific code could live in "plugins" in the hope of eventually convincing the CTO to let us contribute the main "server" to Apache.
>
> Open Source "participation" was a completely new concept to the company at that point, and it took a lot of approval from multiple tiers of tech managers for me to even be able to send that email -- but it helped that my Biz unit site owner (Mark) was really happy with with the engagement numbers on these new facet pages, and how responsive & interactive they were (compared with what we had before), so he was happy to let me brag a little bit.
>
> Mark's boss was pressing us to try and implement "faceted product catalog search" (something that would have never been possible with the commercial search engine we were using at the time) in time for the holiday shopping season. Which meant I was looking into keyword relevancy tuning (facets were cool, but we had to match documents at least as well as our previous engine) while Yonik was putting a lot of effort into performance improvements to handle the additional traffic -- which included upgrading from Lucene 1.4 -> 1.9.
>
> The speed at which the community was moving forward with improvements; combined with our ability to iterate on new feature & performance improvements quickly (due to both the open source nature of Lucene, as well as the community advice to help solve problems we encountered on the way); combined with the enthusiasm of the biz teams; were a huge factor in convincing our previously hostile CTO to go "all in" on contributing "Solr" to the ASF -- less then a year after I got reprimanding for using Lucene at all.
>
>
> I'm skeptical that I'll still be an active member of the Solr community in another 20 years -- but I am pretty confident that there will still be an active community, ... and hopefully they will still be sharing their cool use cases & accomplishments on the mailing list.
>
>
>
>
> -Hoss
> http://www.lucidworks.com/
Re: Tue, 30 Aug 2005 19:25:14 -0700 (PDT)
Posted by Gus Heck <gu...@gmail.com>.
Thanks for sharing that, Hoss. That's a really cool bit of history.
On Mon, Sep 1, 2025 at 3:41 PM Chris Hostetter <ho...@fucit.org>
wrote:
>
> If you will indulge me a moment of (slightly delayed) nostalgia...
>
> This past Saturday (I was traveling) marked the 20th anniversary of the
> first public acknowledgement of the existence of Solr -- albeit, not by
> name...
>
> https://lists.apache.org/thread/g8t11tf2gs5x2wtdj09o15toxo1q91o7
> Subject: Announcement: Lucene powering CNET.com Product Category Listings
>
> >> At a high level there are four major pieces...
> >>
> >> 1) A Servlet which abstracts away most of the Lucene index modification
> >> APIs into an HTTP/XML based "web service" by accepting POSTed XML
> >> documents to add/update in the index. It also replies to GET search
> >> requests using query plugins that have access to an IndexReader.
> >>
> >> ...
>
> IIRC...
>
> At this point in time, Yonik had been working for CNET (on the east coast)
> for about a year focused on prototyping a "Lucene Server" -- initially as
> a skunkworks project. I had been working (on the west coast) on
> "Dynamicly Generated Facet Pages" (secretly) using Lucene since roughly
> the same time as Yonik joined the company -- but we didn't know each
> other.
>
> When I was "caught" using Lucene ~ Dec 2004, and told to use the approved
> database features instead, the performance numbers I showed in my "Either
> replace me or leave me the fuck alone" email gave our CTO something to
> chew on, and the excuse Yonik's boss was looking for to unveil their
> skunkworks project.
>
> So Yonik & I were finally introduced ~ Jan 2005 when our projects were
> blessed & merged. Yonik focused on the lower level "engine" that
> interacted with Lucene, while I focused on implementing the complex biz
> requirements for the facets. We both quietly discussed how to architect
> everything so the biz logic specific code could live in "plugins" in the
> hope of eventually convincing the CTO to let us contribute the main
> "server" to Apache.
>
> Open Source "participation" was a completely new concept to the company at
> that point, and it took a lot of approval from multiple tiers of tech
> managers for me to even be able to send that email -- but it helped that
> my Biz unit site owner (Mark) was really happy with with the engagement
> numbers on these new facet pages, and how responsive & interactive they
> were (compared with what we had before), so he was happy to let me brag a
> little bit.
>
> Mark's boss was pressing us to try and implement "faceted product catalog
> search" (something that would have never been possible with the commercial
> search engine we were using at the time) in time for the holiday shopping
> season. Which meant I was looking into keyword relevancy tuning (facets
> were cool, but we had to match documents at least as well as our previous
> engine) while Yonik was putting a lot of effort into performance
> improvements to handle the additional traffic -- which included upgrading
> from Lucene 1.4 -> 1.9.
>
> The speed at which the community was moving forward with improvements;
> combined with our ability to iterate on new feature & performance
> improvements quickly (due to both the open source nature of Lucene, as
> well as the community advice to help solve problems we encountered on the
> way); combined with the enthusiasm of the biz teams; were a huge factor in
> convincing our previously hostile CTO to go "all in" on contributing
> "Solr" to the ASF -- less then a year after I got reprimanding for using
> Lucene at all.
>
>
> I'm skeptical that I'll still be an active member of the Solr community in
> another 20 years -- but I am pretty confident that there will still be an
> active community, ... and hopefully they will still be sharing their cool
> use cases & accomplishments on the mailing list.
>
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
--
http://www.needhamsoftware.com (work)
https://a.co/d/b2sZLD9 (my fantasy fiction book)
Re: Tue, 30 Aug 2005 19:25:14 -0700 (PDT)
Posted by Robi Petersen <ro...@gmail.com>.
On Tue, Sep 2, 2025 at 3:47 PM Walter Underwood <wu...@wunderwood.org>
wrote:
> One other bit about that first release. I’d been working on the Ultraseek
> search engine for the previous decade. When I installed Solr, I was
> impressed and annoyed in equal measure about how cool and clean the
> analysis chain config was and that we’d never thought to do that in
> Ultraseek. Out of 3000 customers, we’d had maybe 20 that needed something
> special, but still, it was so nicely designed.
>
> Ultraseek bounced around between companies enough that we eventually
> declared “Six Flags Over Ultraseek”. Those were Infoseek, Disney/GO.com <
> http://go.com/>, Inktomi, Verity, and Autonomy.
>
> At Verity, I tried to talk them into moving to Lucene from VDK, but that
> was way out of their comfort zone.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
> > On Sep 1, 2025, at 3:48 PM, Jan Høydahl <ja...@cominvent.com> wrote:
> >
> > Interesting bit of early Solr history, thanks for sharing! Blog post
> candidate for the Solr web page?
> >
> > Let me join you down memory lane as well. Back in 2005, I was working at
> FAST Search & Transfer, which at the time was considered state of the art
> in enterprise search. Our engine, written in C++, offered impressive search
> and faceting capabilities—though it came with a hefty price tag, as well as
> hardware requirements. Only a few years earlier, we had lost a potential
> deal with a well-known classifieds site because our faceting code was still
> buggy and underperforming. Faceting imo was the killer feature of Solr!
> >
> > My first encounter with Lucene came in 2006, while I was consulting for
> FAST on an EU research project called DILIGENT. The project focused on
> building distributed compute infrastructure—they called it GRID
> computing—complete with SOA and all the buzzwords of the day. The indexing
> and search layer was powered by FAST ESP, but I was tasked with integrating
> Lucene as a free, alternative engine. I still remember being struck by how
> Lucene was both faster and produced a far smaller index footprint than
> FAST. A few years later, as FAST customers started looking for a
> replacement when Microsoft (the new owner) discontinued the product, I went
> full-time Solr. It turned out to be the perfect replacement, even for the
> largest clusters!
> >
> > Jan
> >
> >> 1. sep. 2025 kl. 21:39 skrev Chris Hostetter <hossman_lucene@fucit.org
> >:
> >>
> >>
> >> If you will indulge me a moment of (slightly delayed) nostalgia...
> >>
> >> This past Saturday (I was traveling) marked the 20th anniversary of the
> first public acknowledgement of the existence of Solr -- albeit, not by
> name...
> >>
> >> https://lists.apache.org/thread/g8t11tf2gs5x2wtdj09o15toxo1q91o7
> >> Subject: Announcement: Lucene powering CNET.com Product Category
> Listings
> >>
> >>>> At a high level there are four major pieces...
> >>>> 1) A Servlet which abstracts away most of the Lucene index
> modification
> >>>> APIs into an HTTP/XML based "web service" by accepting POSTed XML
> >>>> documents to add/update in the index. It also replies to GET search
> >>>> requests using query plugins that have access to an IndexReader.
> >>>>
> >>>> ...
> >>
> >> IIRC...
> >>
> >> At this point in time, Yonik had been working for CNET (on the east
> coast) for about a year focused on prototyping a "Lucene Server" --
> initially as a skunkworks project. I had been working (on the west coast)
> on "Dynamicly Generated Facet Pages" (secretly) using Lucene since roughly
> the same time as Yonik joined the company -- but we didn't know each other.
> >>
> >> When I was "caught" using Lucene ~ Dec 2004, and told to use the
> approved database features instead, the performance numbers I showed in my
> "Either replace me or leave me the fuck alone" email gave our CTO something
> to chew on, and the excuse Yonik's boss was looking for to unveil their
> skunkworks project.
> >>
> >> So Yonik & I were finally introduced ~ Jan 2005 when our projects were
> blessed & merged. Yonik focused on the lower level "engine" that
> interacted with Lucene, while I focused on implementing the complex biz
> requirements for the facets. We both quietly discussed how to architect
> everything so the biz logic specific code could live in "plugins" in the
> hope of eventually convincing the CTO to let us contribute the main
> "server" to Apache.
> >>
> >> Open Source "participation" was a completely new concept to the company
> at that point, and it took a lot of approval from multiple tiers of tech
> managers for me to even be able to send that email -- but it helped that my
> Biz unit site owner (Mark) was really happy with with the engagement
> numbers on these new facet pages, and how responsive & interactive they
> were (compared with what we had before), so he was happy to let me brag a
> little bit.
> >>
> >> Mark's boss was pressing us to try and implement "faceted product
> catalog search" (something that would have never been possible with the
> commercial search engine we were using at the time) in time for the holiday
> shopping season. Which meant I was looking into keyword relevancy tuning
> (facets were cool, but we had to match documents at least as well as our
> previous engine) while Yonik was putting a lot of effort into performance
> improvements to handle the additional traffic -- which included upgrading
> from Lucene 1.4 -> 1.9.
> >>
> >> The speed at which the community was moving forward with improvements;
> combined with our ability to iterate on new feature & performance
> improvements quickly (due to both the open source nature of Lucene, as well
> as the community advice to help solve problems we encountered on the way);
> combined with the enthusiasm of the biz teams; were a huge factor in
> convincing our previously hostile CTO to go "all in" on contributing "Solr"
> to the ASF -- less then a year after I got reprimanding for using Lucene at
> all.
> >>
> >>
> >> I'm skeptical that I'll still be an active member of the Solr community
> in another 20 years -- but I am pretty confident that there will still be
> an active community, ... and hopefully they will still be sharing their
> cool use cases & accomplishments on the mailing list.
> >>
> >>
> >>
> >>
> >> -Hoss
> >> http://www.lucidworks.com/
> >
>
>
Re: Tue, 30 Aug 2005 19:25:14 -0700 (PDT)
Posted by Robi Petersen <ro...@gmail.com>.
PS to Jan, Fast search came in to buy.com with a presentation during the
above turmoil. I was most impressed with the drag n drop reordering of
search results. It's sad Fast search now lives in M$ sharepoint right?
On Wed, Sep 3, 2025 at 9:20 AM Robi Petersen <ro...@gmail.com> wrote:
> To chime in with wunderwood, I was in a similar boat. I was at buy.com
> and was about to write a wrapper around Lucene. When I took over the search
> department there, the stack was C++ spaghetti code around an alta vista
> search engine DLL with a VB front end for devops. Then thankfully Solr was
> open sourced. ANDing bit vectors together to make facets fall out was
> intense. Instead we got mgmt to allow linux into our MS shop and we rolled
> with solr.. I was a user since version 1.2 and we went live at buy.com
> on version 1.3. Continued using it for literally the rest of my life!
> Haven't stopped yet.
>
> On Tue, Sep 2, 2025 at 3:47 PM Walter Underwood <wu...@wunderwood.org>
> wrote:
>
>> One other bit about that first release. I’d been working on the Ultraseek
>> search engine for the previous decade. When I installed Solr, I was
>> impressed and annoyed in equal measure about how cool and clean the
>> analysis chain config was and that we’d never thought to do that in
>> Ultraseek. Out of 3000 customers, we’d had maybe 20 that needed something
>> special, but still, it was so nicely designed.
>>
>> Ultraseek bounced around between companies enough that we eventually
>> declared “Six Flags Over Ultraseek”. Those were Infoseek, Disney/GO.com <
>> http://go.com/>, Inktomi, Verity, and Autonomy.
>>
>> At Verity, I tried to talk them into moving to Lucene from VDK, but that
>> was way out of their comfort zone.
>>
>> wunder
>> Walter Underwood
>> wunder@wunderwood.org
>> http://observer.wunderwood.org/ (my blog)
>>
>> > On Sep 1, 2025, at 3:48 PM, Jan Høydahl <ja...@cominvent.com> wrote:
>> >
>> > Interesting bit of early Solr history, thanks for sharing! Blog post
>> candidate for the Solr web page?
>> >
>> > Let me join you down memory lane as well. Back in 2005, I was working
>> at FAST Search & Transfer, which at the time was considered state of the
>> art in enterprise search. Our engine, written in C++, offered impressive
>> search and faceting capabilities—though it came with a hefty price tag, as
>> well as hardware requirements. Only a few years earlier, we had lost a
>> potential deal with a well-known classifieds site because our faceting code
>> was still buggy and underperforming. Faceting imo was the killer feature of
>> Solr!
>> >
>> > My first encounter with Lucene came in 2006, while I was consulting for
>> FAST on an EU research project called DILIGENT. The project focused on
>> building distributed compute infrastructure—they called it GRID
>> computing—complete with SOA and all the buzzwords of the day. The indexing
>> and search layer was powered by FAST ESP, but I was tasked with integrating
>> Lucene as a free, alternative engine. I still remember being struck by how
>> Lucene was both faster and produced a far smaller index footprint than
>> FAST. A few years later, as FAST customers started looking for a
>> replacement when Microsoft (the new owner) discontinued the product, I went
>> full-time Solr. It turned out to be the perfect replacement, even for the
>> largest clusters!
>> >
>> > Jan
>> >
>> >> 1. sep. 2025 kl. 21:39 skrev Chris Hostetter <hossman_lucene@fucit.org
>> >:
>> >>
>> >>
>> >> If you will indulge me a moment of (slightly delayed) nostalgia...
>> >>
>> >> This past Saturday (I was traveling) marked the 20th anniversary of
>> the first public acknowledgement of the existence of Solr -- albeit, not by
>> name...
>> >>
>> >> https://lists.apache.org/thread/g8t11tf2gs5x2wtdj09o15toxo1q91o7
>> >> Subject: Announcement: Lucene powering CNET.com Product Category
>> Listings
>> >>
>> >>>> At a high level there are four major pieces...
>> >>>> 1) A Servlet which abstracts away most of the Lucene index
>> modification
>> >>>> APIs into an HTTP/XML based "web service" by accepting POSTed XML
>> >>>> documents to add/update in the index. It also replies to GET search
>> >>>> requests using query plugins that have access to an IndexReader.
>> >>>>
>> >>>> ...
>> >>
>> >> IIRC...
>> >>
>> >> At this point in time, Yonik had been working for CNET (on the east
>> coast) for about a year focused on prototyping a "Lucene Server" --
>> initially as a skunkworks project. I had been working (on the west coast)
>> on "Dynamicly Generated Facet Pages" (secretly) using Lucene since roughly
>> the same time as Yonik joined the company -- but we didn't know each other.
>> >>
>> >> When I was "caught" using Lucene ~ Dec 2004, and told to use the
>> approved database features instead, the performance numbers I showed in my
>> "Either replace me or leave me the fuck alone" email gave our CTO something
>> to chew on, and the excuse Yonik's boss was looking for to unveil their
>> skunkworks project.
>> >>
>> >> So Yonik & I were finally introduced ~ Jan 2005 when our projects were
>> blessed & merged. Yonik focused on the lower level "engine" that
>> interacted with Lucene, while I focused on implementing the complex biz
>> requirements for the facets. We both quietly discussed how to architect
>> everything so the biz logic specific code could live in "plugins" in the
>> hope of eventually convincing the CTO to let us contribute the main
>> "server" to Apache.
>> >>
>> >> Open Source "participation" was a completely new concept to the
>> company at that point, and it took a lot of approval from multiple tiers of
>> tech managers for me to even be able to send that email -- but it helped
>> that my Biz unit site owner (Mark) was really happy with with the
>> engagement numbers on these new facet pages, and how responsive &
>> interactive they were (compared with what we had before), so he was happy
>> to let me brag a little bit.
>> >>
>> >> Mark's boss was pressing us to try and implement "faceted product
>> catalog search" (something that would have never been possible with the
>> commercial search engine we were using at the time) in time for the holiday
>> shopping season. Which meant I was looking into keyword relevancy tuning
>> (facets were cool, but we had to match documents at least as well as our
>> previous engine) while Yonik was putting a lot of effort into performance
>> improvements to handle the additional traffic -- which included upgrading
>> from Lucene 1.4 -> 1.9.
>> >>
>> >> The speed at which the community was moving forward with improvements;
>> combined with our ability to iterate on new feature & performance
>> improvements quickly (due to both the open source nature of Lucene, as well
>> as the community advice to help solve problems we encountered on the way);
>> combined with the enthusiasm of the biz teams; were a huge factor in
>> convincing our previously hostile CTO to go "all in" on contributing "Solr"
>> to the ASF -- less then a year after I got reprimanding for using Lucene at
>> all.
>> >>
>> >>
>> >> I'm skeptical that I'll still be an active member of the Solr
>> community in another 20 years -- but I am pretty confident that there will
>> still be an active community, ... and hopefully they will still be sharing
>> their cool use cases & accomplishments on the mailing list.
>> >>
>> >>
>> >>
>> >>
>> >> -Hoss
>> >> http://www.lucidworks.com/
>> >
>>
>>
Re: Tue, 30 Aug 2005 19:25:14 -0700 (PDT)
Posted by Robi Petersen <ro...@gmail.com>.
To chime in with wunderwood, I was in a similar boat. I was at buy.com and
was about to write a wrapper around Lucene. When I took over the search
department there, the stack was C++ spaghetti code around an alta vista
search engine DLL with a VB front end for devops. Then thankfully Solr was
open sourced. ANDing bit vectors together to make facets fall out was
intense. Instead we got mgmt to allow linux into our MS shop and we rolled
with solr.. I was a user since version 1.2 and we went live at buy.com
on version 1.3. Continued using it for literally the rest of my life!
Haven't stopped yet.
On Tue, Sep 2, 2025 at 3:47 PM Walter Underwood <wu...@wunderwood.org>
wrote:
> One other bit about that first release. I’d been working on the Ultraseek
> search engine for the previous decade. When I installed Solr, I was
> impressed and annoyed in equal measure about how cool and clean the
> analysis chain config was and that we’d never thought to do that in
> Ultraseek. Out of 3000 customers, we’d had maybe 20 that needed something
> special, but still, it was so nicely designed.
>
> Ultraseek bounced around between companies enough that we eventually
> declared “Six Flags Over Ultraseek”. Those were Infoseek, Disney/GO.com <
> http://go.com/>, Inktomi, Verity, and Autonomy.
>
> At Verity, I tried to talk them into moving to Lucene from VDK, but that
> was way out of their comfort zone.
>
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/ (my blog)
>
> > On Sep 1, 2025, at 3:48 PM, Jan Høydahl <ja...@cominvent.com> wrote:
> >
> > Interesting bit of early Solr history, thanks for sharing! Blog post
> candidate for the Solr web page?
> >
> > Let me join you down memory lane as well. Back in 2005, I was working at
> FAST Search & Transfer, which at the time was considered state of the art
> in enterprise search. Our engine, written in C++, offered impressive search
> and faceting capabilities—though it came with a hefty price tag, as well as
> hardware requirements. Only a few years earlier, we had lost a potential
> deal with a well-known classifieds site because our faceting code was still
> buggy and underperforming. Faceting imo was the killer feature of Solr!
> >
> > My first encounter with Lucene came in 2006, while I was consulting for
> FAST on an EU research project called DILIGENT. The project focused on
> building distributed compute infrastructure—they called it GRID
> computing—complete with SOA and all the buzzwords of the day. The indexing
> and search layer was powered by FAST ESP, but I was tasked with integrating
> Lucene as a free, alternative engine. I still remember being struck by how
> Lucene was both faster and produced a far smaller index footprint than
> FAST. A few years later, as FAST customers started looking for a
> replacement when Microsoft (the new owner) discontinued the product, I went
> full-time Solr. It turned out to be the perfect replacement, even for the
> largest clusters!
> >
> > Jan
> >
> >> 1. sep. 2025 kl. 21:39 skrev Chris Hostetter <hossman_lucene@fucit.org
> >:
> >>
> >>
> >> If you will indulge me a moment of (slightly delayed) nostalgia...
> >>
> >> This past Saturday (I was traveling) marked the 20th anniversary of the
> first public acknowledgement of the existence of Solr -- albeit, not by
> name...
> >>
> >> https://lists.apache.org/thread/g8t11tf2gs5x2wtdj09o15toxo1q91o7
> >> Subject: Announcement: Lucene powering CNET.com Product Category
> Listings
> >>
> >>>> At a high level there are four major pieces...
> >>>> 1) A Servlet which abstracts away most of the Lucene index
> modification
> >>>> APIs into an HTTP/XML based "web service" by accepting POSTed XML
> >>>> documents to add/update in the index. It also replies to GET search
> >>>> requests using query plugins that have access to an IndexReader.
> >>>>
> >>>> ...
> >>
> >> IIRC...
> >>
> >> At this point in time, Yonik had been working for CNET (on the east
> coast) for about a year focused on prototyping a "Lucene Server" --
> initially as a skunkworks project. I had been working (on the west coast)
> on "Dynamicly Generated Facet Pages" (secretly) using Lucene since roughly
> the same time as Yonik joined the company -- but we didn't know each other.
> >>
> >> When I was "caught" using Lucene ~ Dec 2004, and told to use the
> approved database features instead, the performance numbers I showed in my
> "Either replace me or leave me the fuck alone" email gave our CTO something
> to chew on, and the excuse Yonik's boss was looking for to unveil their
> skunkworks project.
> >>
> >> So Yonik & I were finally introduced ~ Jan 2005 when our projects were
> blessed & merged. Yonik focused on the lower level "engine" that
> interacted with Lucene, while I focused on implementing the complex biz
> requirements for the facets. We both quietly discussed how to architect
> everything so the biz logic specific code could live in "plugins" in the
> hope of eventually convincing the CTO to let us contribute the main
> "server" to Apache.
> >>
> >> Open Source "participation" was a completely new concept to the company
> at that point, and it took a lot of approval from multiple tiers of tech
> managers for me to even be able to send that email -- but it helped that my
> Biz unit site owner (Mark) was really happy with with the engagement
> numbers on these new facet pages, and how responsive & interactive they
> were (compared with what we had before), so he was happy to let me brag a
> little bit.
> >>
> >> Mark's boss was pressing us to try and implement "faceted product
> catalog search" (something that would have never been possible with the
> commercial search engine we were using at the time) in time for the holiday
> shopping season. Which meant I was looking into keyword relevancy tuning
> (facets were cool, but we had to match documents at least as well as our
> previous engine) while Yonik was putting a lot of effort into performance
> improvements to handle the additional traffic -- which included upgrading
> from Lucene 1.4 -> 1.9.
> >>
> >> The speed at which the community was moving forward with improvements;
> combined with our ability to iterate on new feature & performance
> improvements quickly (due to both the open source nature of Lucene, as well
> as the community advice to help solve problems we encountered on the way);
> combined with the enthusiasm of the biz teams; were a huge factor in
> convincing our previously hostile CTO to go "all in" on contributing "Solr"
> to the ASF -- less then a year after I got reprimanding for using Lucene at
> all.
> >>
> >>
> >> I'm skeptical that I'll still be an active member of the Solr community
> in another 20 years -- but I am pretty confident that there will still be
> an active community, ... and hopefully they will still be sharing their
> cool use cases & accomplishments on the mailing list.
> >>
> >>
> >>
> >>
> >> -Hoss
> >> http://www.lucidworks.com/
> >
>
>
Re: Tue, 30 Aug 2005 19:25:14 -0700 (PDT)
Posted by Walter Underwood <wu...@wunderwood.org>.
One other bit about that first release. I’d been working on the Ultraseek search engine for the previous decade. When I installed Solr, I was impressed and annoyed in equal measure about how cool and clean the analysis chain config was and that we’d never thought to do that in Ultraseek. Out of 3000 customers, we’d had maybe 20 that needed something special, but still, it was so nicely designed.
Ultraseek bounced around between companies enough that we eventually declared “Six Flags Over Ultraseek”. Those were Infoseek, Disney/GO.com <http://go.com/>, Inktomi, Verity, and Autonomy.
At Verity, I tried to talk them into moving to Lucene from VDK, but that was way out of their comfort zone.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Sep 1, 2025, at 3:48 PM, Jan Høydahl <ja...@cominvent.com> wrote:
>
> Interesting bit of early Solr history, thanks for sharing! Blog post candidate for the Solr web page?
>
> Let me join you down memory lane as well. Back in 2005, I was working at FAST Search & Transfer, which at the time was considered state of the art in enterprise search. Our engine, written in C++, offered impressive search and faceting capabilities—though it came with a hefty price tag, as well as hardware requirements. Only a few years earlier, we had lost a potential deal with a well-known classifieds site because our faceting code was still buggy and underperforming. Faceting imo was the killer feature of Solr!
>
> My first encounter with Lucene came in 2006, while I was consulting for FAST on an EU research project called DILIGENT. The project focused on building distributed compute infrastructure—they called it GRID computing—complete with SOA and all the buzzwords of the day. The indexing and search layer was powered by FAST ESP, but I was tasked with integrating Lucene as a free, alternative engine. I still remember being struck by how Lucene was both faster and produced a far smaller index footprint than FAST. A few years later, as FAST customers started looking for a replacement when Microsoft (the new owner) discontinued the product, I went full-time Solr. It turned out to be the perfect replacement, even for the largest clusters!
>
> Jan
>
>> 1. sep. 2025 kl. 21:39 skrev Chris Hostetter <ho...@fucit.org>:
>>
>>
>> If you will indulge me a moment of (slightly delayed) nostalgia...
>>
>> This past Saturday (I was traveling) marked the 20th anniversary of the first public acknowledgement of the existence of Solr -- albeit, not by name...
>>
>> https://lists.apache.org/thread/g8t11tf2gs5x2wtdj09o15toxo1q91o7
>> Subject: Announcement: Lucene powering CNET.com Product Category Listings
>>
>>>> At a high level there are four major pieces...
>>>> 1) A Servlet which abstracts away most of the Lucene index modification
>>>> APIs into an HTTP/XML based "web service" by accepting POSTed XML
>>>> documents to add/update in the index. It also replies to GET search
>>>> requests using query plugins that have access to an IndexReader.
>>>>
>>>> ...
>>
>> IIRC...
>>
>> At this point in time, Yonik had been working for CNET (on the east coast) for about a year focused on prototyping a "Lucene Server" -- initially as a skunkworks project. I had been working (on the west coast) on "Dynamicly Generated Facet Pages" (secretly) using Lucene since roughly the same time as Yonik joined the company -- but we didn't know each other.
>>
>> When I was "caught" using Lucene ~ Dec 2004, and told to use the approved database features instead, the performance numbers I showed in my "Either replace me or leave me the fuck alone" email gave our CTO something to chew on, and the excuse Yonik's boss was looking for to unveil their skunkworks project.
>>
>> So Yonik & I were finally introduced ~ Jan 2005 when our projects were blessed & merged. Yonik focused on the lower level "engine" that interacted with Lucene, while I focused on implementing the complex biz requirements for the facets. We both quietly discussed how to architect everything so the biz logic specific code could live in "plugins" in the hope of eventually convincing the CTO to let us contribute the main "server" to Apache.
>>
>> Open Source "participation" was a completely new concept to the company at that point, and it took a lot of approval from multiple tiers of tech managers for me to even be able to send that email -- but it helped that my Biz unit site owner (Mark) was really happy with with the engagement numbers on these new facet pages, and how responsive & interactive they were (compared with what we had before), so he was happy to let me brag a little bit.
>>
>> Mark's boss was pressing us to try and implement "faceted product catalog search" (something that would have never been possible with the commercial search engine we were using at the time) in time for the holiday shopping season. Which meant I was looking into keyword relevancy tuning (facets were cool, but we had to match documents at least as well as our previous engine) while Yonik was putting a lot of effort into performance improvements to handle the additional traffic -- which included upgrading from Lucene 1.4 -> 1.9.
>>
>> The speed at which the community was moving forward with improvements; combined with our ability to iterate on new feature & performance improvements quickly (due to both the open source nature of Lucene, as well as the community advice to help solve problems we encountered on the way); combined with the enthusiasm of the biz teams; were a huge factor in convincing our previously hostile CTO to go "all in" on contributing "Solr" to the ASF -- less then a year after I got reprimanding for using Lucene at all.
>>
>>
>> I'm skeptical that I'll still be an active member of the Solr community in another 20 years -- but I am pretty confident that there will still be an active community, ... and hopefully they will still be sharing their cool use cases & accomplishments on the mailing list.
>>
>>
>>
>>
>> -Hoss
>> http://www.lucidworks.com/
>
Re: Tue, 30 Aug 2005 19:25:14 -0700 (PDT)
Posted by Jan Høydahl <ja...@cominvent.com>.
Interesting bit of early Solr history, thanks for sharing! Blog post candidate for the Solr web page?
Let me join you down memory lane as well. Back in 2005, I was working at FAST Search & Transfer, which at the time was considered state of the art in enterprise search. Our engine, written in C++, offered impressive search and faceting capabilities—though it came with a hefty price tag, as well as hardware requirements. Only a few years earlier, we had lost a potential deal with a well-known classifieds site because our faceting code was still buggy and underperforming. Faceting imo was the killer feature of Solr!
My first encounter with Lucene came in 2006, while I was consulting for FAST on an EU research project called DILIGENT. The project focused on building distributed compute infrastructure—they called it GRID computing—complete with SOA and all the buzzwords of the day. The indexing and search layer was powered by FAST ESP, but I was tasked with integrating Lucene as a free, alternative engine. I still remember being struck by how Lucene was both faster and produced a far smaller index footprint than FAST. A few years later, as FAST customers started looking for a replacement when Microsoft (the new owner) discontinued the product, I went full-time Solr. It turned out to be the perfect replacement, even for the largest clusters!
Jan
> 1. sep. 2025 kl. 21:39 skrev Chris Hostetter <ho...@fucit.org>:
>
>
> If you will indulge me a moment of (slightly delayed) nostalgia...
>
> This past Saturday (I was traveling) marked the 20th anniversary of the first public acknowledgement of the existence of Solr -- albeit, not by name...
>
> https://lists.apache.org/thread/g8t11tf2gs5x2wtdj09o15toxo1q91o7
> Subject: Announcement: Lucene powering CNET.com Product Category Listings
>
>>> At a high level there are four major pieces...
>>> 1) A Servlet which abstracts away most of the Lucene index modification
>>> APIs into an HTTP/XML based "web service" by accepting POSTed XML
>>> documents to add/update in the index. It also replies to GET search
>>> requests using query plugins that have access to an IndexReader.
>>>
>>> ...
>
> IIRC...
>
> At this point in time, Yonik had been working for CNET (on the east coast) for about a year focused on prototyping a "Lucene Server" -- initially as a skunkworks project. I had been working (on the west coast) on "Dynamicly Generated Facet Pages" (secretly) using Lucene since roughly the same time as Yonik joined the company -- but we didn't know each other.
>
> When I was "caught" using Lucene ~ Dec 2004, and told to use the approved database features instead, the performance numbers I showed in my "Either replace me or leave me the fuck alone" email gave our CTO something to chew on, and the excuse Yonik's boss was looking for to unveil their skunkworks project.
>
> So Yonik & I were finally introduced ~ Jan 2005 when our projects were blessed & merged. Yonik focused on the lower level "engine" that interacted with Lucene, while I focused on implementing the complex biz requirements for the facets. We both quietly discussed how to architect everything so the biz logic specific code could live in "plugins" in the hope of eventually convincing the CTO to let us contribute the main "server" to Apache.
>
> Open Source "participation" was a completely new concept to the company at that point, and it took a lot of approval from multiple tiers of tech managers for me to even be able to send that email -- but it helped that my Biz unit site owner (Mark) was really happy with with the engagement numbers on these new facet pages, and how responsive & interactive they were (compared with what we had before), so he was happy to let me brag a little bit.
>
> Mark's boss was pressing us to try and implement "faceted product catalog search" (something that would have never been possible with the commercial search engine we were using at the time) in time for the holiday shopping season. Which meant I was looking into keyword relevancy tuning (facets were cool, but we had to match documents at least as well as our previous engine) while Yonik was putting a lot of effort into performance improvements to handle the additional traffic -- which included upgrading from Lucene 1.4 -> 1.9.
>
> The speed at which the community was moving forward with improvements; combined with our ability to iterate on new feature & performance improvements quickly (due to both the open source nature of Lucene, as well as the community advice to help solve problems we encountered on the way); combined with the enthusiasm of the biz teams; were a huge factor in convincing our previously hostile CTO to go "all in" on contributing "Solr" to the ASF -- less then a year after I got reprimanding for using Lucene at all.
>
>
> I'm skeptical that I'll still be an active member of the Solr community in another 20 years -- but I am pretty confident that there will still be an active community, ... and hopefully they will still be sharing their cool use cases & accomplishments on the mailing list.
>
>
>
>
> -Hoss
> http://www.lucidworks.com/