You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@solr.apache.org by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net> on 2024/04/01 19:53:17 UTC

solr query alerting

Hi All,

A few months ago I wrote the user list about potentially integrating lucene monitor into solr. I have raised this PR with a first attempt at implementing this integration. I'd greatly appreciate any feedback on this even though I still have it marked as draft. I want to make sure I'm heading in the right direction here so input from solr dev community would be extremely valuable :-)

Many thanks,
Luke

Re: solr query alerting

Posted by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net>.

Hi All,

I wanted to bump this and share this recent blog post about netflix using elastic's percolator for reverse search https://netflixtechblog.com/reverse-searching-netflixs-federated-graph-222ac5d23576 to highlight this feature's relevance into 2024 (even if it isn't the highest priority).

I know that we have some different opinions about where a solr equivalent should exist and wanted to stop by to encourage more discussion about this but also encourage more people to peek at the PR. And also thank you Christine for your comments thus far!

Thanks,
Luke 

From: dev@solr.apache.org At: 04/18/24 14:54:46 UTC-4:00To:  dev@solr.apache.org
Subject: Re: solr query alerting

Houston,

I really appreciate the offer to take a look. I would be happy to raise another 
PR to the sandbox repo if you think that this will help get some community 
usage/feedback before it is polished enough to be merged in the main project.

David,

I totally agree with the naming. Speaking from experience, searching for "solr 
lucene monitor" (and any variation thereof) on the Internet was a real pain so 
not sure why I wanted to pass that hardship on to anyone else! In all 
seriousness, I wanted to be consistent with the naming of the lucene feature 
that this "mapped to". Clearly, from the feedback received so far, this is not 
a good enough reason to cascade such a poor name.

Now regarding the sandbox/solr.cool point.. I'll start by saying you probably 
have the best intuition regarding what is useful enough to meet the threshold 
of being maintained in the core project. If this doesn't meet that threshold 
then there is not much more I can say. Having said that, I have some skepticism 
regarding this project being useful outside of solr (even with a link in 
solr.cool or the like). Take for example this solr.cool-advertised one 
https://github.com/SOLR4189/solcolator which implements saved search in a less 
integrated way (and IMO in a less robust way). It hasn't been touched in four 
years and even if it were, any user deciding on developing a saved 
search/alerting platform will trust such a project less than if it were 
maintained in the core project. This hurts adoption and makes it more likely to 
be relegated to the "attic".

The other reasons I can think of merging this idea into the core project are 1. 
Elastic supports this feature as part of its core offering and 2. (as Houston 
stated) this is a feature that is supported in the core lucene project. This 
makes me think it has enough traction (or it did at some point at least).

I'll also add that I am currently trying to convince folks at my company to 
adopt this approach so hopefully I'll soon have a more concrete sense about the 
potential here. Bloomberg is a big user of solr and also has a large footprint 
of custom-built, saved-search platforms, some of which painfully depend on solr 
without reaping any of the benefits regarding its distributed state management.

Luke

From: dev@solr.apache.org At: 04/18/24 10:50:23 UTC-4:00To:  dev@solr.apache.org
Subject: Re: solr query alerting

Thanks for the work here Luke!

I tried to work on something similar years ago and it was far more work
than I initially thought to do it right, so I'm so glad someone is coming
and giving it a go finally!

I'll be able to do some review in a few weeks, but will be on vacation
myself for the next week.

As per the Solr sandbox or http://solr.cool, I think it's fine to add this
in the sandbox for now, but given that this is making a Lucene
functionality available in Solr, I see no reason why it can't eventually
reside in the project.
There are issues with maintenance burden, but that burden exists whether it
is in Solr or outside of it.
So given that this is a very cool feature, that I have heard lots of
requests for, I think it's worth having inside.

- Houston

On Thu, Apr 18, 2024 at 9:10 AM David Smiley <ds...@apache.org> wrote:

> I'm probably not a great reviewer of this TBH as I have only heard &
> read about lucene-monitor/percolator; haven't used such a thing yet.
>
> I did a *quick* review to see how the PR touches the codebase.  It's
> wonderful that it is purely isolated; it requires no changes of Solr
> itself.  I might argue why is it here vs the Solr sandbox or somewhere
> else with publicity in http://solr.cool
>
> The name "Monitor" sounds like an infrastructure monitoring feature.
>
> p.s. I'm on vacation this week with minimal time to collaborate
>
> On Mon, Apr 15, 2024 at 10:27 AM Luke Kot-Zaniewski (BLOOMBERG/ 919
> 3RD A) <lk...@bloomberg.net> wrote:
> >
> > Hey David,
> >
> > Just wanted to bump this. I appreciate you taking a look, and wanted to
> stress that I am looking for even just higher level feedback at this point.
> >
> > Basically, I am questioning the direction I am going in which involves
> exposing some of the lucene-monitor internals (so currently have an
> implicit dependence on a lucene change). OTOH I couldn't figure out a
> better way to apply the lucene-monitor optimizations while still utilizing
> solr's sophisticated index management and scaling. Lucene-monitor is a very
> sealed interface and the way it manages the index and caching (via Monitor
> interface) is not easy to integrate on its own.
> >
> > Again, I'd appreciate any feedback, even partial, that you may have!
> >
> > Thanks,
> > Luke
> >
> > From: dev@solr.apache.org At: 04/08/24 13:43:52 UTC-4:00To:
> dev@solr.apache.org
> > Subject: Re: solr query alerting
> >
> > I'm so glad someone has started this!  Thanks for contributing.  I'll
> > take a look
> >
> > On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> > A) <lk...@bloomberg.net> wrote:
> > >
> > > Hi All,
> > >
> > > A few months ago I wrote the user list about potentially integrating
> lucene
> > monitor into solr. I have raised this PR with a first attempt at
> implementing
> > this integration. I'd greatly appreciate any feedback on this even
> though I
> > still have it marked as draft. I want to make sure I'm heading in the
> right
> > direction here so input from solr dev community would be extremely
> valuable :-)
> > >
> > > Many thanks,
> > > Luke
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: solr query alerting

Posted by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net>.

Houston,

I really appreciate the offer to take a look. I would be happy to raise another PR to the sandbox repo if you think that this will help get some community usage/feedback before it is polished enough to be merged in the main project.

David,

I totally agree with the naming. Speaking from experience, searching for "solr lucene monitor" (and any variation thereof) on the Internet was a real pain so not sure why I wanted to pass that hardship on to anyone else! In all seriousness, I wanted to be consistent with the naming of the lucene feature that this "mapped to". Clearly, from the feedback received so far, this is not a good enough reason to cascade such a poor name.

Now regarding the sandbox/solr.cool point.. I'll start by saying you probably have the best intuition regarding what is useful enough to meet the threshold of being maintained in the core project. If this doesn't meet that threshold then there is not much more I can say. Having said that, I have some skepticism regarding this project being useful outside of solr (even with a link in solr.cool or the like). Take for example this solr.cool-advertised one https://github.com/SOLR4189/solcolator which implements saved search in a less integrated way (and IMO in a less robust way). It hasn't been touched in four years and even if it were, any user deciding on developing a saved search/alerting platform will trust such a project less than if it were maintained in the core project. This hurts adoption and makes it more likely to be relegated to the "attic".

The other reasons I can think of merging this idea into the core project are 1. Elastic supports this feature as part of its core offering and 2. (as Houston stated) this is a feature that is supported in the core lucene project. This makes me think it has enough traction (or it did at some point at least).

I'll also add that I am currently trying to convince folks at my company to adopt this approach so hopefully I'll soon have a more concrete sense about the potential here. Bloomberg is a big user of solr and also has a large footprint of custom-built, saved-search platforms, some of which painfully depend on solr without reaping any of the benefits regarding its distributed state management.

Luke

From: dev@solr.apache.org At: 04/18/24 10:50:23 UTC-4:00To:  dev@solr.apache.org
Subject: Re: solr query alerting

Thanks for the work here Luke!

I tried to work on something similar years ago and it was far more work
than I initially thought to do it right, so I'm so glad someone is coming
and giving it a go finally!

I'll be able to do some review in a few weeks, but will be on vacation
myself for the next week.

As per the Solr sandbox or http://solr.cool, I think it's fine to add this
in the sandbox for now, but given that this is making a Lucene
functionality available in Solr, I see no reason why it can't eventually
reside in the project.
There are issues with maintenance burden, but that burden exists whether it
is in Solr or outside of it.
So given that this is a very cool feature, that I have heard lots of
requests for, I think it's worth having inside.

- Houston

On Thu, Apr 18, 2024 at 9:10 AM David Smiley <ds...@apache.org> wrote:

> I'm probably not a great reviewer of this TBH as I have only heard &
> read about lucene-monitor/percolator; haven't used such a thing yet.
>
> I did a *quick* review to see how the PR touches the codebase.  It's
> wonderful that it is purely isolated; it requires no changes of Solr
> itself.  I might argue why is it here vs the Solr sandbox or somewhere
> else with publicity in http://solr.cool
>
> The name "Monitor" sounds like an infrastructure monitoring feature.
>
> p.s. I'm on vacation this week with minimal time to collaborate
>
> On Mon, Apr 15, 2024 at 10:27 AM Luke Kot-Zaniewski (BLOOMBERG/ 919
> 3RD A) <lk...@bloomberg.net> wrote:
> >
> > Hey David,
> >
> > Just wanted to bump this. I appreciate you taking a look, and wanted to
> stress that I am looking for even just higher level feedback at this point.
> >
> > Basically, I am questioning the direction I am going in which involves
> exposing some of the lucene-monitor internals (so currently have an
> implicit dependence on a lucene change). OTOH I couldn't figure out a
> better way to apply the lucene-monitor optimizations while still utilizing
> solr's sophisticated index management and scaling. Lucene-monitor is a very
> sealed interface and the way it manages the index and caching (via Monitor
> interface) is not easy to integrate on its own.
> >
> > Again, I'd appreciate any feedback, even partial, that you may have!
> >
> > Thanks,
> > Luke
> >
> > From: dev@solr.apache.org At: 04/08/24 13:43:52 UTC-4:00To:
> dev@solr.apache.org
> > Subject: Re: solr query alerting
> >
> > I'm so glad someone has started this!  Thanks for contributing.  I'll
> > take a look
> >
> > On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> > A) <lk...@bloomberg.net> wrote:
> > >
> > > Hi All,
> > >
> > > A few months ago I wrote the user list about potentially integrating
> lucene
> > monitor into solr. I have raised this PR with a first attempt at
> implementing
> > this integration. I'd greatly appreciate any feedback on this even
> though I
> > still have it marked as draft. I want to make sure I'm heading in the
> right
> > direction here so input from solr dev community would be extremely
> valuable :-)
> > >
> > > Many thanks,
> > > Luke
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: solr query alerting

Posted by Houston Putman <ho...@apache.org>.

Thanks for the work here Luke!

I tried to work on something similar years ago and it was far more work
than I initially thought to do it right, so I'm so glad someone is coming
and giving it a go finally!

I'll be able to do some review in a few weeks, but will be on vacation
myself for the next week.

As per the Solr sandbox or http://solr.cool, I think it's fine to add this
in the sandbox for now, but given that this is making a Lucene
functionality available in Solr, I see no reason why it can't eventually
reside in the project.
There are issues with maintenance burden, but that burden exists whether it
is in Solr or outside of it.
So given that this is a very cool feature, that I have heard lots of
requests for, I think it's worth having inside.

- Houston

On Thu, Apr 18, 2024 at 9:10 AM David Smiley <ds...@apache.org> wrote:

> I'm probably not a great reviewer of this TBH as I have only heard &
> read about lucene-monitor/percolator; haven't used such a thing yet.
>
> I did a *quick* review to see how the PR touches the codebase.  It's
> wonderful that it is purely isolated; it requires no changes of Solr
> itself.  I might argue why is it here vs the Solr sandbox or somewhere
> else with publicity in http://solr.cool
>
> The name "Monitor" sounds like an infrastructure monitoring feature.
>
> p.s. I'm on vacation this week with minimal time to collaborate
>
> On Mon, Apr 15, 2024 at 10:27 AM Luke Kot-Zaniewski (BLOOMBERG/ 919
> 3RD A) <lk...@bloomberg.net> wrote:
> >
> > Hey David,
> >
> > Just wanted to bump this. I appreciate you taking a look, and wanted to
> stress that I am looking for even just higher level feedback at this point.
> >
> > Basically, I am questioning the direction I am going in which involves
> exposing some of the lucene-monitor internals (so currently have an
> implicit dependence on a lucene change). OTOH I couldn't figure out a
> better way to apply the lucene-monitor optimizations while still utilizing
> solr's sophisticated index management and scaling. Lucene-monitor is a very
> sealed interface and the way it manages the index and caching (via Monitor
> interface) is not easy to integrate on its own.
> >
> > Again, I'd appreciate any feedback, even partial, that you may have!
> >
> > Thanks,
> > Luke
> >
> > From: dev@solr.apache.org At: 04/08/24 13:43:52 UTC-4:00To:
> dev@solr.apache.org
> > Subject: Re: solr query alerting
> >
> > I'm so glad someone has started this!  Thanks for contributing.  I'll
> > take a look
> >
> > On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> > A) <lk...@bloomberg.net> wrote:
> > >
> > > Hi All,
> > >
> > > A few months ago I wrote the user list about potentially integrating
> lucene
> > monitor into solr. I have raised this PR with a first attempt at
> implementing
> > this integration. I'd greatly appreciate any feedback on this even
> though I
> > still have it marked as draft. I want to make sure I'm heading in the
> right
> > direction here so input from solr dev community would be extremely
> valuable :-)
> > >
> > > Many thanks,
> > > Luke
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: solr query alerting

Posted by David Smiley <ds...@apache.org>.

I'm probably not a great reviewer of this TBH as I have only heard &
read about lucene-monitor/percolator; haven't used such a thing yet.

I did a *quick* review to see how the PR touches the codebase.  It's
wonderful that it is purely isolated; it requires no changes of Solr
itself.  I might argue why is it here vs the Solr sandbox or somewhere
else with publicity in http://solr.cool

The name "Monitor" sounds like an infrastructure monitoring feature.

p.s. I'm on vacation this week with minimal time to collaborate

On Mon, Apr 15, 2024 at 10:27 AM Luke Kot-Zaniewski (BLOOMBERG/ 919
3RD A) <lk...@bloomberg.net> wrote:
>
> Hey David,
>
> Just wanted to bump this. I appreciate you taking a look, and wanted to stress that I am looking for even just higher level feedback at this point.
>
> Basically, I am questioning the direction I am going in which involves exposing some of the lucene-monitor internals (so currently have an implicit dependence on a lucene change). OTOH I couldn't figure out a better way to apply the lucene-monitor optimizations while still utilizing solr's sophisticated index management and scaling. Lucene-monitor is a very sealed interface and the way it manages the index and caching (via Monitor interface) is not easy to integrate on its own.
>
> Again, I'd appreciate any feedback, even partial, that you may have!
>
> Thanks,
> Luke
>
> From: dev@solr.apache.org At: 04/08/24 13:43:52 UTC-4:00To:  dev@solr.apache.org
> Subject: Re: solr query alerting
>
> I'm so glad someone has started this!  Thanks for contributing.  I'll
> take a look
>
> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> A) <lk...@bloomberg.net> wrote:
> >
> > Hi All,
> >
> > A few months ago I wrote the user list about potentially integrating lucene
> monitor into solr. I have raised this PR with a first attempt at implementing
> this integration. I'd greatly appreciate any feedback on this even though I
> still have it marked as draft. I want to make sure I'm heading in the right
> direction here so input from solr dev community would be extremely valuable :-)
> >
> > Many thanks,
> > Luke
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: solr query alerting

Posted by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net>.

Hey David,

Just wanted to bump this. I appreciate you taking a look, and wanted to stress that I am looking for even just higher level feedback at this point. 

Basically, I am questioning the direction I am going in which involves exposing some of the lucene-monitor internals (so currently have an implicit dependence on a lucene change). OTOH I couldn't figure out a better way to apply the lucene-monitor optimizations while still utilizing solr's sophisticated index management and scaling. Lucene-monitor is a very sealed interface and the way it manages the index and caching (via Monitor interface) is not easy to integrate on its own.

Again, I'd appreciate any feedback, even partial, that you may have!

Thanks,
Luke

From: dev@solr.apache.org At: 04/08/24 13:43:52 UTC-4:00To:  dev@solr.apache.org
Subject: Re: solr query alerting

I'm so glad someone has started this!  Thanks for contributing.  I'll
take a look

On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
A) <lk...@bloomberg.net> wrote:
>
> Hi All,
>
> A few months ago I wrote the user list about potentially integrating lucene 
monitor into solr. I have raised this PR with a first attempt at implementing 
this integration. I'd greatly appreciate any feedback on this even though I 
still have it marked as draft. I want to make sure I'm heading in the right 
direction here so input from solr dev community would be extremely valuable :-)
>
> Many thanks,
> Luke

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: solr query alerting

Posted by David Smiley <ds...@apache.org>.

I'm so glad someone has started this!  Thanks for contributing.  I'll
take a look

On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
A) <lk...@bloomberg.net> wrote:
>
> Hi All,
>
> A few months ago I wrote the user list about potentially integrating lucene monitor into solr. I have raised this PR with a first attempt at implementing this integration. I'd greatly appreciate any feedback on this even though I still have it marked as draft. I want to make sure I'm heading in the right direction here so input from solr dev community would be extremely valuable :-)
>
> Many thanks,
> Luke

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: solr query alerting

Posted by Walter Underwood <wu...@wunderwood.org>.

Do people want to spend the next ten years explaining that the the alerting feature is called “Luwak”? I’d call it “Alerting” or “Alerts".  —wonder

> On May 1, 2024, at 1:16 PM, Houston Putman <ho...@apache.org> wrote:
> 
> I think I'd prefer a more self-descriptive name than "Luwak", which is just
> a product name that was decided a while ago.
> 
> I kind of like "search-alerts". "query-alerts" sounds like alerting on
> query metrics, but IMO "search-alerts" doesn't come with the same baggage.
> 
> Luwak is fine though if everyone agrees on that.
> 
> On one hand we have a number of committers here from
>> Bloomberg, yet the abandoned and now-removed "analytics" component
>> shows that abandonment is a risk nonetheless.
>> 
> 
> I don't want to bikeshed here, but I'm not sure this is a fair
> assessment of what happened with the analytics module.
> Sure there wasn't a ton of development, but in general it was feature rich
> and had very little feature requests.
> It was removed in 10, because a lack of user usage, not because it was
> "abandoned" IMO. If there were requests from users
> to keep it or improve it, then it would be a much different story. The
> whole "thrown over the wall" comment is fair, but
> not particularly relevant to this PR, which is being worked on in public.
> 
> I think this is certainly something that can start in the sandbox and move
> into the main repo once it's clear that there is interest from
> multiple committers and community members in using and maintaining it.
> 
> - Houston
> 
> On Wed, May 1, 2024 at 2:32 PM David Smiley <ds...@apache.org> wrote:
> 
>> Luwak is good to me!
>> 
>> On Tue, Apr 30, 2024 at 4:01 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
>> A) <lk...@bloomberg.net> wrote:
>>> 
>>> I love the name "luwak"! I was about to suggest the same but was worried
>> about the trademark concerns and I assumed there was a reason they changed
>> the name when donating it to lucene.
>>> 
>>> From: dev@solr.apache.org At: 04/30/24 15:56:22 UTC-4:00To:
>> dev@solr.apache.org
>>> Subject: Re: solr query alerting
>>> 
>>> Luwak is the original name of the Lucene monitor, contributed by Flax
>> back in
>>> the days: https://github.com/flaxsearch/luwak
>>> 
>>> Perhaps we could go full circle (if no trademark issues) to call it the
>> Solr
>>> luwak module? Luwak is a type of coffee, and thus related to percolator
>> 😉
>>> 
>>> Otherwise “stored-queries” is an option.
>>> 
>>> Jan Høydahl
>>> 
>>>> 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
>>>> 
>>>> I agree the feature is relevant / useful.
>>>> 
>>>> Another angle on the module vs sandbox or wherever else is maintenance
>>>> cost.  If a lot of code is being contributed as is here, then as a PMC
>>>> member I hope to get a subjective sense that folks are interested in
>>>> maintaining it.  On one hand we have a number of committers here from
>>>> Bloomberg, yet the abandoned and now-removed "analytics" component
>>>> shows that abandonment is a risk nonetheless.  I don't know how to
>>>> conclude this thought but I'm hoping to hear from folks that they
>>>> intend to look after this module.  It's not just being "thrown over
>>>> the wall", so to speak.
>>>> 
>>>> Naming is hard...
>>>> * ...-monitor-....: sorry I hate it
>>>> * ...-percolator-.... No clue why this was chosen for ElasticSearch.
>>>> I can appreciate a curious/non-obvious name like this that is not
>>>> going to conflict with anyone's guesses at what a general name might
>>>> convey.
>>>> * "indexed-queries" or "query-indexing" would be a good name?  This is
>>>> the best technical name I can think of.
>>>> *  "reverse search" came to mind (based on the Netflix article)
>>>> although that makes me think of leading-wildcard / suffix-search.
>>>> * "inverted-search"
>>>> *  "indexed-query-alerts" incorporates "alerts" thus might better
>>>> convey the use-case
>>>> 
>>>>> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
>>>>> A) <lk...@bloomberg.net> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> A few months ago I wrote the user list about potentially integrating
>> lucene
>>> monitor into solr. I have raised this PR with a first attempt at
>> implementing
>>> this integration. I'd greatly appreciate any feedback on this even
>> though I
>>> still have it marked as draft. I want to make sure I'm heading in the
>> right
>>> direction here so input from solr dev community would be extremely
>> valuable :-)
>>>>> 
>>>>> Many thanks,
>>>>> Luke
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>> 
>>> 
>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: solr query alerting

Posted by Mikhail Khludnev <mk...@apache.org>.

May I propose Query Index?

On Thu, May 2, 2024 at 3:03 PM Gus Heck <gu...@gmail.com> wrote:

> It detects documents that match a set of queries, so maybe "detector"?
>
> On Thu, May 2, 2024 at 4:30 AM Jan Høydahl <ja...@cominvent.com> wrote:
>
> > Alerting, while overloaded is probably the most precise name we could
> > choose - documentation would help explain the scope.
> > And if someone made an example project with a UI for experimentation that
> > would make the feature much more approachable.
> >
> > Jan
> >
> > > 2. mai 2024 kl. 03:18 skrev Walter Underwood <wu...@wunderwood.org>:
> > >
> > > The functionality is alerts, but that doesn’t mean it has to be a push
> > API. Alerts can be fetched just as easily as pushed.
> > >
> > > I don’t know the limits of this proposal, but LexisNexis needs alerting
> > as we move all of our 114 billion documents onto Solr. I’m retiring this
> > week, so I won’t be around to implement it, but that is one potential
> large
> > customer.
> > >
> > > wunder
> > > Walter Underwood
> > > wunder@wunderwood.org
> > > http://observer.wunderwood.org/  (my blog)
> > >
> > >> On May 1, 2024, at 2:26 PM, Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)
> <
> > lkotzaniewsk@bloomberg.net> wrote:
> > >>
> > >>> I kind of like "search-alerts". "query-alerts" sounds like alerting
> on
> > >>> query metrics, but IMO "search-alerts" doesn't come with the same
> > baggage.
> > >>
> > >> Someone in the PR had mentioned that "alerts" is a bit off because the
> > proposal does not really manage alerts and it feels too far out of solr's
> > domain. The current approach, much like percolator, simply exposes a
> > request/response API that then can be **used** by an alerting system
> > (request/stream<response> could also be considered if there is worry
> about
> > scaling the number of queries one request can match).
> > >>
> > >>> I think this is certainly something that can start in the sandbox and
> > move> into the main repo once it's clear that there is interest from
> > >>> multiple committers and community members in using and maintaining
> it.
> > >>
> > >> I've seen many homegrown/complex solutions of percolator-type
> > functionality so even this narrower "inverted search" solution has
> **some**
> > use but admittedly this is a niche area. It might not really gain
> traction
> > unless it is marketed the right way as there are probably very few solr
> > users that happen to be thinking about revamping their saved-search
> > platform in any given year. Given that, what do you think I can do to
> reach
> > them? :-)
> > >>
> > >> I am trying my best to talk about this within my firm but the sample
> is
> > obviously smaller.
> > >>
> > >> From: dev@solr.apache.org At: 05/01/24 16:16:50 UTC-4:00To:
> > dev@solr.apache.org
> > >> Subject: Re: solr query alerting
> > >>
> > >> I think I'd prefer a more self-descriptive name than "Luwak", which is
> > just
> > >> a product name that was decided a while ago.
> > >>
> > >> I kind of like "search-alerts". "query-alerts" sounds like alerting on
> > >> query metrics, but IMO "search-alerts" doesn't come with the same
> > baggage.
> > >>
> > >> Luwak is fine though if everyone agrees on that.
> > >>
> > >> On one hand we have a number of committers here from
> > >>> Bloomberg, yet the abandoned and now-removed "analytics" component
> > >>> shows that abandonment is a risk nonetheless.
> > >>>
> > >>
> > >> I don't want to bikeshed here, but I'm not sure this is a fair
> > >> assessment of what happened with the analytics module.
> > >> Sure there wasn't a ton of development, but in general it was feature
> > rich
> > >> and had very little feature requests.
> > >> It was removed in 10, because a lack of user usage, not because it was
> > >> "abandoned" IMO. If there were requests from users
> > >> to keep it or improve it, then it would be a much different story. The
> > >> whole "thrown over the wall" comment is fair, but
> > >> not particularly relevant to this PR, which is being worked on in
> > public.
> > >>
> > >> I think this is certainly something that can start in the sandbox and
> > move
> > >> into the main repo once it's clear that there is interest from
> > >> multiple committers and community members in using and maintaining it.
> > >>
> > >> - Houston
> > >>
> > >> On Wed, May 1, 2024 at 2:32 PM David Smiley <ds...@apache.org>
> wrote:
> > >>
> > >>> Luwak is good to me!
> > >>>
> > >>> On Tue, Apr 30, 2024 at 4:01 PM Luke Kot-Zaniewski (BLOOMBERG/ 919
> 3RD
> > >>> A) <lk...@bloomberg.net> wrote:
> > >>>>
> > >>>> I love the name "luwak"! I was about to suggest the same but was
> > worried
> > >>> about the trademark concerns and I assumed there was a reason they
> > changed
> > >>> the name when donating it to lucene.
> > >>>>
> > >>>> From: dev@solr.apache.org At: 04/30/24 15:56:22 UTC-4:00To:
> > >>> dev@solr.apache.org
> > >>>> Subject: Re: solr query alerting
> > >>>>
> > >>>> Luwak is the original name of the Lucene monitor, contributed by
> Flax
> > >>> back in
> > >>>> the days: https://github.com/flaxsearch/luwak
> > >>>>
> > >>>> Perhaps we could go full circle (if no trademark issues) to call it
> > the
> > >>> Solr
> > >>>> luwak module? Luwak is a type of coffee, and thus related to
> > percolator
> > >>> 😉
> > >>>>
> > >>>> Otherwise “stored-queries” is an option.
> > >>>>
> > >>>> Jan Høydahl
> > >>>>
> > >>>>> 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
> > >>>>>
> > >>>>> I agree the feature is relevant / useful.
> > >>>>>
> > >>>>> Another angle on the module vs sandbox or wherever else is
> > maintenance
> > >>>>> cost.  If a lot of code is being contributed as is here, then as a
> > PMC
> > >>>>> member I hope to get a subjective sense that folks are interested
> in
> > >>>>> maintaining it.  On one hand we have a number of committers here
> from
> > >>>>> Bloomberg, yet the abandoned and now-removed "analytics" component
> > >>>>> shows that abandonment is a risk nonetheless.  I don't know how to
> > >>>>> conclude this thought but I'm hoping to hear from folks that they
> > >>>>> intend to look after this module.  It's not just being "thrown over
> > >>>>> the wall", so to speak.
> > >>>>>
> > >>>>> Naming is hard...
> > >>>>> * ...-monitor-....: sorry I hate it
> > >>>>> * ...-percolator-.... No clue why this was chosen for
> ElasticSearch.
> > >>>>> I can appreciate a curious/non-obvious name like this that is not
> > >>>>> going to conflict with anyone's guesses at what a general name
> might
> > >>>>> convey.
> > >>>>> * "indexed-queries" or "query-indexing" would be a good name?  This
> > is
> > >>>>> the best technical name I can think of.
> > >>>>> *  "reverse search" came to mind (based on the Netflix article)
> > >>>>> although that makes me think of leading-wildcard / suffix-search.
> > >>>>> * "inverted-search"
> > >>>>> *  "indexed-query-alerts" incorporates "alerts" thus might better
> > >>>>> convey the use-case
> > >>>>>
> > >>>>>> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919
> > 3RD
> > >>>>>> A) <lk...@bloomberg.net> wrote:
> > >>>>>>
> > >>>>>> Hi All,
> > >>>>>>
> > >>>>>> A few months ago I wrote the user list about potentially
> integrating
> > >>> lucene
> > >>>> monitor into solr. I have raised this PR with a first attempt at
> > >>> implementing
> > >>>> this integration. I'd greatly appreciate any feedback on this even
> > >>> though I
> > >>>> still have it marked as draft. I want to make sure I'm heading in
> the
> > >>> right
> > >>>> direction here so input from solr dev community would be extremely
> > >>> valuable :-)
> > >>>>>>
> > >>>>>> Many thanks,
> > >>>>>> Luke
> > >>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >>>>> For additional commands, e-mail: dev-help@solr.apache.org
> > >>>>>
> > >>>>
> > >>>>
> > >>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >>> For additional commands, e-mail: dev-help@solr.apache.org
> > >>>
> > >>>
> > >>
> > >>
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
> >
>
> --
> http://www.needhamsoftware.com (work)
> https://a.co/d/b2sZLD9 (my fantasy fiction book)
>


-- 
Sincerely yours
Mikhail Khludnev

Re: solr query alerting

Posted by Gus Heck <gu...@gmail.com>.

It detects documents that match a set of queries, so maybe "detector"?

On Thu, May 2, 2024 at 4:30 AM Jan Høydahl <ja...@cominvent.com> wrote:

> Alerting, while overloaded is probably the most precise name we could
> choose - documentation would help explain the scope.
> And if someone made an example project with a UI for experimentation that
> would make the feature much more approachable.
>
> Jan
>
> > 2. mai 2024 kl. 03:18 skrev Walter Underwood <wu...@wunderwood.org>:
> >
> > The functionality is alerts, but that doesn’t mean it has to be a push
> API. Alerts can be fetched just as easily as pushed.
> >
> > I don’t know the limits of this proposal, but LexisNexis needs alerting
> as we move all of our 114 billion documents onto Solr. I’m retiring this
> week, so I won’t be around to implement it, but that is one potential large
> customer.
> >
> > wunder
> > Walter Underwood
> > wunder@wunderwood.org
> > http://observer.wunderwood.org/  (my blog)
> >
> >> On May 1, 2024, at 2:26 PM, Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <
> lkotzaniewsk@bloomberg.net> wrote:
> >>
> >>> I kind of like "search-alerts". "query-alerts" sounds like alerting on
> >>> query metrics, but IMO "search-alerts" doesn't come with the same
> baggage.
> >>
> >> Someone in the PR had mentioned that "alerts" is a bit off because the
> proposal does not really manage alerts and it feels too far out of solr's
> domain. The current approach, much like percolator, simply exposes a
> request/response API that then can be **used** by an alerting system
> (request/stream<response> could also be considered if there is worry about
> scaling the number of queries one request can match).
> >>
> >>> I think this is certainly something that can start in the sandbox and
> move> into the main repo once it's clear that there is interest from
> >>> multiple committers and community members in using and maintaining it.
> >>
> >> I've seen many homegrown/complex solutions of percolator-type
> functionality so even this narrower "inverted search" solution has **some**
> use but admittedly this is a niche area. It might not really gain traction
> unless it is marketed the right way as there are probably very few solr
> users that happen to be thinking about revamping their saved-search
> platform in any given year. Given that, what do you think I can do to reach
> them? :-)
> >>
> >> I am trying my best to talk about this within my firm but the sample is
> obviously smaller.
> >>
> >> From: dev@solr.apache.org At: 05/01/24 16:16:50 UTC-4:00To:
> dev@solr.apache.org
> >> Subject: Re: solr query alerting
> >>
> >> I think I'd prefer a more self-descriptive name than "Luwak", which is
> just
> >> a product name that was decided a while ago.
> >>
> >> I kind of like "search-alerts". "query-alerts" sounds like alerting on
> >> query metrics, but IMO "search-alerts" doesn't come with the same
> baggage.
> >>
> >> Luwak is fine though if everyone agrees on that.
> >>
> >> On one hand we have a number of committers here from
> >>> Bloomberg, yet the abandoned and now-removed "analytics" component
> >>> shows that abandonment is a risk nonetheless.
> >>>
> >>
> >> I don't want to bikeshed here, but I'm not sure this is a fair
> >> assessment of what happened with the analytics module.
> >> Sure there wasn't a ton of development, but in general it was feature
> rich
> >> and had very little feature requests.
> >> It was removed in 10, because a lack of user usage, not because it was
> >> "abandoned" IMO. If there were requests from users
> >> to keep it or improve it, then it would be a much different story. The
> >> whole "thrown over the wall" comment is fair, but
> >> not particularly relevant to this PR, which is being worked on in
> public.
> >>
> >> I think this is certainly something that can start in the sandbox and
> move
> >> into the main repo once it's clear that there is interest from
> >> multiple committers and community members in using and maintaining it.
> >>
> >> - Houston
> >>
> >> On Wed, May 1, 2024 at 2:32 PM David Smiley <ds...@apache.org> wrote:
> >>
> >>> Luwak is good to me!
> >>>
> >>> On Tue, Apr 30, 2024 at 4:01 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> >>> A) <lk...@bloomberg.net> wrote:
> >>>>
> >>>> I love the name "luwak"! I was about to suggest the same but was
> worried
> >>> about the trademark concerns and I assumed there was a reason they
> changed
> >>> the name when donating it to lucene.
> >>>>
> >>>> From: dev@solr.apache.org At: 04/30/24 15:56:22 UTC-4:00To:
> >>> dev@solr.apache.org
> >>>> Subject: Re: solr query alerting
> >>>>
> >>>> Luwak is the original name of the Lucene monitor, contributed by Flax
> >>> back in
> >>>> the days: https://github.com/flaxsearch/luwak
> >>>>
> >>>> Perhaps we could go full circle (if no trademark issues) to call it
> the
> >>> Solr
> >>>> luwak module? Luwak is a type of coffee, and thus related to
> percolator
> >>> 😉
> >>>>
> >>>> Otherwise “stored-queries” is an option.
> >>>>
> >>>> Jan Høydahl
> >>>>
> >>>>> 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
> >>>>>
> >>>>> I agree the feature is relevant / useful.
> >>>>>
> >>>>> Another angle on the module vs sandbox or wherever else is
> maintenance
> >>>>> cost.  If a lot of code is being contributed as is here, then as a
> PMC
> >>>>> member I hope to get a subjective sense that folks are interested in
> >>>>> maintaining it.  On one hand we have a number of committers here from
> >>>>> Bloomberg, yet the abandoned and now-removed "analytics" component
> >>>>> shows that abandonment is a risk nonetheless.  I don't know how to
> >>>>> conclude this thought but I'm hoping to hear from folks that they
> >>>>> intend to look after this module.  It's not just being "thrown over
> >>>>> the wall", so to speak.
> >>>>>
> >>>>> Naming is hard...
> >>>>> * ...-monitor-....: sorry I hate it
> >>>>> * ...-percolator-.... No clue why this was chosen for ElasticSearch.
> >>>>> I can appreciate a curious/non-obvious name like this that is not
> >>>>> going to conflict with anyone's guesses at what a general name might
> >>>>> convey.
> >>>>> * "indexed-queries" or "query-indexing" would be a good name?  This
> is
> >>>>> the best technical name I can think of.
> >>>>> *  "reverse search" came to mind (based on the Netflix article)
> >>>>> although that makes me think of leading-wildcard / suffix-search.
> >>>>> * "inverted-search"
> >>>>> *  "indexed-query-alerts" incorporates "alerts" thus might better
> >>>>> convey the use-case
> >>>>>
> >>>>>> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919
> 3RD
> >>>>>> A) <lk...@bloomberg.net> wrote:
> >>>>>>
> >>>>>> Hi All,
> >>>>>>
> >>>>>> A few months ago I wrote the user list about potentially integrating
> >>> lucene
> >>>> monitor into solr. I have raised this PR with a first attempt at
> >>> implementing
> >>>> this integration. I'd greatly appreciate any feedback on this even
> >>> though I
> >>>> still have it marked as draft. I want to make sure I'm heading in the
> >>> right
> >>>> direction here so input from solr dev community would be extremely
> >>> valuable :-)
> >>>>>>
> >>>>>> Many thanks,
> >>>>>> Luke
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>>>> For additional commands, e-mail: dev-help@solr.apache.org
> >>>>>
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>> For additional commands, e-mail: dev-help@solr.apache.org
> >>>
> >>>
> >>
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

-- 
http://www.needhamsoftware.com (work)
https://a.co/d/b2sZLD9 (my fantasy fiction book)

Re: solr query alerting

Posted by Jan Høydahl <ja...@cominvent.com>.

Alerting, while overloaded is probably the most precise name we could choose - documentation would help explain the scope.
And if someone made an example project with a UI for experimentation that would make the feature much more approachable.

Jan

> 2. mai 2024 kl. 03:18 skrev Walter Underwood <wu...@wunderwood.org>:
> 
> The functionality is alerts, but that doesn’t mean it has to be a push API. Alerts can be fetched just as easily as pushed.
> 
> I don’t know the limits of this proposal, but LexisNexis needs alerting as we move all of our 114 billion documents onto Solr. I’m retiring this week, so I won’t be around to implement it, but that is one potential large customer.
> 
> wunder
> Walter Underwood
> wunder@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
> 
>> On May 1, 2024, at 2:26 PM, Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <lk...@bloomberg.net> wrote:
>> 
>>> I kind of like "search-alerts". "query-alerts" sounds like alerting on
>>> query metrics, but IMO "search-alerts" doesn't come with the same baggage.
>> 
>> Someone in the PR had mentioned that "alerts" is a bit off because the proposal does not really manage alerts and it feels too far out of solr's domain. The current approach, much like percolator, simply exposes a request/response API that then can be **used** by an alerting system (request/stream<response> could also be considered if there is worry about scaling the number of queries one request can match). 
>> 
>>> I think this is certainly something that can start in the sandbox and move> into the main repo once it's clear that there is interest from
>>> multiple committers and community members in using and maintaining it.
>> 
>> I've seen many homegrown/complex solutions of percolator-type functionality so even this narrower "inverted search" solution has **some** use but admittedly this is a niche area. It might not really gain traction unless it is marketed the right way as there are probably very few solr users that happen to be thinking about revamping their saved-search platform in any given year. Given that, what do you think I can do to reach them? :-)
>> 
>> I am trying my best to talk about this within my firm but the sample is obviously smaller.
>> 
>> From: dev@solr.apache.org At: 05/01/24 16:16:50 UTC-4:00To:  dev@solr.apache.org
>> Subject: Re: solr query alerting
>> 
>> I think I'd prefer a more self-descriptive name than "Luwak", which is just
>> a product name that was decided a while ago.
>> 
>> I kind of like "search-alerts". "query-alerts" sounds like alerting on
>> query metrics, but IMO "search-alerts" doesn't come with the same baggage.
>> 
>> Luwak is fine though if everyone agrees on that.
>> 
>> On one hand we have a number of committers here from
>>> Bloomberg, yet the abandoned and now-removed "analytics" component
>>> shows that abandonment is a risk nonetheless.
>>> 
>> 
>> I don't want to bikeshed here, but I'm not sure this is a fair
>> assessment of what happened with the analytics module.
>> Sure there wasn't a ton of development, but in general it was feature rich
>> and had very little feature requests.
>> It was removed in 10, because a lack of user usage, not because it was
>> "abandoned" IMO. If there were requests from users
>> to keep it or improve it, then it would be a much different story. The
>> whole "thrown over the wall" comment is fair, but
>> not particularly relevant to this PR, which is being worked on in public.
>> 
>> I think this is certainly something that can start in the sandbox and move
>> into the main repo once it's clear that there is interest from
>> multiple committers and community members in using and maintaining it.
>> 
>> - Houston
>> 
>> On Wed, May 1, 2024 at 2:32 PM David Smiley <ds...@apache.org> wrote:
>> 
>>> Luwak is good to me!
>>> 
>>> On Tue, Apr 30, 2024 at 4:01 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
>>> A) <lk...@bloomberg.net> wrote:
>>>> 
>>>> I love the name "luwak"! I was about to suggest the same but was worried
>>> about the trademark concerns and I assumed there was a reason they changed
>>> the name when donating it to lucene.
>>>> 
>>>> From: dev@solr.apache.org At: 04/30/24 15:56:22 UTC-4:00To:
>>> dev@solr.apache.org
>>>> Subject: Re: solr query alerting
>>>> 
>>>> Luwak is the original name of the Lucene monitor, contributed by Flax
>>> back in
>>>> the days: https://github.com/flaxsearch/luwak
>>>> 
>>>> Perhaps we could go full circle (if no trademark issues) to call it the
>>> Solr
>>>> luwak module? Luwak is a type of coffee, and thus related to percolator
>>> 😉
>>>> 
>>>> Otherwise “stored-queries” is an option.
>>>> 
>>>> Jan Høydahl
>>>> 
>>>>> 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
>>>>> 
>>>>> I agree the feature is relevant / useful.
>>>>> 
>>>>> Another angle on the module vs sandbox or wherever else is maintenance
>>>>> cost.  If a lot of code is being contributed as is here, then as a PMC
>>>>> member I hope to get a subjective sense that folks are interested in
>>>>> maintaining it.  On one hand we have a number of committers here from
>>>>> Bloomberg, yet the abandoned and now-removed "analytics" component
>>>>> shows that abandonment is a risk nonetheless.  I don't know how to
>>>>> conclude this thought but I'm hoping to hear from folks that they
>>>>> intend to look after this module.  It's not just being "thrown over
>>>>> the wall", so to speak.
>>>>> 
>>>>> Naming is hard...
>>>>> * ...-monitor-....: sorry I hate it
>>>>> * ...-percolator-.... No clue why this was chosen for ElasticSearch.
>>>>> I can appreciate a curious/non-obvious name like this that is not
>>>>> going to conflict with anyone's guesses at what a general name might
>>>>> convey.
>>>>> * "indexed-queries" or "query-indexing" would be a good name?  This is
>>>>> the best technical name I can think of.
>>>>> *  "reverse search" came to mind (based on the Netflix article)
>>>>> although that makes me think of leading-wildcard / suffix-search.
>>>>> * "inverted-search"
>>>>> *  "indexed-query-alerts" incorporates "alerts" thus might better
>>>>> convey the use-case
>>>>> 
>>>>>> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
>>>>>> A) <lk...@bloomberg.net> wrote:
>>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> A few months ago I wrote the user list about potentially integrating
>>> lucene
>>>> monitor into solr. I have raised this PR with a first attempt at
>>> implementing
>>>> this integration. I'd greatly appreciate any feedback on this even
>>> though I
>>>> still have it marked as draft. I want to make sure I'm heading in the
>>> right
>>>> direction here so input from solr dev community would be extremely
>>> valuable :-)
>>>>>> 
>>>>>> Many thanks,
>>>>>> Luke
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>>> 
>>>> 
>>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> For additional commands, e-mail: dev-help@solr.apache.org
>>> 
>>> 
>> 
>> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: solr query alerting

Posted by Walter Underwood <wu...@wunderwood.org>.

The functionality is alerts, but that doesn’t mean it has to be a push API. Alerts can be fetched just as easily as pushed.

I don’t know the limits of this proposal, but LexisNexis needs alerting as we move all of our 114 billion documents onto Solr. I’m retiring this week, so I won’t be around to implement it, but that is one potential large customer.

wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On May 1, 2024, at 2:26 PM, Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) <lk...@bloomberg.net> wrote:
> 
>> I kind of like "search-alerts". "query-alerts" sounds like alerting on
>> query metrics, but IMO "search-alerts" doesn't come with the same baggage.
> 
> Someone in the PR had mentioned that "alerts" is a bit off because the proposal does not really manage alerts and it feels too far out of solr's domain. The current approach, much like percolator, simply exposes a request/response API that then can be **used** by an alerting system (request/stream<response> could also be considered if there is worry about scaling the number of queries one request can match). 
> 
>> I think this is certainly something that can start in the sandbox and move> into the main repo once it's clear that there is interest from
>> multiple committers and community members in using and maintaining it.
> 
> I've seen many homegrown/complex solutions of percolator-type functionality so even this narrower "inverted search" solution has **some** use but admittedly this is a niche area. It might not really gain traction unless it is marketed the right way as there are probably very few solr users that happen to be thinking about revamping their saved-search platform in any given year. Given that, what do you think I can do to reach them? :-)
> 
> I am trying my best to talk about this within my firm but the sample is obviously smaller.
> 
> From: dev@solr.apache.org At: 05/01/24 16:16:50 UTC-4:00To:  dev@solr.apache.org
> Subject: Re: solr query alerting
> 
> I think I'd prefer a more self-descriptive name than "Luwak", which is just
> a product name that was decided a while ago.
> 
> I kind of like "search-alerts". "query-alerts" sounds like alerting on
> query metrics, but IMO "search-alerts" doesn't come with the same baggage.
> 
> Luwak is fine though if everyone agrees on that.
> 
> On one hand we have a number of committers here from
>> Bloomberg, yet the abandoned and now-removed "analytics" component
>> shows that abandonment is a risk nonetheless.
>> 
> 
> I don't want to bikeshed here, but I'm not sure this is a fair
> assessment of what happened with the analytics module.
> Sure there wasn't a ton of development, but in general it was feature rich
> and had very little feature requests.
> It was removed in 10, because a lack of user usage, not because it was
> "abandoned" IMO. If there were requests from users
> to keep it or improve it, then it would be a much different story. The
> whole "thrown over the wall" comment is fair, but
> not particularly relevant to this PR, which is being worked on in public.
> 
> I think this is certainly something that can start in the sandbox and move
> into the main repo once it's clear that there is interest from
> multiple committers and community members in using and maintaining it.
> 
> - Houston
> 
> On Wed, May 1, 2024 at 2:32 PM David Smiley <ds...@apache.org> wrote:
> 
>> Luwak is good to me!
>> 
>> On Tue, Apr 30, 2024 at 4:01 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
>> A) <lk...@bloomberg.net> wrote:
>>> 
>>> I love the name "luwak"! I was about to suggest the same but was worried
>> about the trademark concerns and I assumed there was a reason they changed
>> the name when donating it to lucene.
>>> 
>>> From: dev@solr.apache.org At: 04/30/24 15:56:22 UTC-4:00To:
>> dev@solr.apache.org
>>> Subject: Re: solr query alerting
>>> 
>>> Luwak is the original name of the Lucene monitor, contributed by Flax
>> back in
>>> the days: https://github.com/flaxsearch/luwak
>>> 
>>> Perhaps we could go full circle (if no trademark issues) to call it the
>> Solr
>>> luwak module? Luwak is a type of coffee, and thus related to percolator
>> 😉
>>> 
>>> Otherwise “stored-queries” is an option.
>>> 
>>> Jan Høydahl
>>> 
>>>> 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
>>>> 
>>>> I agree the feature is relevant / useful.
>>>> 
>>>> Another angle on the module vs sandbox or wherever else is maintenance
>>>> cost.  If a lot of code is being contributed as is here, then as a PMC
>>>> member I hope to get a subjective sense that folks are interested in
>>>> maintaining it.  On one hand we have a number of committers here from
>>>> Bloomberg, yet the abandoned and now-removed "analytics" component
>>>> shows that abandonment is a risk nonetheless.  I don't know how to
>>>> conclude this thought but I'm hoping to hear from folks that they
>>>> intend to look after this module.  It's not just being "thrown over
>>>> the wall", so to speak.
>>>> 
>>>> Naming is hard...
>>>> * ...-monitor-....: sorry I hate it
>>>> * ...-percolator-.... No clue why this was chosen for ElasticSearch.
>>>> I can appreciate a curious/non-obvious name like this that is not
>>>> going to conflict with anyone's guesses at what a general name might
>>>> convey.
>>>> * "indexed-queries" or "query-indexing" would be a good name?  This is
>>>> the best technical name I can think of.
>>>> *  "reverse search" came to mind (based on the Netflix article)
>>>> although that makes me think of leading-wildcard / suffix-search.
>>>> * "inverted-search"
>>>> *  "indexed-query-alerts" incorporates "alerts" thus might better
>>>> convey the use-case
>>>> 
>>>>> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
>>>>> A) <lk...@bloomberg.net> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> A few months ago I wrote the user list about potentially integrating
>> lucene
>>> monitor into solr. I have raised this PR with a first attempt at
>> implementing
>>> this integration. I'd greatly appreciate any feedback on this even
>> though I
>>> still have it marked as draft. I want to make sure I'm heading in the
>> right
>>> direction here so input from solr dev community would be extremely
>> valuable :-)
>>>>> 
>>>>> Many thanks,
>>>>> Luke
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>> 
>>> 
>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>> 
>> 
> 
>

Re: solr query alerting

Posted by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net>.

> I kind of like "search-alerts". "query-alerts" sounds like alerting on
> query metrics, but IMO "search-alerts" doesn't come with the same baggage.

Someone in the PR had mentioned that "alerts" is a bit off because the proposal does not really manage alerts and it feels too far out of solr's domain. The current approach, much like percolator, simply exposes a request/response API that then can be **used** by an alerting system (request/stream<response> could also be considered if there is worry about scaling the number of queries one request can match). 

> I think this is certainly something that can start in the sandbox and move> into the main repo once it's clear that there is interest from
> multiple committers and community members in using and maintaining it.

I've seen many homegrown/complex solutions of percolator-type functionality so even this narrower "inverted search" solution has **some** use but admittedly this is a niche area. It might not really gain traction unless it is marketed the right way as there are probably very few solr users that happen to be thinking about revamping their saved-search platform in any given year. Given that, what do you think I can do to reach them? :-)

I am trying my best to talk about this within my firm but the sample is obviously smaller.

From: dev@solr.apache.org At: 05/01/24 16:16:50 UTC-4:00To:  dev@solr.apache.org
Subject: Re: solr query alerting

I think I'd prefer a more self-descriptive name than "Luwak", which is just
a product name that was decided a while ago.

I kind of like "search-alerts". "query-alerts" sounds like alerting on
query metrics, but IMO "search-alerts" doesn't come with the same baggage.

Luwak is fine though if everyone agrees on that.

On one hand we have a number of committers here from
> Bloomberg, yet the abandoned and now-removed "analytics" component
> shows that abandonment is a risk nonetheless.
>

I don't want to bikeshed here, but I'm not sure this is a fair
assessment of what happened with the analytics module.
Sure there wasn't a ton of development, but in general it was feature rich
and had very little feature requests.
It was removed in 10, because a lack of user usage, not because it was
"abandoned" IMO. If there were requests from users
to keep it or improve it, then it would be a much different story. The
whole "thrown over the wall" comment is fair, but
not particularly relevant to this PR, which is being worked on in public.

I think this is certainly something that can start in the sandbox and move
into the main repo once it's clear that there is interest from
multiple committers and community members in using and maintaining it.

- Houston

On Wed, May 1, 2024 at 2:32 PM David Smiley <ds...@apache.org> wrote:

> Luwak is good to me!
>
> On Tue, Apr 30, 2024 at 4:01 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> A) <lk...@bloomberg.net> wrote:
> >
> > I love the name "luwak"! I was about to suggest the same but was worried
> about the trademark concerns and I assumed there was a reason they changed
> the name when donating it to lucene.
> >
> > From: dev@solr.apache.org At: 04/30/24 15:56:22 UTC-4:00To:
> dev@solr.apache.org
> > Subject: Re: solr query alerting
> >
> > Luwak is the original name of the Lucene monitor, contributed by Flax
> back in
> > the days: https://github.com/flaxsearch/luwak
> >
> > Perhaps we could go full circle (if no trademark issues) to call it the
> Solr
> > luwak module? Luwak is a type of coffee, and thus related to percolator
> 😉
> >
> > Otherwise “stored-queries” is an option.
> >
> > Jan Høydahl
> >
> > > 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
> > >
> > > I agree the feature is relevant / useful.
> > >
> > > Another angle on the module vs sandbox or wherever else is maintenance
> > > cost.  If a lot of code is being contributed as is here, then as a PMC
> > > member I hope to get a subjective sense that folks are interested in
> > > maintaining it.  On one hand we have a number of committers here from
> > > Bloomberg, yet the abandoned and now-removed "analytics" component
> > > shows that abandonment is a risk nonetheless.  I don't know how to
> > > conclude this thought but I'm hoping to hear from folks that they
> > > intend to look after this module.  It's not just being "thrown over
> > > the wall", so to speak.
> > >
> > > Naming is hard...
> > > * ...-monitor-....: sorry I hate it
> > > * ...-percolator-.... No clue why this was chosen for ElasticSearch.
> > > I can appreciate a curious/non-obvious name like this that is not
> > > going to conflict with anyone's guesses at what a general name might
> > > convey.
> > > * "indexed-queries" or "query-indexing" would be a good name?  This is
> > > the best technical name I can think of.
> > > *  "reverse search" came to mind (based on the Netflix article)
> > > although that makes me think of leading-wildcard / suffix-search.
> > > * "inverted-search"
> > > *  "indexed-query-alerts" incorporates "alerts" thus might better
> > > convey the use-case
> > >
> > >> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> > >> A) <lk...@bloomberg.net> wrote:
> > >>
> > >> Hi All,
> > >>
> > >> A few months ago I wrote the user list about potentially integrating
> lucene
> > monitor into solr. I have raised this PR with a first attempt at
> implementing
> > this integration. I'd greatly appreciate any feedback on this even
> though I
> > still have it marked as draft. I want to make sure I'm heading in the
> right
> > direction here so input from solr dev community would be extremely
> valuable :-)
> > >>
> > >> Many thanks,
> > >> Luke
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > For additional commands, e-mail: dev-help@solr.apache.org
> > >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: solr query alerting

Posted by Houston Putman <ho...@apache.org>.

I think I'd prefer a more self-descriptive name than "Luwak", which is just
a product name that was decided a while ago.

I kind of like "search-alerts". "query-alerts" sounds like alerting on
query metrics, but IMO "search-alerts" doesn't come with the same baggage.

Luwak is fine though if everyone agrees on that.

On one hand we have a number of committers here from
> Bloomberg, yet the abandoned and now-removed "analytics" component
> shows that abandonment is a risk nonetheless.
>

I don't want to bikeshed here, but I'm not sure this is a fair
assessment of what happened with the analytics module.
Sure there wasn't a ton of development, but in general it was feature rich
and had very little feature requests.
It was removed in 10, because a lack of user usage, not because it was
"abandoned" IMO. If there were requests from users
to keep it or improve it, then it would be a much different story. The
whole "thrown over the wall" comment is fair, but
not particularly relevant to this PR, which is being worked on in public.

I think this is certainly something that can start in the sandbox and move
into the main repo once it's clear that there is interest from
multiple committers and community members in using and maintaining it.

- Houston

On Wed, May 1, 2024 at 2:32 PM David Smiley <ds...@apache.org> wrote:

> Luwak is good to me!
>
> On Tue, Apr 30, 2024 at 4:01 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> A) <lk...@bloomberg.net> wrote:
> >
> > I love the name "luwak"! I was about to suggest the same but was worried
> about the trademark concerns and I assumed there was a reason they changed
> the name when donating it to lucene.
> >
> > From: dev@solr.apache.org At: 04/30/24 15:56:22 UTC-4:00To:
> dev@solr.apache.org
> > Subject: Re: solr query alerting
> >
> > Luwak is the original name of the Lucene monitor, contributed by Flax
> back in
> > the days: https://github.com/flaxsearch/luwak
> >
> > Perhaps we could go full circle (if no trademark issues) to call it the
> Solr
> > luwak module? Luwak is a type of coffee, and thus related to percolator
> 😉
> >
> > Otherwise “stored-queries” is an option.
> >
> > Jan Høydahl
> >
> > > 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
> > >
> > > I agree the feature is relevant / useful.
> > >
> > > Another angle on the module vs sandbox or wherever else is maintenance
> > > cost.  If a lot of code is being contributed as is here, then as a PMC
> > > member I hope to get a subjective sense that folks are interested in
> > > maintaining it.  On one hand we have a number of committers here from
> > > Bloomberg, yet the abandoned and now-removed "analytics" component
> > > shows that abandonment is a risk nonetheless.  I don't know how to
> > > conclude this thought but I'm hoping to hear from folks that they
> > > intend to look after this module.  It's not just being "thrown over
> > > the wall", so to speak.
> > >
> > > Naming is hard...
> > > * ...-monitor-....: sorry I hate it
> > > * ...-percolator-.... No clue why this was chosen for ElasticSearch.
> > > I can appreciate a curious/non-obvious name like this that is not
> > > going to conflict with anyone's guesses at what a general name might
> > > convey.
> > > * "indexed-queries" or "query-indexing" would be a good name?  This is
> > > the best technical name I can think of.
> > > *  "reverse search" came to mind (based on the Netflix article)
> > > although that makes me think of leading-wildcard / suffix-search.
> > > * "inverted-search"
> > > *  "indexed-query-alerts" incorporates "alerts" thus might better
> > > convey the use-case
> > >
> > >> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> > >> A) <lk...@bloomberg.net> wrote:
> > >>
> > >> Hi All,
> > >>
> > >> A few months ago I wrote the user list about potentially integrating
> lucene
> > monitor into solr. I have raised this PR with a first attempt at
> implementing
> > this integration. I'd greatly appreciate any feedback on this even
> though I
> > still have it marked as draft. I want to make sure I'm heading in the
> right
> > direction here so input from solr dev community would be extremely
> valuable :-)
> > >>
> > >> Many thanks,
> > >> Luke
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > For additional commands, e-mail: dev-help@solr.apache.org
> > >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: solr query alerting

Posted by David Smiley <ds...@apache.org>.

Luwak is good to me!

On Tue, Apr 30, 2024 at 4:01 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
A) <lk...@bloomberg.net> wrote:
>
> I love the name "luwak"! I was about to suggest the same but was worried about the trademark concerns and I assumed there was a reason they changed the name when donating it to lucene.
>
> From: dev@solr.apache.org At: 04/30/24 15:56:22 UTC-4:00To:  dev@solr.apache.org
> Subject: Re: solr query alerting
>
> Luwak is the original name of the Lucene monitor, contributed by Flax back in
> the days: https://github.com/flaxsearch/luwak
>
> Perhaps we could go full circle (if no trademark issues) to call it the Solr
> luwak module? Luwak is a type of coffee, and thus related to percolator 😉
>
> Otherwise “stored-queries” is an option.
>
> Jan Høydahl
>
> > 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
> >
> > I agree the feature is relevant / useful.
> >
> > Another angle on the module vs sandbox or wherever else is maintenance
> > cost.  If a lot of code is being contributed as is here, then as a PMC
> > member I hope to get a subjective sense that folks are interested in
> > maintaining it.  On one hand we have a number of committers here from
> > Bloomberg, yet the abandoned and now-removed "analytics" component
> > shows that abandonment is a risk nonetheless.  I don't know how to
> > conclude this thought but I'm hoping to hear from folks that they
> > intend to look after this module.  It's not just being "thrown over
> > the wall", so to speak.
> >
> > Naming is hard...
> > * ...-monitor-....: sorry I hate it
> > * ...-percolator-.... No clue why this was chosen for ElasticSearch.
> > I can appreciate a curious/non-obvious name like this that is not
> > going to conflict with anyone's guesses at what a general name might
> > convey.
> > * "indexed-queries" or "query-indexing" would be a good name?  This is
> > the best technical name I can think of.
> > *  "reverse search" came to mind (based on the Netflix article)
> > although that makes me think of leading-wildcard / suffix-search.
> > * "inverted-search"
> > *  "indexed-query-alerts" incorporates "alerts" thus might better
> > convey the use-case
> >
> >> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
> >> A) <lk...@bloomberg.net> wrote:
> >>
> >> Hi All,
> >>
> >> A few months ago I wrote the user list about potentially integrating lucene
> monitor into solr. I have raised this PR with a first attempt at implementing
> this integration. I'd greatly appreciate any feedback on this even though I
> still have it marked as draft. I want to make sure I'm heading in the right
> direction here so input from solr dev community would be extremely valuable :-)
> >>
> >> Many thanks,
> >> Luke
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: solr query alerting

Posted by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net>.

I love the name "luwak"! I was about to suggest the same but was worried about the trademark concerns and I assumed there was a reason they changed the name when donating it to lucene.

From: dev@solr.apache.org At: 04/30/24 15:56:22 UTC-4:00To:  dev@solr.apache.org
Subject: Re: solr query alerting

Luwak is the original name of the Lucene monitor, contributed by Flax back in 
the days: https://github.com/flaxsearch/luwak

Perhaps we could go full circle (if no trademark issues) to call it the Solr 
luwak module? Luwak is a type of coffee, and thus related to percolator 😉

Otherwise “stored-queries” is an option.

Jan Høydahl

> 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
> 
> I agree the feature is relevant / useful.
> 
> Another angle on the module vs sandbox or wherever else is maintenance
> cost.  If a lot of code is being contributed as is here, then as a PMC
> member I hope to get a subjective sense that folks are interested in
> maintaining it.  On one hand we have a number of committers here from
> Bloomberg, yet the abandoned and now-removed "analytics" component
> shows that abandonment is a risk nonetheless.  I don't know how to
> conclude this thought but I'm hoping to hear from folks that they
> intend to look after this module.  It's not just being "thrown over
> the wall", so to speak.
> 
> Naming is hard...
> * ...-monitor-....: sorry I hate it
> * ...-percolator-.... No clue why this was chosen for ElasticSearch.
> I can appreciate a curious/non-obvious name like this that is not
> going to conflict with anyone's guesses at what a general name might
> convey.
> * "indexed-queries" or "query-indexing" would be a good name?  This is
> the best technical name I can think of.
> *  "reverse search" came to mind (based on the Netflix article)
> although that makes me think of leading-wildcard / suffix-search.
> * "inverted-search"
> *  "indexed-query-alerts" incorporates "alerts" thus might better
> convey the use-case
> 
>> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
>> A) <lk...@bloomberg.net> wrote:
>> 
>> Hi All,
>> 
>> A few months ago I wrote the user list about potentially integrating lucene 
monitor into solr. I have raised this PR with a first attempt at implementing 
this integration. I'd greatly appreciate any feedback on this even though I 
still have it marked as draft. I want to make sure I'm heading in the right 
direction here so input from solr dev community would be extremely valuable :-)
>> 
>> Many thanks,
>> Luke
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>

Re: solr query alerting

Posted by Jan Høydahl <ja...@cominvent.com>.

Luwak is the original name of the Lucene monitor, contributed by Flax back in the days: https://github.com/flaxsearch/luwak

Perhaps we could go full circle (if no trademark issues) to call it the Solr luwak module? Luwak is a type of coffee, and thus related to percolator 😉

Otherwise “stored-queries” is an option.

Jan Høydahl

> 30. apr. 2024 kl. 19:26 skrev David Smiley <ds...@apache.org>:
> 
> I agree the feature is relevant / useful.
> 
> Another angle on the module vs sandbox or wherever else is maintenance
> cost.  If a lot of code is being contributed as is here, then as a PMC
> member I hope to get a subjective sense that folks are interested in
> maintaining it.  On one hand we have a number of committers here from
> Bloomberg, yet the abandoned and now-removed "analytics" component
> shows that abandonment is a risk nonetheless.  I don't know how to
> conclude this thought but I'm hoping to hear from folks that they
> intend to look after this module.  It's not just being "thrown over
> the wall", so to speak.
> 
> Naming is hard...
> * ...-monitor-....: sorry I hate it
> * ...-percolator-.... No clue why this was chosen for ElasticSearch.
> I can appreciate a curious/non-obvious name like this that is not
> going to conflict with anyone's guesses at what a general name might
> convey.
> * "indexed-queries" or "query-indexing" would be a good name?  This is
> the best technical name I can think of.
> *  "reverse search" came to mind (based on the Netflix article)
> although that makes me think of leading-wildcard / suffix-search.
> * "inverted-search"
> *  "indexed-query-alerts" incorporates "alerts" thus might better
> convey the use-case
> 
>> On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
>> A) <lk...@bloomberg.net> wrote:
>> 
>> Hi All,
>> 
>> A few months ago I wrote the user list about potentially integrating lucene monitor into solr. I have raised this PR with a first attempt at implementing this integration. I'd greatly appreciate any feedback on this even though I still have it marked as draft. I want to make sure I'm heading in the right direction here so input from solr dev community would be extremely valuable :-)
>> 
>> Many thanks,
>> Luke
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>

Re: solr query alerting

Posted by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net>.

The point about maintenance is obviously important. I understand how a contribution by a (relatively) new community member always carries extra abandonment risk.

Regarding the naming, I like inverted-search over indexed-queries because indexing the queries is just a part of the feature (albeit an important one). Someone in the PR also suggested "saved search" which coincidentally also appeared in the netflix blogpost. 

Completely guessing here but percolator makes me think of a coffee percolator, and that this is a metaphor for straining the grounds (matching queries) from the liquid space that encompasses them. I was also trying to think of a similar metaphor like this but didn't get very far!

From: dev@solr.apache.org At: 04/30/24 13:26:25 UTC-4:00To:  dev@solr.apache.org
Subject: Re: solr query alerting

I agree the feature is relevant / useful.

Another angle on the module vs sandbox or wherever else is maintenance
cost.  If a lot of code is being contributed as is here, then as a PMC
member I hope to get a subjective sense that folks are interested in
maintaining it.  On one hand we have a number of committers here from
Bloomberg, yet the abandoned and now-removed "analytics" component
shows that abandonment is a risk nonetheless.  I don't know how to
conclude this thought but I'm hoping to hear from folks that they
intend to look after this module.  It's not just being "thrown over
the wall", so to speak.

Naming is hard...
* ...-monitor-....: sorry I hate it
* ...-percolator-.... No clue why this was chosen for ElasticSearch.
I can appreciate a curious/non-obvious name like this that is not
going to conflict with anyone's guesses at what a general name might
convey.
* "indexed-queries" or "query-indexing" would be a good name?  This is
the best technical name I can think of.
*  "reverse search" came to mind (based on the Netflix article)
although that makes me think of leading-wildcard / suffix-search.
* "inverted-search"
*  "indexed-query-alerts" incorporates "alerts" thus might better
convey the use-case

On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
A) <lk...@bloomberg.net> wrote:
>
> Hi All,
>
> A few months ago I wrote the user list about potentially integrating lucene 
monitor into solr. I have raised this PR with a first attempt at implementing 
this integration. I'd greatly appreciate any feedback on this even though I 
still have it marked as draft. I want to make sure I'm heading in the right 
direction here so input from solr dev community would be extremely valuable :-)
>
> Many thanks,
> Luke

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: solr query alerting

Posted by David Smiley <ds...@apache.org>.

I agree the feature is relevant / useful.

Another angle on the module vs sandbox or wherever else is maintenance
cost.  If a lot of code is being contributed as is here, then as a PMC
member I hope to get a subjective sense that folks are interested in
maintaining it.  On one hand we have a number of committers here from
Bloomberg, yet the abandoned and now-removed "analytics" component
shows that abandonment is a risk nonetheless.  I don't know how to
conclude this thought but I'm hoping to hear from folks that they
intend to look after this module.  It's not just being "thrown over
the wall", so to speak.

Naming is hard...
* ...-monitor-....: sorry I hate it
* ...-percolator-.... No clue why this was chosen for ElasticSearch.
I can appreciate a curious/non-obvious name like this that is not
going to conflict with anyone's guesses at what a general name might
convey.
* "indexed-queries" or "query-indexing" would be a good name?  This is
the best technical name I can think of.
*  "reverse search" came to mind (based on the Netflix article)
although that makes me think of leading-wildcard / suffix-search.
* "inverted-search"
*  "indexed-query-alerts" incorporates "alerts" thus might better
convey the use-case

On Mon, Apr 1, 2024 at 3:53 PM Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD
A) <lk...@bloomberg.net> wrote:
>
> Hi All,
>
> A few months ago I wrote the user list about potentially integrating lucene monitor into solr. I have raised this PR with a first attempt at implementing this integration. I'd greatly appreciate any feedback on this even though I still have it marked as draft. I want to make sure I'm heading in the right direction here so input from solr dev community would be extremely valuable :-)
>
> Many thanks,
> Luke

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re:solr query alerting

Posted by "Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A)" <lk...@bloomberg.net>.

Apologies .. The links didn't get parsed correctly:

My initial email to users: https://www.mail-archive.com/users@solr.apache.org/msg07028.html
The PR: https://github.com/apache/solr/pull/2382

From: Luke Kot-Zaniewski (BLOOMBERG/ 919 3RD A) At: 04/01/24 15:53:17 UTC-4:00To:  dev@solr.apache.org
Subject: solr query alerting

Hi All,

A few months ago I wrote the user list about potentially integrating lucene monitor into solr. I have raised this PR with a first attempt at implementing this integration. I'd greatly appreciate any feedback on this even though I still have it marked as draft. I want to make sure I'm heading in the right direction here so input from solr dev community would be extremely valuable :-)

Many thanks,
Luke