You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by Eric Pugh <ep...@opensourceconnections.com> on 2023/05/09 10:45:47 UTC
Dueling Umbrellas ;-)
We have some dueling Solr CLI umbrella JIRA issues. If images were allowed, I would have a picture of Shawn and I waving folded umbrellas around like 17th century swordsmen ;-).
https://issues.apache.org/jira/browse/SOLR-16757 Is the one I started, and it’s really just a grab bag of various JIRA’s that have been open around the Solr CLI.
https://issues.apache.org/jira/browse/SOLR-16790 Is one that Shawn opened that is a bit more targeted around how we start Solr from the CLI.
What if we rename SOLR-16790 to just “Improve Solr CLI tools startup of Solr” (removing the umbrella term since just two tickets under it)? If it makes sense to make it linked form SOLR-16757 then we could do that, but I don’t have a strong opinion there.
I do think it’s important that Shawn and I (and anyone involved!) be communicative on what we’re working on and the state of the JIRAs so that we don’t work at cross purposes. As I finish tickets, I’ll try to let folks know, and which is the next one so that we don’t work against each other.
Eric
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Dueling Umbrellas ;-)
Posted by Eric Pugh <ep...@opensourceconnections.com>.
Wouldn’t it be cool if SolrCLI used Graal under the covers?
One thing about the various bin/solr tools that folks mentioned was that they can be handy when you have complex auth scenarios, and that they can support these natively. One thing I’d like to see in the future is to have all the commands support not just BASIC auth, but go beyond and support our JWT auth. I’ve seen other CLI’s where you are prompted to open up the web app, do the auth, and then go back to the CLI. Those types of situations are a lot harder in Curl. See SOLR-14496, SOLR-13071.
I’ll also ask folks to maybe go back and look at the totality of the work in SOLR-16757 and maybe think about how this all fits together.
For example, SOLR-14673 is about adding a CLI for streaming expressions. I want a CLI for streaming expressions to support all the auths from the start if possible ;-)
SOLR-15831 and SOLR-11206 is going to slim down how much raw text and code is in our platform specific shell commands.
If there are specific tickets that folks have opinions on, either about moving forward, not moving forward, or the design approach, I ask that you start adding your thoughts to them now. So that as I pick them up, hopefully I’m starting with some consensus already ;-).
Assuming our current SolrCLI is pretty clean, then maybe it makes it easier in the future for other languages etc to be used?
> On May 19, 2023, at 11:28 AM, Ishan Chattopadhyaya <ic...@gmail.com> wrote:
>
> +1 on exploring ways to address slow startup. My experience too, Houston.
>
> On Fri, 19 May, 2023, 8:25 pm Houston Putman, <houstonputman@gmail.com <ma...@gmail.com>> wrote:
>> Somewhat related question/thoughts on the Java SolrCLI:
>>
>> I have noticed that, at least for me, the SolrCLI is very slow. I added timing information to the gradle integration tests to show how slow it actually is.
>>
>> It looks like a baseline of starting a SolrCLI process is around 2.5 seconds for me, not including interacting with Solr at all.
>> I'm not really sure how we could think of improving this, since it just takes a while for the JVM to start unfortunately.
>>
>> I would love to use the SolrCLI to do auth checks by default for the Solr Operator, but the fact that it takes 4-5 seconds for a healthy check to complete is worrisome. Also creating the JVM every 10 seconds might be a burden on the system. (not really sure how big of a burden though)
>>
>> A few questions:
>> Are my settings messed up, and other people see very quick SolrCLI times?
>> Are these times generally ok for everyone?
>> Do we want to look into the possibility of using other languages/tools to build out the SolrCLI?
>> This is even more possible once the v2 API work is done and we can create APIs in any language.
>> Though the ZK interaction and Auth stuff will certainly be more complicated...
>> - Houston
>>
>> On Tue, May 9, 2023 at 1:05 PM Eric Pugh <epugh@opensourceconnections.com <ma...@opensourceconnections.com>> wrote:
>>> We have some dueling Solr CLI umbrella JIRA issues. If images were allowed, I would have a picture of Shawn and I waving folded umbrellas around like 17th century swordsmen ;-).
>>>
>>> [SOLR-16757] Umbrella Ticket for Reading Solr CLI's for the Future - ASF JIRA
>>> issues.apache.org
>>>
>>> <https://issues.apache.org/jira/browse/SOLR-16757>[SOLR-16757] Umbrella Ticket for Reading Solr CLI's for the Future - ASF JIRA <https://issues.apache.org/jira/browse/SOLR-16757>
>>> issues.apache.org <https://issues.apache.org/jira/browse/SOLR-16757> <https://issues.apache.org/jira/browse/SOLR-16757> Is the one I started, and it’s really just a grab bag of various JIRA’s that have been open around the Solr CLI.
>>>
>>> [SOLR-16790] Umbrella - Improve Solr CLI tools - ASF JIRA
>>> issues.apache.org
>>>
>>> <https://issues.apache.org/jira/browse/SOLR-16790>[SOLR-16790] Umbrella - Improve Solr CLI tools - ASF JIRA <https://issues.apache.org/jira/browse/SOLR-16790>
>>> issues.apache.org <https://issues.apache.org/jira/browse/SOLR-16790> <https://issues.apache.org/jira/browse/SOLR-16790> Is one that Shawn opened that is a bit more targeted around how we start Solr from the CLI.
>>>
>>> What if we rename SOLR-16790 to just “Improve Solr CLI tools startup of Solr” (removing the umbrella term since just two tickets under it)? If it makes sense to make it linked form SOLR-16757 then we could do that, but I don’t have a strong opinion there.
>>>
>>> I do think it’s important that Shawn and I (and anyone involved!) be communicative on what we’re working on and the state of the JIRAs so that we don’t work at cross purposes. As I finish tickets, I’ll try to let folks know, and which is the next one so that we don’t work against each other.
>>>
>>>
>>> Eric
>>>
>>>
>>> _______________________
>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>>> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org <ma...@solr.apache.org>
>> For additional commands, e-mail: dev-help@solr.apache.org <ma...@solr.apache.org>
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Dueling Umbrellas ;-)
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
+1 on exploring ways to address slow startup. My experience too, Houston.
On Fri, 19 May, 2023, 8:25 pm Houston Putman, <ho...@gmail.com>
wrote:
> Somewhat related question/thoughts on the Java SolrCLI:
>
> I have noticed that, at least for me, the SolrCLI is very slow. I added
> timing information to the gradle integration tests to show how slow it
> actually is.
>
> It looks like a baseline of starting a SolrCLI process is around 2.5
> seconds for me, not including interacting with Solr at all.
> I'm not really sure how we could think of improving this, since it just
> takes a while for the JVM to start unfortunately.
>
> I would love to use the SolrCLI to do auth checks by default for the Solr
> Operator, but the fact that it takes 4-5 seconds for a healthy check to
> complete is worrisome. Also creating the JVM every 10 seconds might be a
> burden on the system. (not really sure how big of a burden though)
>
> A few questions:
>
> - Are my settings messed up, and other people see very quick SolrCLI
> times?
> - Are these times generally ok for everyone?
> - Do we want to look into the possibility of using other
> languages/tools to build out the SolrCLI?
> - This is even more possible once the v2 API work is done and we
> can create APIs in any language.
> - Though the ZK interaction and Auth stuff will certainly be more
> complicated...
>
> - Houston
>
> On Tue, May 9, 2023 at 1:05 PM Eric Pugh <ep...@opensourceconnections.com>
> wrote:
>
>> We have some dueling Solr CLI umbrella JIRA issues. If images were
>> allowed, I would have a picture of Shawn and I waving folded umbrellas
>> around like 17th century swordsmen ;-).
>>
>> [SOLR-16757] Umbrella Ticket for Reading Solr CLI's for the Future - ASF
>> JIRA <https://issues.apache.org/jira/browse/SOLR-16757>
>> issues.apache.org <https://issues.apache.org/jira/browse/SOLR-16757>
>> [image: fav-jsw.png] <https://issues.apache.org/jira/browse/SOLR-16757>
>> <https://issues.apache.org/jira/browse/SOLR-16757>
>> Is the one I started, and it’s really just a grab bag of various JIRA’s
>> that have been open around the Solr CLI.
>>
>> [SOLR-16790] Umbrella - Improve Solr CLI tools - ASF JIRA
>> <https://issues.apache.org/jira/browse/SOLR-16790>
>> issues.apache.org <https://issues.apache.org/jira/browse/SOLR-16790>
>> [image: fav-jsw.png] <https://issues.apache.org/jira/browse/SOLR-16790>
>> <https://issues.apache.org/jira/browse/SOLR-16790>
>> Is one that Shawn opened that is a bit more targeted around how we start
>> Solr from the CLI.
>>
>> What if we rename SOLR-16790 to just “Improve Solr CLI tools startup of
>> Solr” (removing the umbrella term since just two tickets under it)? If
>> it makes sense to make it linked form SOLR-16757 then we could do that, but
>> I don’t have a strong opinion there.
>>
>> I do think it’s important that Shawn and I (and anyone involved!) be
>> communicative on what we’re working on and the state of the JIRAs so that
>> we don’t work at cross purposes. As I finish tickets, I’ll try to let
>> folks know, and which is the next one so that we don’t work against each
>> other.
>>
>>
>> Eric
>>
>>
>> _______________________
>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>> | http://www.opensourceconnections.com | My Free/Busy
>> <http://tinyurl.com/eric-cal>
>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>> This e-mail and all contents, including attachments, is considered to be
>> Company Confidential unless explicitly stated otherwise, regardless
>> of whether attachments are marked as such.
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
Re: Dueling Umbrellas ;-)
Posted by Houston Putman <ho...@gmail.com>.
Somewhat related question/thoughts on the Java SolrCLI:
I have noticed that, at least for me, the SolrCLI is very slow. I added
timing information to the gradle integration tests to show how slow it
actually is.
It looks like a baseline of starting a SolrCLI process is around 2.5
seconds for me, not including interacting with Solr at all.
I'm not really sure how we could think of improving this, since it just
takes a while for the JVM to start unfortunately.
I would love to use the SolrCLI to do auth checks by default for the Solr
Operator, but the fact that it takes 4-5 seconds for a healthy check to
complete is worrisome. Also creating the JVM every 10 seconds might be a
burden on the system. (not really sure how big of a burden though)
A few questions:
- Are my settings messed up, and other people see very quick SolrCLI
times?
- Are these times generally ok for everyone?
- Do we want to look into the possibility of using other languages/tools
to build out the SolrCLI?
- This is even more possible once the v2 API work is done and we can
create APIs in any language.
- Though the ZK interaction and Auth stuff will certainly be more
complicated...
- Houston
On Tue, May 9, 2023 at 1:05 PM Eric Pugh <ep...@opensourceconnections.com>
wrote:
> We have some dueling Solr CLI umbrella JIRA issues. If images were
> allowed, I would have a picture of Shawn and I waving folded umbrellas
> around like 17th century swordsmen ;-).
>
> [SOLR-16757] Umbrella Ticket for Reading Solr CLI's for the Future - ASF
> JIRA <https://issues.apache.org/jira/browse/SOLR-16757>
> issues.apache.org <https://issues.apache.org/jira/browse/SOLR-16757>
> [image: fav-jsw.png] <https://issues.apache.org/jira/browse/SOLR-16757>
> <https://issues.apache.org/jira/browse/SOLR-16757>
> Is the one I started, and it’s really just a grab bag of various JIRA’s
> that have been open around the Solr CLI.
>
> [SOLR-16790] Umbrella - Improve Solr CLI tools - ASF JIRA
> <https://issues.apache.org/jira/browse/SOLR-16790>
> issues.apache.org <https://issues.apache.org/jira/browse/SOLR-16790>
> [image: fav-jsw.png] <https://issues.apache.org/jira/browse/SOLR-16790>
> <https://issues.apache.org/jira/browse/SOLR-16790>
> Is one that Shawn opened that is a bit more targeted around how we start
> Solr from the CLI.
>
> What if we rename SOLR-16790 to just “Improve Solr CLI tools startup of
> Solr” (removing the umbrella term since just two tickets under it)? If
> it makes sense to make it linked form SOLR-16757 then we could do that, but
> I don’t have a strong opinion there.
>
> I do think it’s important that Shawn and I (and anyone involved!) be
> communicative on what we’re working on and the state of the JIRAs so that
> we don’t work at cross purposes. As I finish tickets, I’ll try to let
> folks know, and which is the next one so that we don’t work against each
> other.
>
>
> Eric
>
>
> _______________________
> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> | http://www.opensourceconnections.com | My Free/Busy
> <http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
>