You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by Eric Pugh <ep...@opensourceconnections.com> on 2023/05/17 11:32:17 UTC
Anyone using more obscure features of SimplePostTool?
Hi all,
As part of SOLR-6994 I’m migrating the SimplePostTool to be part of bin/solr commands.
We document a number of example use cases:
* JSON file: bin/solr post -url http://localhost:8983/wizbang/update events.json
* XML files: bin/solr post -url http://localhost:8983/records/update article*.xml
* CSV file: bin/solr post -url http://localhost:8983/signals/update LATEST-signals.csv
* Directory of files: bin/solr post -url http://localhost:8983/myfiles/update ~/Documents
* Web crawl: bin/solr post -url http://localhost:8983/gettingstarted/update https://solr.apache.org/ -recursive 1 -delay 1
* Standard input (stdin): echo '{commit: {}}' | bin/solr post -url http://localhost:8983/my_collection/update -type application/json -out yes -d
* Data as string: bin/solr post -url http://localhost:8983/signals/update -type text/csv -out yes -d $'id,value\n1,0.47'
The last two, stdin and data as a string, feel rather obscure to me, and I’d like to not port them over to being supported by the bin/solr post tool equivalent. Thoughts?
Eric
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Anyone using more obscure features of SimplePostTool?
Posted by Eric Pugh <ep...@opensourceconnections.com>.
I’ve heard both “We can’t just change it in Solr 9 because folks are using it, so we need to do proper deprecation etc” and “no one should use it” ;-). Also, the biggest reason to move from bin/post over to bin/solr post was to take advantage of the revamped Solr CLI stuff to reduce tech debt and duplication of code, so keeping bin/post would go against that goal ;-). I’d also like to keep the tool for all the same reasons that we might want to use bin/solr create etc, sometimes having user authentication is easier via a tool versus straight up curl….
How about this as some concrete steps to move forward:
* Do the "bin/solr post" migration
* Add a message about “bin/solr post isn’t meant for production use” or other messaging to highlight “toy” nature that is emitted when you use "bin/solr post".
* Narrow it down to just the file posting…? I feel a bit sad on the web crawler if it works well, but….
???
Eric
> On May 22, 2023, at 8:06 AM, Jan Høydahl <ja...@cominvent.com> wrote:
>
>> 17. mai 2023 kl. 15:35 skrev Gus Heck <gu...@gmail.com>:
>>
>> I suspect we could do without the webcrawl one too. Honestly these should
>> only be considered for demonstration or extremely small installations, and
>> anything not used in the tutorials could be cut IMHO.
>
> Agree, I commented the same a few days ago on the PR: https://github.com/apache/solr/pull/1634#issuecomment-1549574758
>
> I'm not necessarily against moving the tool to bin/solr, but at the same time, perhaps it would be easier to label it as a dev or non-production tool by keeping it as bin/post (but with the new Java backend, so it also works for windows)?
>
> Ishan, you did not give an argument for why you believe bin/post should go away. Do you feel it is better to document all the cURL commands to be more "standard"? I can buy such an argument for the simple use case of posting one file, as it will also teach the user how the API works. But as a dev tool for quickly ingesting a bunch of sample docs into Solr, I still think bin/post has a place, unless we can point people to some similar 3rd party tool?
>
> Jan
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Anyone using more obscure features of SimplePostTool?
Posted by Jan Høydahl <ja...@cominvent.com>.
> 17. mai 2023 kl. 15:35 skrev Gus Heck <gu...@gmail.com>:
>
> I suspect we could do without the webcrawl one too. Honestly these should
> only be considered for demonstration or extremely small installations, and
> anything not used in the tutorials could be cut IMHO.
Agree, I commented the same a few days ago on the PR: https://github.com/apache/solr/pull/1634#issuecomment-1549574758
I'm not necessarily against moving the tool to bin/solr, but at the same time, perhaps it would be easier to label it as a dev or non-production tool by keeping it as bin/post (but with the new Java backend, so it also works for windows)?
Ishan, you did not give an argument for why you believe bin/post should go away. Do you feel it is better to document all the cURL commands to be more "standard"? I can buy such an argument for the simple use case of posting one file, as it will also teach the user how the API works. But as a dev tool for quickly ingesting a bunch of sample docs into Solr, I still think bin/post has a place, unless we can point people to some similar 3rd party tool?
Jan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: Anyone using more obscure features of SimplePostTool?
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
The post tool should be removed completely.
On Wed, 17 May, 2023, 7:06 pm Gus Heck, <gu...@gmail.com> wrote:
> I suspect we could do without the webcrawl one too. Honestly these should
> only be considered for demonstration or extremely small installations, and
> anything not used in the tutorials could be cut IMHO.
>
> On Wed, May 17, 2023 at 7:32 AM Eric Pugh <epugh@opensourceconnections.com
> >
> wrote:
>
> > Hi all,
> >
> > As part of SOLR-6994 I’m migrating the SimplePostTool to be part of
> > bin/solr commands.
> >
> >
> > We document a number of example use cases:
> >
> > * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
> > events.json
> > * XML files: bin/solr post -url http://localhost:8983/records/update
> > article*.xml
> > * CSV file: bin/solr post -url http://localhost:8983/signals/update
> > LATEST-signals.csv
> > * Directory of files: bin/solr post -url
> > http://localhost:8983/myfiles/update ~/Documents
> > * Web crawl: bin/solr post -url
> > http://localhost:8983/gettingstarted/update https://solr.apache.org/
> > -recursive 1 -delay 1
> > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> > http://localhost:8983/my_collection/update -type application/json -out
> > yes -d
> > * Data as string: bin/solr post -url
> http://localhost:8983/signals/update
> > -type text/csv -out yes -d $'id,value\n1,0.47'
> >
> >
> > The last two, stdin and data as a string, feel rather obscure to me, and
> > I’d like to not port them over to being supported by the bin/solr post
> tool
> > equivalent. Thoughts?
> >
> > Eric
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> > http://www.opensourceconnections.com <
> > http://www.opensourceconnections.com/> | My Free/Busy <
> > http://tinyurl.com/eric-cal>
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> >
> >
> > This e-mail and all contents, including attachments, is considered to be
> > Company Confidential unless explicitly stated otherwise, regardless of
> > whether attachments are marked as such.
> >
> >
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
Re: Anyone using more obscure features of SimplePostTool?
Posted by Gus Heck <gu...@gmail.com>.
I suspect we could do without the webcrawl one too. Honestly these should
only be considered for demonstration or extremely small installations, and
anything not used in the tutorials could be cut IMHO.
On Wed, May 17, 2023 at 7:32 AM Eric Pugh <ep...@opensourceconnections.com>
wrote:
> Hi all,
>
> As part of SOLR-6994 I’m migrating the SimplePostTool to be part of
> bin/solr commands.
>
>
> We document a number of example use cases:
>
> * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
> events.json
> * XML files: bin/solr post -url http://localhost:8983/records/update
> article*.xml
> * CSV file: bin/solr post -url http://localhost:8983/signals/update
> LATEST-signals.csv
> * Directory of files: bin/solr post -url
> http://localhost:8983/myfiles/update ~/Documents
> * Web crawl: bin/solr post -url
> http://localhost:8983/gettingstarted/update https://solr.apache.org/
> -recursive 1 -delay 1
> * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> http://localhost:8983/my_collection/update -type application/json -out
> yes -d
> * Data as string: bin/solr post -url http://localhost:8983/signals/update
> -type text/csv -out yes -d $'id,value\n1,0.47'
>
>
> The last two, stdin and data as a string, feel rather obscure to me, and
> I’d like to not port them over to being supported by the bin/solr post tool
> equivalent. Thoughts?
>
> Eric
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Anyone using more obscure features of SimplePostTool?
Posted by "Arrieta, Alejandro" <aa...@perrinsoftware.com>.
Hello Team,
In previous Solr versions, afaik examples post tool did not support
Kerberos (have not checked Solr 9.x), so each time a customer asked about
the Apache Solr tutorial I recommended them to use curl command as it works
with/without kerberos (curl --negotiate ....). There are some curls that do
not support Kerberos (anaconda curl), but the linux distro default supports
it.
Yes, I work with kerberized Solr :-)
Kind Regards,
Alejandro Arrieta
On Tue, May 23, 2023 at 6:05 AM Eric Pugh <ep...@opensourceconnections.com>
wrote:
> Lots of feedback!
>
> So, would renaming SOLR-6994 from "Implement Windows version of bin/post"
> to “Implement Windows version of bin/post via implementing bin/solr post”
> address your concern David?
>
> It’s both for Solr technical simplicity, but also to let Window’s users
> actually have better support. In the not too distant future I want to
> get BASIC and JWT auth across all our CLI tools and not having this one
> orphaned “bin/post” and random checked in “post.jar” hanging out would help
> that.
>
> I don’t think we can remove the capability to post documents completely,
> that it has to be deprecated. And honestly, if the darn thing worked a
> bit better (like in the aforementioned more secure scenarios) then it would
> provide plenty of value.
>
>
>
>
> > On May 23, 2023, at 2:16 AM, Marcus Eagan <ma...@gmail.com> wrote:
> >
> > Smiley, that's actually a pretty good use case.
> >
> > On Tue, May 23, 2023 at 2:15 AM David Smiley <ds...@apache.org> wrote:
> >
> >> Glad to see we can limit this discussion to "bin/post" and not
> >> other non-prod/maybe-prod things listed.
> >>
> >> I think bin/post should either exist where it is or not exist; moving
> it to
> >> the sandbox misses the point.
> >> "curl" may very well be plenty adequate. FWIW your arguments work for
> me
> >> to ditch bin/post, but it could be useful to hear from
> educators/trainers
> >> as well. It's been a while since I could count myself as such.
> >>
> >> BTW I know of at least one advantage over curl. I was once posting
> massive
> >> many-gigabyte files to Solr in CSV (for a POC scenario). Curl wanted to
> >> memory-map the whole thing and failed to get the RAM to do it.
> "bin/post"
> >> streamed the whole thing quite fast. I know I made improvements to
> help it
> >> in this specific regard. That said, if bin/post is mostly redundant, I
> >> wouldn't want us to maintain bin/post forever *just* because of this.
> >>
> >> ~ David Smiley
> >> Apache Lucene/Solr Search Developer
> >> http://www.linkedin.com/in/davidwsmiley
> >>
> >>
> >> On Tue, May 23, 2023 at 2:05 AM Ishan Chattopadhyaya <
> >> ichattopadhyaya@gmail.com> wrote:
> >>
> >>> We have advertised the post tool in the past in our examples/tutorials.
> >>> Difficult to erase all traces of it from the web.
> >>> It is better to remove the tool from the Solr distro, and, maybe,
> >> relocate
> >>> it to the sandbox/extras repo, from where a user can consciously choose
> >> to
> >>> download and use it if needed.
> >>>
> >>> If we have proper documentation, I don't see how anyone building POCs
> >> will
> >>> miss out on the post tool. Is there a use case that users can't achieve
> >>> easily through a documented method that the post tool caters to?
> >>>
> >>> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org> wrote:
> >>>
> >>>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your
> point
> >>> of
> >>>> view... is the key word of your response "advertise" and we can then
> >>> debate
> >>>> what that means? In other words, are you saying bin/post (and other
> >>> things
> >>>> you don't think should be used in production) should not exist at all
> >> or
> >>> we
> >>>> should be very careful about how we "advertise" its existence? I
> think
> >>>> features that help people build POCs / explore / learn are good
> things,
> >>>> even if not used typically in production. At times we over-advertised
> >>>> features without enough disclaimers on the suitability of the feature
> >> in
> >>>> question.
> >>>>
> >>>> ~ David Smiley
> >>>> Apache Lucene/Solr Search Developer
> >>>> http://www.linkedin.com/in/davidwsmiley
> >>>>
> >>>>
> >>>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
> >>>> ichattopadhyaya@gmail.com> wrote:
> >>>>
> >>>>>> Ishan, you did not give an argument for why you believe bin/post
> >>> should
> >>>>> go away. Do you feel it is better to document all the cURL commands
> >> to
> >>> be
> >>>>> more "standard"?
> >>>>>
> >>>>> Precisely! We should never advertise anything that shouldn't be used
> >> at
> >>>>> scale or in production. That includes the post tool, example modes of
> >>>>> startup, schemaless/data-driven etc.
> >>>>>
> >>>>> Curl is popular enough on Windows as well, through ports or WSL.
> >> There
> >>>> are
> >>>>> tools like Insomnia or Postman too. We should encourage their use,
> >>> rather
> >>>>> than promote our home grown tool by shipping oob and/or referring to
> >> it
> >>>> in
> >>>>> tutorials.
> >>>>>
> >>>>> On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
> >>> wrote:
> >>>>>
> >>>>>> *If* bin/solr is to subsume bin/post, I think it deserves its own
> >>>> issue;
> >>>>>> should not be a detail of SOLR-6994 -- "Implement Windows version
> >> of
> >>>>>> bin/post". Really; wow that'd be sneaky to do something so visible
> >>> to
> >>>>>> users / documentation under a OS/platform compatibility oriented
> >>> title.
> >>>>>>
> >>>>>> I am strongly not a fan of having bin/solr subsume bin/post.
> >>> bin/post
> >>>> is
> >>>>>> very distinct enough in purpose to remain separate. The stated
> >>>>> motivation
> >>>>>> seems out of convenience to Solr internal workings, which doesn't
> >>> serve
> >>>>> the
> >>>>>> user's best interests IMO.
> >>>>>>
> >>>>>> ~ David Smiley
> >>>>>> Apache Lucene/Solr Search Developer
> >>>>>> http://www.linkedin.com/in/davidwsmiley
> >>>>>>
> >>>>>>
> >>>>>> On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> >>>>> epugh@opensourceconnections.com
> >>>>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> As part of SOLR-6994 I’m migrating the SimplePostTool to be part
> >> of
> >>>>>>> bin/solr commands.
> >>>>>>>
> >>>>>>>
> >>>>>>> We document a number of example use cases:
> >>>>>>>
> >>>>>>> * JSON file: bin/solr post -url
> >>> http://localhost:8983/wizbang/update
> >>>>>>> events.json
> >>>>>>> * XML files: bin/solr post -url
> >>> http://localhost:8983/records/update
> >>>>>>> article*.xml
> >>>>>>> * CSV file: bin/solr post -url
> >>> http://localhost:8983/signals/update
> >>>>>>> LATEST-signals.csv
> >>>>>>> * Directory of files: bin/solr post -url
> >>>>>>> http://localhost:8983/myfiles/update ~/Documents
> >>>>>>> * Web crawl: bin/solr post -url
> >>>>>>> http://localhost:8983/gettingstarted/update
> >>> https://solr.apache.org/
> >>>>>>> -recursive 1 -delay 1
> >>>>>>> * Standard input (stdin): echo '{commit: {}}' | bin/solr post
> >> -url
> >>>>>>> http://localhost:8983/my_collection/update -type
> >> application/json
> >>>> -out
> >>>>>>> yes -d
> >>>>>>> * Data as string: bin/solr post -url
> >>>>>> http://localhost:8983/signals/update
> >>>>>>> -type text/csv -out yes -d $'id,value\n1,0.47'
> >>>>>>>
> >>>>>>>
> >>>>>>> The last two, stdin and data as a string, feel rather obscure to
> >>> me,
> >>>>> and
> >>>>>>> I’d like to not port them over to being supported by the bin/solr
> >>>> post
> >>>>>> tool
> >>>>>>> equivalent. Thoughts?
> >>>>>>>
> >>>>>>> Eric
> >>>>>>> _______________________
> >>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> >>>> 434.466.1467
> >>>>> |
> >>>>>>> http://www.opensourceconnections.com <
> >>>>>>> http://www.opensourceconnections.com/> | My Free/Busy <
> >>>>>>> http://tinyurl.com/eric-cal>
> >>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> >>>>>>>
> >>>>>>>
> >>>>>>> This e-mail and all contents, including attachments, is
> >> considered
> >>> to
> >>>>> be
> >>>>>>> Company Confidential unless explicitly stated otherwise,
> >> regardless
> >>>> of
> >>>>>>> whether attachments are marked as such.
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
> >
> > --
> > Marcus Eagan
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>
Re: Anyone using more obscure features of SimplePostTool?
Posted by Eric Pugh <ep...@opensourceconnections.com>.
I poked around it, and honestly, some of the more obscure features are kind of cool…. I can *maybe* see how they could be very useful if they weren’t so hard to find….
It may also be that since I’ve crawled through SimplePostTool and done some minor cleanups and written tests for the functionality (finding some bugs) that maybe I’m more bought in….
I think for me, the value of some of this may really come in focus once we get full on security onto Solr (for those who want it!), and that interacting with Solr via straight up curl becomes a bit more difficult, and if that becomes easier with the tools, then maybe more usage??/
I tried “brew install Solr-cli”, too bad that didn’t work, but that’s for another JIRA ;-).
> On May 24, 2023, at 12:35 AM, Marcus Eagan <ma...@gmail.com> wrote:
>
> +1 on deprecating bin/post. I haven't used it in many years either.
>
> On Tue, May 23, 2023 at 7:30 PM Joel Bernstein <jo...@gmail.com> wrote:
>
>> I have mixed feelings about this because I love command line tools. But I
>> haven't actually used the bin/post tool in years.
>>
>>
>>
>> On Tue, May 23, 2023 at 10:13 AM David Smiley <ds...@apache.org> wrote:
>>
>>> +1 to deprecate bin/post.
>>>
>>> Eric, RE the issue title... it's better. Still, if I were advocating for
>>> the work (and I'm definitely not), I'd have a dedicated issue
>> nonetheless,
>>> may or may not even have a separate Windows issue if it gets fixed by
>> side
>>> effect of moving the functionality.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Tue, May 23, 2023 at 6:05 AM Eric Pugh <
>> epugh@opensourceconnections.com
>>>>
>>> wrote:
>>>
>>>> Lots of feedback!
>>>>
>>>> So, would renaming SOLR-6994 from "Implement Windows version of
>> bin/post"
>>>> to “Implement Windows version of bin/post via implementing bin/solr
>> post”
>>>> address your concern David?
>>>>
>>>> It’s both for Solr technical simplicity, but also to let Window’s users
>>>> actually have better support. In the not too distant future I want
>> to
>>>> get BASIC and JWT auth across all our CLI tools and not having this one
>>>> orphaned “bin/post” and random checked in “post.jar” hanging out would
>>> help
>>>> that.
>>>>
>>>> I don’t think we can remove the capability to post documents
>> completely,
>>>> that it has to be deprecated. And honestly, if the darn thing worked
>> a
>>>> bit better (like in the aforementioned more secure scenarios) then it
>>> would
>>>> provide plenty of value.
>>>>
>>>>
>>>>
>>>>
>>>>> On May 23, 2023, at 2:16 AM, Marcus Eagan <ma...@gmail.com>
>>> wrote:
>>>>>
>>>>> Smiley, that's actually a pretty good use case.
>>>>>
>>>>> On Tue, May 23, 2023 at 2:15 AM David Smiley <ds...@apache.org>
>>> wrote:
>>>>>
>>>>>> Glad to see we can limit this discussion to "bin/post" and not
>>>>>> other non-prod/maybe-prod things listed.
>>>>>>
>>>>>> I think bin/post should either exist where it is or not exist;
>> moving
>>>> it to
>>>>>> the sandbox misses the point.
>>>>>> "curl" may very well be plenty adequate. FWIW your arguments work
>> for
>>>> me
>>>>>> to ditch bin/post, but it could be useful to hear from
>>>> educators/trainers
>>>>>> as well. It's been a while since I could count myself as such.
>>>>>>
>>>>>> BTW I know of at least one advantage over curl. I was once posting
>>>> massive
>>>>>> many-gigabyte files to Solr in CSV (for a POC scenario). Curl
>> wanted
>>> to
>>>>>> memory-map the whole thing and failed to get the RAM to do it.
>>>> "bin/post"
>>>>>> streamed the whole thing quite fast. I know I made improvements to
>>>> help it
>>>>>> in this specific regard. That said, if bin/post is mostly
>> redundant,
>>> I
>>>>>> wouldn't want us to maintain bin/post forever *just* because of
>> this.
>>>>>>
>>>>>> ~ David Smiley
>>>>>> Apache Lucene/Solr Search Developer
>>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>>
>>>>>>
>>>>>> On Tue, May 23, 2023 at 2:05 AM Ishan Chattopadhyaya <
>>>>>> ichattopadhyaya@gmail.com> wrote:
>>>>>>
>>>>>>> We have advertised the post tool in the past in our
>>> examples/tutorials.
>>>>>>> Difficult to erase all traces of it from the web.
>>>>>>> It is better to remove the tool from the Solr distro, and, maybe,
>>>>>> relocate
>>>>>>> it to the sandbox/extras repo, from where a user can consciously
>>> choose
>>>>>> to
>>>>>>> download and use it if needed.
>>>>>>>
>>>>>>> If we have proper documentation, I don't see how anyone building
>> POCs
>>>>>> will
>>>>>>> miss out on the post tool. Is there a use case that users can't
>>> achieve
>>>>>>> easily through a documented method that the post tool caters to?
>>>>>>>
>>>>>>> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org>
>>> wrote:
>>>>>>>
>>>>>>>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your
>>>> point
>>>>>>> of
>>>>>>>> view... is the key word of your response "advertise" and we can
>> then
>>>>>>> debate
>>>>>>>> what that means? In other words, are you saying bin/post (and
>> other
>>>>>>> things
>>>>>>>> you don't think should be used in production) should not exist at
>>> all
>>>>>> or
>>>>>>> we
>>>>>>>> should be very careful about how we "advertise" its existence? I
>>>> think
>>>>>>>> features that help people build POCs / explore / learn are good
>>>> things,
>>>>>>>> even if not used typically in production. At times we
>>> over-advertised
>>>>>>>> features without enough disclaimers on the suitability of the
>>> feature
>>>>>> in
>>>>>>>> question.
>>>>>>>>
>>>>>>>> ~ David Smiley
>>>>>>>> Apache Lucene/Solr Search Developer
>>>>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
>>>>>>>> ichattopadhyaya@gmail.com> wrote:
>>>>>>>>
>>>>>>>>>> Ishan, you did not give an argument for why you believe bin/post
>>>>>>> should
>>>>>>>>> go away. Do you feel it is better to document all the cURL
>> commands
>>>>>> to
>>>>>>> be
>>>>>>>>> more "standard"?
>>>>>>>>>
>>>>>>>>> Precisely! We should never advertise anything that shouldn't be
>>> used
>>>>>> at
>>>>>>>>> scale or in production. That includes the post tool, example
>> modes
>>> of
>>>>>>>>> startup, schemaless/data-driven etc.
>>>>>>>>>
>>>>>>>>> Curl is popular enough on Windows as well, through ports or WSL.
>>>>>> There
>>>>>>>> are
>>>>>>>>> tools like Insomnia or Postman too. We should encourage their
>> use,
>>>>>>> rather
>>>>>>>>> than promote our home grown tool by shipping oob and/or referring
>>> to
>>>>>> it
>>>>>>>> in
>>>>>>>>> tutorials.
>>>>>>>>>
>>>>>>>>> On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> *If* bin/solr is to subsume bin/post, I think it deserves its
>> own
>>>>>>>> issue;
>>>>>>>>>> should not be a detail of SOLR-6994 -- "Implement Windows
>> version
>>>>>> of
>>>>>>>>>> bin/post". Really; wow that'd be sneaky to do something so
>>> visible
>>>>>>> to
>>>>>>>>>> users / documentation under a OS/platform compatibility oriented
>>>>>>> title.
>>>>>>>>>>
>>>>>>>>>> I am strongly not a fan of having bin/solr subsume bin/post.
>>>>>>> bin/post
>>>>>>>> is
>>>>>>>>>> very distinct enough in purpose to remain separate. The stated
>>>>>>>>> motivation
>>>>>>>>>> seems out of convenience to Solr internal workings, which
>> doesn't
>>>>>>> serve
>>>>>>>>> the
>>>>>>>>>> user's best interests IMO.
>>>>>>>>>>
>>>>>>>>>> ~ David Smiley
>>>>>>>>>> Apache Lucene/Solr Search Developer
>>>>>>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
>>>>>>>>> epugh@opensourceconnections.com
>>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> As part of SOLR-6994 I’m migrating the SimplePostTool to be
>> part
>>>>>> of
>>>>>>>>>>> bin/solr commands.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> We document a number of example use cases:
>>>>>>>>>>>
>>>>>>>>>>> * JSON file: bin/solr post -url
>>>>>>> http://localhost:8983/wizbang/update
>>>>>>>>>>> events.json
>>>>>>>>>>> * XML files: bin/solr post -url
>>>>>>> http://localhost:8983/records/update
>>>>>>>>>>> article*.xml
>>>>>>>>>>> * CSV file: bin/solr post -url
>>>>>>> http://localhost:8983/signals/update
>>>>>>>>>>> LATEST-signals.csv
>>>>>>>>>>> * Directory of files: bin/solr post -url
>>>>>>>>>>> http://localhost:8983/myfiles/update ~/Documents
>>>>>>>>>>> * Web crawl: bin/solr post -url
>>>>>>>>>>> http://localhost:8983/gettingstarted/update
>>>>>>> https://solr.apache.org/
>>>>>>>>>>> -recursive 1 -delay 1
>>>>>>>>>>> * Standard input (stdin): echo '{commit: {}}' | bin/solr post
>>>>>> -url
>>>>>>>>>>> http://localhost:8983/my_collection/update -type
>>>>>> application/json
>>>>>>>> -out
>>>>>>>>>>> yes -d
>>>>>>>>>>> * Data as string: bin/solr post -url
>>>>>>>>>> http://localhost:8983/signals/update
>>>>>>>>>>> -type text/csv -out yes -d $'id,value\n1,0.47'
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> The last two, stdin and data as a string, feel rather obscure
>> to
>>>>>>> me,
>>>>>>>>> and
>>>>>>>>>>> I’d like to not port them over to being supported by the
>> bin/solr
>>>>>>>> post
>>>>>>>>>> tool
>>>>>>>>>>> equivalent. Thoughts?
>>>>>>>>>>>
>>>>>>>>>>> Eric
>>>>>>>>>>> _______________________
>>>>>>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
>>>>>>>> 434.466.1467
>>>>>>>>> |
>>>>>>>>>>> http://www.opensourceconnections.com <
>>>>>>>>>>> http://www.opensourceconnections.com/> | My Free/Busy <
>>>>>>>>>>> http://tinyurl.com/eric-cal>
>>>>>>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This e-mail and all contents, including attachments, is
>>>>>> considered
>>>>>>> to
>>>>>>>>> be
>>>>>>>>>>> Company Confidential unless explicitly stated otherwise,
>>>>>> regardless
>>>>>>>> of
>>>>>>>>>>> whether attachments are marked as such.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Marcus Eagan
>>>>
>>>> _______________________
>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>> |
>>>> http://www.opensourceconnections.com <
>>>> http://www.opensourceconnections.com/> | My Free/Busy <
>>>> http://tinyurl.com/eric-cal>
>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
>>>>
>>>
>> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
>>>>
>>>>
>>>> This e-mail and all contents, including attachments, is considered to
>> be
>>>> Company Confidential unless explicitly stated otherwise, regardless of
>>>> whether attachments are marked as such.
>>>>
>>>>
>>>
>>
>
>
> --
> Marcus Eagan
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Anyone using more obscure features of SimplePostTool?
Posted by Marcus Eagan <ma...@gmail.com>.
+1 on deprecating bin/post. I haven't used it in many years either.
On Tue, May 23, 2023 at 7:30 PM Joel Bernstein <jo...@gmail.com> wrote:
> I have mixed feelings about this because I love command line tools. But I
> haven't actually used the bin/post tool in years.
>
>
>
> On Tue, May 23, 2023 at 10:13 AM David Smiley <ds...@apache.org> wrote:
>
> > +1 to deprecate bin/post.
> >
> > Eric, RE the issue title... it's better. Still, if I were advocating for
> > the work (and I'm definitely not), I'd have a dedicated issue
> nonetheless,
> > may or may not even have a separate Windows issue if it gets fixed by
> side
> > effect of moving the functionality.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Tue, May 23, 2023 at 6:05 AM Eric Pugh <
> epugh@opensourceconnections.com
> > >
> > wrote:
> >
> > > Lots of feedback!
> > >
> > > So, would renaming SOLR-6994 from "Implement Windows version of
> bin/post"
> > > to “Implement Windows version of bin/post via implementing bin/solr
> post”
> > > address your concern David?
> > >
> > > It’s both for Solr technical simplicity, but also to let Window’s users
> > > actually have better support. In the not too distant future I want
> to
> > > get BASIC and JWT auth across all our CLI tools and not having this one
> > > orphaned “bin/post” and random checked in “post.jar” hanging out would
> > help
> > > that.
> > >
> > > I don’t think we can remove the capability to post documents
> completely,
> > > that it has to be deprecated. And honestly, if the darn thing worked
> a
> > > bit better (like in the aforementioned more secure scenarios) then it
> > would
> > > provide plenty of value.
> > >
> > >
> > >
> > >
> > > > On May 23, 2023, at 2:16 AM, Marcus Eagan <ma...@gmail.com>
> > wrote:
> > > >
> > > > Smiley, that's actually a pretty good use case.
> > > >
> > > > On Tue, May 23, 2023 at 2:15 AM David Smiley <ds...@apache.org>
> > wrote:
> > > >
> > > >> Glad to see we can limit this discussion to "bin/post" and not
> > > >> other non-prod/maybe-prod things listed.
> > > >>
> > > >> I think bin/post should either exist where it is or not exist;
> moving
> > > it to
> > > >> the sandbox misses the point.
> > > >> "curl" may very well be plenty adequate. FWIW your arguments work
> for
> > > me
> > > >> to ditch bin/post, but it could be useful to hear from
> > > educators/trainers
> > > >> as well. It's been a while since I could count myself as such.
> > > >>
> > > >> BTW I know of at least one advantage over curl. I was once posting
> > > massive
> > > >> many-gigabyte files to Solr in CSV (for a POC scenario). Curl
> wanted
> > to
> > > >> memory-map the whole thing and failed to get the RAM to do it.
> > > "bin/post"
> > > >> streamed the whole thing quite fast. I know I made improvements to
> > > help it
> > > >> in this specific regard. That said, if bin/post is mostly
> redundant,
> > I
> > > >> wouldn't want us to maintain bin/post forever *just* because of
> this.
> > > >>
> > > >> ~ David Smiley
> > > >> Apache Lucene/Solr Search Developer
> > > >> http://www.linkedin.com/in/davidwsmiley
> > > >>
> > > >>
> > > >> On Tue, May 23, 2023 at 2:05 AM Ishan Chattopadhyaya <
> > > >> ichattopadhyaya@gmail.com> wrote:
> > > >>
> > > >>> We have advertised the post tool in the past in our
> > examples/tutorials.
> > > >>> Difficult to erase all traces of it from the web.
> > > >>> It is better to remove the tool from the Solr distro, and, maybe,
> > > >> relocate
> > > >>> it to the sandbox/extras repo, from where a user can consciously
> > choose
> > > >> to
> > > >>> download and use it if needed.
> > > >>>
> > > >>> If we have proper documentation, I don't see how anyone building
> POCs
> > > >> will
> > > >>> miss out on the post tool. Is there a use case that users can't
> > achieve
> > > >>> easily through a documented method that the post tool caters to?
> > > >>>
> > > >>> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org>
> > wrote:
> > > >>>
> > > >>>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your
> > > point
> > > >>> of
> > > >>>> view... is the key word of your response "advertise" and we can
> then
> > > >>> debate
> > > >>>> what that means? In other words, are you saying bin/post (and
> other
> > > >>> things
> > > >>>> you don't think should be used in production) should not exist at
> > all
> > > >> or
> > > >>> we
> > > >>>> should be very careful about how we "advertise" its existence? I
> > > think
> > > >>>> features that help people build POCs / explore / learn are good
> > > things,
> > > >>>> even if not used typically in production. At times we
> > over-advertised
> > > >>>> features without enough disclaimers on the suitability of the
> > feature
> > > >> in
> > > >>>> question.
> > > >>>>
> > > >>>> ~ David Smiley
> > > >>>> Apache Lucene/Solr Search Developer
> > > >>>> http://www.linkedin.com/in/davidwsmiley
> > > >>>>
> > > >>>>
> > > >>>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
> > > >>>> ichattopadhyaya@gmail.com> wrote:
> > > >>>>
> > > >>>>>> Ishan, you did not give an argument for why you believe bin/post
> > > >>> should
> > > >>>>> go away. Do you feel it is better to document all the cURL
> commands
> > > >> to
> > > >>> be
> > > >>>>> more "standard"?
> > > >>>>>
> > > >>>>> Precisely! We should never advertise anything that shouldn't be
> > used
> > > >> at
> > > >>>>> scale or in production. That includes the post tool, example
> modes
> > of
> > > >>>>> startup, schemaless/data-driven etc.
> > > >>>>>
> > > >>>>> Curl is popular enough on Windows as well, through ports or WSL.
> > > >> There
> > > >>>> are
> > > >>>>> tools like Insomnia or Postman too. We should encourage their
> use,
> > > >>> rather
> > > >>>>> than promote our home grown tool by shipping oob and/or referring
> > to
> > > >> it
> > > >>>> in
> > > >>>>> tutorials.
> > > >>>>>
> > > >>>>> On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
> > > >>> wrote:
> > > >>>>>
> > > >>>>>> *If* bin/solr is to subsume bin/post, I think it deserves its
> own
> > > >>>> issue;
> > > >>>>>> should not be a detail of SOLR-6994 -- "Implement Windows
> version
> > > >> of
> > > >>>>>> bin/post". Really; wow that'd be sneaky to do something so
> > visible
> > > >>> to
> > > >>>>>> users / documentation under a OS/platform compatibility oriented
> > > >>> title.
> > > >>>>>>
> > > >>>>>> I am strongly not a fan of having bin/solr subsume bin/post.
> > > >>> bin/post
> > > >>>> is
> > > >>>>>> very distinct enough in purpose to remain separate. The stated
> > > >>>>> motivation
> > > >>>>>> seems out of convenience to Solr internal workings, which
> doesn't
> > > >>> serve
> > > >>>>> the
> > > >>>>>> user's best interests IMO.
> > > >>>>>>
> > > >>>>>> ~ David Smiley
> > > >>>>>> Apache Lucene/Solr Search Developer
> > > >>>>>> http://www.linkedin.com/in/davidwsmiley
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> > > >>>>> epugh@opensourceconnections.com
> > > >>>>>>>
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>>> Hi all,
> > > >>>>>>>
> > > >>>>>>> As part of SOLR-6994 I’m migrating the SimplePostTool to be
> part
> > > >> of
> > > >>>>>>> bin/solr commands.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> We document a number of example use cases:
> > > >>>>>>>
> > > >>>>>>> * JSON file: bin/solr post -url
> > > >>> http://localhost:8983/wizbang/update
> > > >>>>>>> events.json
> > > >>>>>>> * XML files: bin/solr post -url
> > > >>> http://localhost:8983/records/update
> > > >>>>>>> article*.xml
> > > >>>>>>> * CSV file: bin/solr post -url
> > > >>> http://localhost:8983/signals/update
> > > >>>>>>> LATEST-signals.csv
> > > >>>>>>> * Directory of files: bin/solr post -url
> > > >>>>>>> http://localhost:8983/myfiles/update ~/Documents
> > > >>>>>>> * Web crawl: bin/solr post -url
> > > >>>>>>> http://localhost:8983/gettingstarted/update
> > > >>> https://solr.apache.org/
> > > >>>>>>> -recursive 1 -delay 1
> > > >>>>>>> * Standard input (stdin): echo '{commit: {}}' | bin/solr post
> > > >> -url
> > > >>>>>>> http://localhost:8983/my_collection/update -type
> > > >> application/json
> > > >>>> -out
> > > >>>>>>> yes -d
> > > >>>>>>> * Data as string: bin/solr post -url
> > > >>>>>> http://localhost:8983/signals/update
> > > >>>>>>> -type text/csv -out yes -d $'id,value\n1,0.47'
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> The last two, stdin and data as a string, feel rather obscure
> to
> > > >>> me,
> > > >>>>> and
> > > >>>>>>> I’d like to not port them over to being supported by the
> bin/solr
> > > >>>> post
> > > >>>>>> tool
> > > >>>>>>> equivalent. Thoughts?
> > > >>>>>>>
> > > >>>>>>> Eric
> > > >>>>>>> _______________________
> > > >>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> > > >>>> 434.466.1467
> > > >>>>> |
> > > >>>>>>> http://www.opensourceconnections.com <
> > > >>>>>>> http://www.opensourceconnections.com/> | My Free/Busy <
> > > >>>>>>> http://tinyurl.com/eric-cal>
> > > >>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> This e-mail and all contents, including attachments, is
> > > >> considered
> > > >>> to
> > > >>>>> be
> > > >>>>>>> Company Confidential unless explicitly stated otherwise,
> > > >> regardless
> > > >>>> of
> > > >>>>>>> whether attachments are marked as such.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > > >
> > > >
> > > > --
> > > > Marcus Eagan
> > >
> > > _______________________
> > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> |
> > > http://www.opensourceconnections.com <
> > > http://www.opensourceconnections.com/> | My Free/Busy <
> > > http://tinyurl.com/eric-cal>
> > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> > >
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> > >
> > >
> > > This e-mail and all contents, including attachments, is considered to
> be
> > > Company Confidential unless explicitly stated otherwise, regardless of
> > > whether attachments are marked as such.
> > >
> > >
> >
>
--
Marcus Eagan
Re: Anyone using more obscure features of SimplePostTool?
Posted by Joel Bernstein <jo...@gmail.com>.
I have mixed feelings about this because I love command line tools. But I
haven't actually used the bin/post tool in years.
On Tue, May 23, 2023 at 10:13 AM David Smiley <ds...@apache.org> wrote:
> +1 to deprecate bin/post.
>
> Eric, RE the issue title... it's better. Still, if I were advocating for
> the work (and I'm definitely not), I'd have a dedicated issue nonetheless,
> may or may not even have a separate Windows issue if it gets fixed by side
> effect of moving the functionality.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Tue, May 23, 2023 at 6:05 AM Eric Pugh <epugh@opensourceconnections.com
> >
> wrote:
>
> > Lots of feedback!
> >
> > So, would renaming SOLR-6994 from "Implement Windows version of bin/post"
> > to “Implement Windows version of bin/post via implementing bin/solr post”
> > address your concern David?
> >
> > It’s both for Solr technical simplicity, but also to let Window’s users
> > actually have better support. In the not too distant future I want to
> > get BASIC and JWT auth across all our CLI tools and not having this one
> > orphaned “bin/post” and random checked in “post.jar” hanging out would
> help
> > that.
> >
> > I don’t think we can remove the capability to post documents completely,
> > that it has to be deprecated. And honestly, if the darn thing worked a
> > bit better (like in the aforementioned more secure scenarios) then it
> would
> > provide plenty of value.
> >
> >
> >
> >
> > > On May 23, 2023, at 2:16 AM, Marcus Eagan <ma...@gmail.com>
> wrote:
> > >
> > > Smiley, that's actually a pretty good use case.
> > >
> > > On Tue, May 23, 2023 at 2:15 AM David Smiley <ds...@apache.org>
> wrote:
> > >
> > >> Glad to see we can limit this discussion to "bin/post" and not
> > >> other non-prod/maybe-prod things listed.
> > >>
> > >> I think bin/post should either exist where it is or not exist; moving
> > it to
> > >> the sandbox misses the point.
> > >> "curl" may very well be plenty adequate. FWIW your arguments work for
> > me
> > >> to ditch bin/post, but it could be useful to hear from
> > educators/trainers
> > >> as well. It's been a while since I could count myself as such.
> > >>
> > >> BTW I know of at least one advantage over curl. I was once posting
> > massive
> > >> many-gigabyte files to Solr in CSV (for a POC scenario). Curl wanted
> to
> > >> memory-map the whole thing and failed to get the RAM to do it.
> > "bin/post"
> > >> streamed the whole thing quite fast. I know I made improvements to
> > help it
> > >> in this specific regard. That said, if bin/post is mostly redundant,
> I
> > >> wouldn't want us to maintain bin/post forever *just* because of this.
> > >>
> > >> ~ David Smiley
> > >> Apache Lucene/Solr Search Developer
> > >> http://www.linkedin.com/in/davidwsmiley
> > >>
> > >>
> > >> On Tue, May 23, 2023 at 2:05 AM Ishan Chattopadhyaya <
> > >> ichattopadhyaya@gmail.com> wrote:
> > >>
> > >>> We have advertised the post tool in the past in our
> examples/tutorials.
> > >>> Difficult to erase all traces of it from the web.
> > >>> It is better to remove the tool from the Solr distro, and, maybe,
> > >> relocate
> > >>> it to the sandbox/extras repo, from where a user can consciously
> choose
> > >> to
> > >>> download and use it if needed.
> > >>>
> > >>> If we have proper documentation, I don't see how anyone building POCs
> > >> will
> > >>> miss out on the post tool. Is there a use case that users can't
> achieve
> > >>> easily through a documented method that the post tool caters to?
> > >>>
> > >>> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org>
> wrote:
> > >>>
> > >>>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your
> > point
> > >>> of
> > >>>> view... is the key word of your response "advertise" and we can then
> > >>> debate
> > >>>> what that means? In other words, are you saying bin/post (and other
> > >>> things
> > >>>> you don't think should be used in production) should not exist at
> all
> > >> or
> > >>> we
> > >>>> should be very careful about how we "advertise" its existence? I
> > think
> > >>>> features that help people build POCs / explore / learn are good
> > things,
> > >>>> even if not used typically in production. At times we
> over-advertised
> > >>>> features without enough disclaimers on the suitability of the
> feature
> > >> in
> > >>>> question.
> > >>>>
> > >>>> ~ David Smiley
> > >>>> Apache Lucene/Solr Search Developer
> > >>>> http://www.linkedin.com/in/davidwsmiley
> > >>>>
> > >>>>
> > >>>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
> > >>>> ichattopadhyaya@gmail.com> wrote:
> > >>>>
> > >>>>>> Ishan, you did not give an argument for why you believe bin/post
> > >>> should
> > >>>>> go away. Do you feel it is better to document all the cURL commands
> > >> to
> > >>> be
> > >>>>> more "standard"?
> > >>>>>
> > >>>>> Precisely! We should never advertise anything that shouldn't be
> used
> > >> at
> > >>>>> scale or in production. That includes the post tool, example modes
> of
> > >>>>> startup, schemaless/data-driven etc.
> > >>>>>
> > >>>>> Curl is popular enough on Windows as well, through ports or WSL.
> > >> There
> > >>>> are
> > >>>>> tools like Insomnia or Postman too. We should encourage their use,
> > >>> rather
> > >>>>> than promote our home grown tool by shipping oob and/or referring
> to
> > >> it
> > >>>> in
> > >>>>> tutorials.
> > >>>>>
> > >>>>> On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
> > >>> wrote:
> > >>>>>
> > >>>>>> *If* bin/solr is to subsume bin/post, I think it deserves its own
> > >>>> issue;
> > >>>>>> should not be a detail of SOLR-6994 -- "Implement Windows version
> > >> of
> > >>>>>> bin/post". Really; wow that'd be sneaky to do something so
> visible
> > >>> to
> > >>>>>> users / documentation under a OS/platform compatibility oriented
> > >>> title.
> > >>>>>>
> > >>>>>> I am strongly not a fan of having bin/solr subsume bin/post.
> > >>> bin/post
> > >>>> is
> > >>>>>> very distinct enough in purpose to remain separate. The stated
> > >>>>> motivation
> > >>>>>> seems out of convenience to Solr internal workings, which doesn't
> > >>> serve
> > >>>>> the
> > >>>>>> user's best interests IMO.
> > >>>>>>
> > >>>>>> ~ David Smiley
> > >>>>>> Apache Lucene/Solr Search Developer
> > >>>>>> http://www.linkedin.com/in/davidwsmiley
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> > >>>>> epugh@opensourceconnections.com
> > >>>>>>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi all,
> > >>>>>>>
> > >>>>>>> As part of SOLR-6994 I’m migrating the SimplePostTool to be part
> > >> of
> > >>>>>>> bin/solr commands.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> We document a number of example use cases:
> > >>>>>>>
> > >>>>>>> * JSON file: bin/solr post -url
> > >>> http://localhost:8983/wizbang/update
> > >>>>>>> events.json
> > >>>>>>> * XML files: bin/solr post -url
> > >>> http://localhost:8983/records/update
> > >>>>>>> article*.xml
> > >>>>>>> * CSV file: bin/solr post -url
> > >>> http://localhost:8983/signals/update
> > >>>>>>> LATEST-signals.csv
> > >>>>>>> * Directory of files: bin/solr post -url
> > >>>>>>> http://localhost:8983/myfiles/update ~/Documents
> > >>>>>>> * Web crawl: bin/solr post -url
> > >>>>>>> http://localhost:8983/gettingstarted/update
> > >>> https://solr.apache.org/
> > >>>>>>> -recursive 1 -delay 1
> > >>>>>>> * Standard input (stdin): echo '{commit: {}}' | bin/solr post
> > >> -url
> > >>>>>>> http://localhost:8983/my_collection/update -type
> > >> application/json
> > >>>> -out
> > >>>>>>> yes -d
> > >>>>>>> * Data as string: bin/solr post -url
> > >>>>>> http://localhost:8983/signals/update
> > >>>>>>> -type text/csv -out yes -d $'id,value\n1,0.47'
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> The last two, stdin and data as a string, feel rather obscure to
> > >>> me,
> > >>>>> and
> > >>>>>>> I’d like to not port them over to being supported by the bin/solr
> > >>>> post
> > >>>>>> tool
> > >>>>>>> equivalent. Thoughts?
> > >>>>>>>
> > >>>>>>> Eric
> > >>>>>>> _______________________
> > >>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> > >>>> 434.466.1467
> > >>>>> |
> > >>>>>>> http://www.opensourceconnections.com <
> > >>>>>>> http://www.opensourceconnections.com/> | My Free/Busy <
> > >>>>>>> http://tinyurl.com/eric-cal>
> > >>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> This e-mail and all contents, including attachments, is
> > >> considered
> > >>> to
> > >>>>> be
> > >>>>>>> Company Confidential unless explicitly stated otherwise,
> > >> regardless
> > >>>> of
> > >>>>>>> whether attachments are marked as such.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >
> > >
> > > --
> > > Marcus Eagan
> >
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> > http://www.opensourceconnections.com <
> > http://www.opensourceconnections.com/> | My Free/Busy <
> > http://tinyurl.com/eric-cal>
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> >
> >
> > This e-mail and all contents, including attachments, is considered to be
> > Company Confidential unless explicitly stated otherwise, regardless of
> > whether attachments are marked as such.
> >
> >
>
Re: Anyone using more obscure features of SimplePostTool?
Posted by David Smiley <ds...@apache.org>.
+1 to deprecate bin/post.
Eric, RE the issue title... it's better. Still, if I were advocating for
the work (and I'm definitely not), I'd have a dedicated issue nonetheless,
may or may not even have a separate Windows issue if it gets fixed by side
effect of moving the functionality.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Tue, May 23, 2023 at 6:05 AM Eric Pugh <ep...@opensourceconnections.com>
wrote:
> Lots of feedback!
>
> So, would renaming SOLR-6994 from "Implement Windows version of bin/post"
> to “Implement Windows version of bin/post via implementing bin/solr post”
> address your concern David?
>
> It’s both for Solr technical simplicity, but also to let Window’s users
> actually have better support. In the not too distant future I want to
> get BASIC and JWT auth across all our CLI tools and not having this one
> orphaned “bin/post” and random checked in “post.jar” hanging out would help
> that.
>
> I don’t think we can remove the capability to post documents completely,
> that it has to be deprecated. And honestly, if the darn thing worked a
> bit better (like in the aforementioned more secure scenarios) then it would
> provide plenty of value.
>
>
>
>
> > On May 23, 2023, at 2:16 AM, Marcus Eagan <ma...@gmail.com> wrote:
> >
> > Smiley, that's actually a pretty good use case.
> >
> > On Tue, May 23, 2023 at 2:15 AM David Smiley <ds...@apache.org> wrote:
> >
> >> Glad to see we can limit this discussion to "bin/post" and not
> >> other non-prod/maybe-prod things listed.
> >>
> >> I think bin/post should either exist where it is or not exist; moving
> it to
> >> the sandbox misses the point.
> >> "curl" may very well be plenty adequate. FWIW your arguments work for
> me
> >> to ditch bin/post, but it could be useful to hear from
> educators/trainers
> >> as well. It's been a while since I could count myself as such.
> >>
> >> BTW I know of at least one advantage over curl. I was once posting
> massive
> >> many-gigabyte files to Solr in CSV (for a POC scenario). Curl wanted to
> >> memory-map the whole thing and failed to get the RAM to do it.
> "bin/post"
> >> streamed the whole thing quite fast. I know I made improvements to
> help it
> >> in this specific regard. That said, if bin/post is mostly redundant, I
> >> wouldn't want us to maintain bin/post forever *just* because of this.
> >>
> >> ~ David Smiley
> >> Apache Lucene/Solr Search Developer
> >> http://www.linkedin.com/in/davidwsmiley
> >>
> >>
> >> On Tue, May 23, 2023 at 2:05 AM Ishan Chattopadhyaya <
> >> ichattopadhyaya@gmail.com> wrote:
> >>
> >>> We have advertised the post tool in the past in our examples/tutorials.
> >>> Difficult to erase all traces of it from the web.
> >>> It is better to remove the tool from the Solr distro, and, maybe,
> >> relocate
> >>> it to the sandbox/extras repo, from where a user can consciously choose
> >> to
> >>> download and use it if needed.
> >>>
> >>> If we have proper documentation, I don't see how anyone building POCs
> >> will
> >>> miss out on the post tool. Is there a use case that users can't achieve
> >>> easily through a documented method that the post tool caters to?
> >>>
> >>> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org> wrote:
> >>>
> >>>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your
> point
> >>> of
> >>>> view... is the key word of your response "advertise" and we can then
> >>> debate
> >>>> what that means? In other words, are you saying bin/post (and other
> >>> things
> >>>> you don't think should be used in production) should not exist at all
> >> or
> >>> we
> >>>> should be very careful about how we "advertise" its existence? I
> think
> >>>> features that help people build POCs / explore / learn are good
> things,
> >>>> even if not used typically in production. At times we over-advertised
> >>>> features without enough disclaimers on the suitability of the feature
> >> in
> >>>> question.
> >>>>
> >>>> ~ David Smiley
> >>>> Apache Lucene/Solr Search Developer
> >>>> http://www.linkedin.com/in/davidwsmiley
> >>>>
> >>>>
> >>>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
> >>>> ichattopadhyaya@gmail.com> wrote:
> >>>>
> >>>>>> Ishan, you did not give an argument for why you believe bin/post
> >>> should
> >>>>> go away. Do you feel it is better to document all the cURL commands
> >> to
> >>> be
> >>>>> more "standard"?
> >>>>>
> >>>>> Precisely! We should never advertise anything that shouldn't be used
> >> at
> >>>>> scale or in production. That includes the post tool, example modes of
> >>>>> startup, schemaless/data-driven etc.
> >>>>>
> >>>>> Curl is popular enough on Windows as well, through ports or WSL.
> >> There
> >>>> are
> >>>>> tools like Insomnia or Postman too. We should encourage their use,
> >>> rather
> >>>>> than promote our home grown tool by shipping oob and/or referring to
> >> it
> >>>> in
> >>>>> tutorials.
> >>>>>
> >>>>> On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
> >>> wrote:
> >>>>>
> >>>>>> *If* bin/solr is to subsume bin/post, I think it deserves its own
> >>>> issue;
> >>>>>> should not be a detail of SOLR-6994 -- "Implement Windows version
> >> of
> >>>>>> bin/post". Really; wow that'd be sneaky to do something so visible
> >>> to
> >>>>>> users / documentation under a OS/platform compatibility oriented
> >>> title.
> >>>>>>
> >>>>>> I am strongly not a fan of having bin/solr subsume bin/post.
> >>> bin/post
> >>>> is
> >>>>>> very distinct enough in purpose to remain separate. The stated
> >>>>> motivation
> >>>>>> seems out of convenience to Solr internal workings, which doesn't
> >>> serve
> >>>>> the
> >>>>>> user's best interests IMO.
> >>>>>>
> >>>>>> ~ David Smiley
> >>>>>> Apache Lucene/Solr Search Developer
> >>>>>> http://www.linkedin.com/in/davidwsmiley
> >>>>>>
> >>>>>>
> >>>>>> On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> >>>>> epugh@opensourceconnections.com
> >>>>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> As part of SOLR-6994 I’m migrating the SimplePostTool to be part
> >> of
> >>>>>>> bin/solr commands.
> >>>>>>>
> >>>>>>>
> >>>>>>> We document a number of example use cases:
> >>>>>>>
> >>>>>>> * JSON file: bin/solr post -url
> >>> http://localhost:8983/wizbang/update
> >>>>>>> events.json
> >>>>>>> * XML files: bin/solr post -url
> >>> http://localhost:8983/records/update
> >>>>>>> article*.xml
> >>>>>>> * CSV file: bin/solr post -url
> >>> http://localhost:8983/signals/update
> >>>>>>> LATEST-signals.csv
> >>>>>>> * Directory of files: bin/solr post -url
> >>>>>>> http://localhost:8983/myfiles/update ~/Documents
> >>>>>>> * Web crawl: bin/solr post -url
> >>>>>>> http://localhost:8983/gettingstarted/update
> >>> https://solr.apache.org/
> >>>>>>> -recursive 1 -delay 1
> >>>>>>> * Standard input (stdin): echo '{commit: {}}' | bin/solr post
> >> -url
> >>>>>>> http://localhost:8983/my_collection/update -type
> >> application/json
> >>>> -out
> >>>>>>> yes -d
> >>>>>>> * Data as string: bin/solr post -url
> >>>>>> http://localhost:8983/signals/update
> >>>>>>> -type text/csv -out yes -d $'id,value\n1,0.47'
> >>>>>>>
> >>>>>>>
> >>>>>>> The last two, stdin and data as a string, feel rather obscure to
> >>> me,
> >>>>> and
> >>>>>>> I’d like to not port them over to being supported by the bin/solr
> >>>> post
> >>>>>> tool
> >>>>>>> equivalent. Thoughts?
> >>>>>>>
> >>>>>>> Eric
> >>>>>>> _______________________
> >>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> >>>> 434.466.1467
> >>>>> |
> >>>>>>> http://www.opensourceconnections.com <
> >>>>>>> http://www.opensourceconnections.com/> | My Free/Busy <
> >>>>>>> http://tinyurl.com/eric-cal>
> >>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> >>>>>>>
> >>>>>>>
> >>>>>>> This e-mail and all contents, including attachments, is
> >> considered
> >>> to
> >>>>> be
> >>>>>>> Company Confidential unless explicitly stated otherwise,
> >> regardless
> >>>> of
> >>>>>>> whether attachments are marked as such.
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
> >
> > --
> > Marcus Eagan
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>
Re: Anyone using more obscure features of SimplePostTool?
Posted by Eric Pugh <ep...@opensourceconnections.com>.
Lots of feedback!
So, would renaming SOLR-6994 from "Implement Windows version of bin/post" to “Implement Windows version of bin/post via implementing bin/solr post” address your concern David?
It’s both for Solr technical simplicity, but also to let Window’s users actually have better support. In the not too distant future I want to get BASIC and JWT auth across all our CLI tools and not having this one orphaned “bin/post” and random checked in “post.jar” hanging out would help that.
I don’t think we can remove the capability to post documents completely, that it has to be deprecated. And honestly, if the darn thing worked a bit better (like in the aforementioned more secure scenarios) then it would provide plenty of value.
> On May 23, 2023, at 2:16 AM, Marcus Eagan <ma...@gmail.com> wrote:
>
> Smiley, that's actually a pretty good use case.
>
> On Tue, May 23, 2023 at 2:15 AM David Smiley <ds...@apache.org> wrote:
>
>> Glad to see we can limit this discussion to "bin/post" and not
>> other non-prod/maybe-prod things listed.
>>
>> I think bin/post should either exist where it is or not exist; moving it to
>> the sandbox misses the point.
>> "curl" may very well be plenty adequate. FWIW your arguments work for me
>> to ditch bin/post, but it could be useful to hear from educators/trainers
>> as well. It's been a while since I could count myself as such.
>>
>> BTW I know of at least one advantage over curl. I was once posting massive
>> many-gigabyte files to Solr in CSV (for a POC scenario). Curl wanted to
>> memory-map the whole thing and failed to get the RAM to do it. "bin/post"
>> streamed the whole thing quite fast. I know I made improvements to help it
>> in this specific regard. That said, if bin/post is mostly redundant, I
>> wouldn't want us to maintain bin/post forever *just* because of this.
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Tue, May 23, 2023 at 2:05 AM Ishan Chattopadhyaya <
>> ichattopadhyaya@gmail.com> wrote:
>>
>>> We have advertised the post tool in the past in our examples/tutorials.
>>> Difficult to erase all traces of it from the web.
>>> It is better to remove the tool from the Solr distro, and, maybe,
>> relocate
>>> it to the sandbox/extras repo, from where a user can consciously choose
>> to
>>> download and use it if needed.
>>>
>>> If we have proper documentation, I don't see how anyone building POCs
>> will
>>> miss out on the post tool. Is there a use case that users can't achieve
>>> easily through a documented method that the post tool caters to?
>>>
>>> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org> wrote:
>>>
>>>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your point
>>> of
>>>> view... is the key word of your response "advertise" and we can then
>>> debate
>>>> what that means? In other words, are you saying bin/post (and other
>>> things
>>>> you don't think should be used in production) should not exist at all
>> or
>>> we
>>>> should be very careful about how we "advertise" its existence? I think
>>>> features that help people build POCs / explore / learn are good things,
>>>> even if not used typically in production. At times we over-advertised
>>>> features without enough disclaimers on the suitability of the feature
>> in
>>>> question.
>>>>
>>>> ~ David Smiley
>>>> Apache Lucene/Solr Search Developer
>>>> http://www.linkedin.com/in/davidwsmiley
>>>>
>>>>
>>>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
>>>> ichattopadhyaya@gmail.com> wrote:
>>>>
>>>>>> Ishan, you did not give an argument for why you believe bin/post
>>> should
>>>>> go away. Do you feel it is better to document all the cURL commands
>> to
>>> be
>>>>> more "standard"?
>>>>>
>>>>> Precisely! We should never advertise anything that shouldn't be used
>> at
>>>>> scale or in production. That includes the post tool, example modes of
>>>>> startup, schemaless/data-driven etc.
>>>>>
>>>>> Curl is popular enough on Windows as well, through ports or WSL.
>> There
>>>> are
>>>>> tools like Insomnia or Postman too. We should encourage their use,
>>> rather
>>>>> than promote our home grown tool by shipping oob and/or referring to
>> it
>>>> in
>>>>> tutorials.
>>>>>
>>>>> On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
>>> wrote:
>>>>>
>>>>>> *If* bin/solr is to subsume bin/post, I think it deserves its own
>>>> issue;
>>>>>> should not be a detail of SOLR-6994 -- "Implement Windows version
>> of
>>>>>> bin/post". Really; wow that'd be sneaky to do something so visible
>>> to
>>>>>> users / documentation under a OS/platform compatibility oriented
>>> title.
>>>>>>
>>>>>> I am strongly not a fan of having bin/solr subsume bin/post.
>>> bin/post
>>>> is
>>>>>> very distinct enough in purpose to remain separate. The stated
>>>>> motivation
>>>>>> seems out of convenience to Solr internal workings, which doesn't
>>> serve
>>>>> the
>>>>>> user's best interests IMO.
>>>>>>
>>>>>> ~ David Smiley
>>>>>> Apache Lucene/Solr Search Developer
>>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>>
>>>>>>
>>>>>> On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
>>>>> epugh@opensourceconnections.com
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> As part of SOLR-6994 I’m migrating the SimplePostTool to be part
>> of
>>>>>>> bin/solr commands.
>>>>>>>
>>>>>>>
>>>>>>> We document a number of example use cases:
>>>>>>>
>>>>>>> * JSON file: bin/solr post -url
>>> http://localhost:8983/wizbang/update
>>>>>>> events.json
>>>>>>> * XML files: bin/solr post -url
>>> http://localhost:8983/records/update
>>>>>>> article*.xml
>>>>>>> * CSV file: bin/solr post -url
>>> http://localhost:8983/signals/update
>>>>>>> LATEST-signals.csv
>>>>>>> * Directory of files: bin/solr post -url
>>>>>>> http://localhost:8983/myfiles/update ~/Documents
>>>>>>> * Web crawl: bin/solr post -url
>>>>>>> http://localhost:8983/gettingstarted/update
>>> https://solr.apache.org/
>>>>>>> -recursive 1 -delay 1
>>>>>>> * Standard input (stdin): echo '{commit: {}}' | bin/solr post
>> -url
>>>>>>> http://localhost:8983/my_collection/update -type
>> application/json
>>>> -out
>>>>>>> yes -d
>>>>>>> * Data as string: bin/solr post -url
>>>>>> http://localhost:8983/signals/update
>>>>>>> -type text/csv -out yes -d $'id,value\n1,0.47'
>>>>>>>
>>>>>>>
>>>>>>> The last two, stdin and data as a string, feel rather obscure to
>>> me,
>>>>> and
>>>>>>> I’d like to not port them over to being supported by the bin/solr
>>>> post
>>>>>> tool
>>>>>>> equivalent. Thoughts?
>>>>>>>
>>>>>>> Eric
>>>>>>> _______________________
>>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
>>>> 434.466.1467
>>>>> |
>>>>>>> http://www.opensourceconnections.com <
>>>>>>> http://www.opensourceconnections.com/> | My Free/Busy <
>>>>>>> http://tinyurl.com/eric-cal>
>>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
>>>>>>>
>>>>>>>
>>>>>>> This e-mail and all contents, including attachments, is
>> considered
>>> to
>>>>> be
>>>>>>> Company Confidential unless explicitly stated otherwise,
>> regardless
>>>> of
>>>>>>> whether attachments are marked as such.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
> --
> Marcus Eagan
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: Anyone using more obscure features of SimplePostTool?
Posted by Marcus Eagan <ma...@gmail.com>.
Smiley, that's actually a pretty good use case.
On Tue, May 23, 2023 at 2:15 AM David Smiley <ds...@apache.org> wrote:
> Glad to see we can limit this discussion to "bin/post" and not
> other non-prod/maybe-prod things listed.
>
> I think bin/post should either exist where it is or not exist; moving it to
> the sandbox misses the point.
> "curl" may very well be plenty adequate. FWIW your arguments work for me
> to ditch bin/post, but it could be useful to hear from educators/trainers
> as well. It's been a while since I could count myself as such.
>
> BTW I know of at least one advantage over curl. I was once posting massive
> many-gigabyte files to Solr in CSV (for a POC scenario). Curl wanted to
> memory-map the whole thing and failed to get the RAM to do it. "bin/post"
> streamed the whole thing quite fast. I know I made improvements to help it
> in this specific regard. That said, if bin/post is mostly redundant, I
> wouldn't want us to maintain bin/post forever *just* because of this.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Tue, May 23, 2023 at 2:05 AM Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
> > We have advertised the post tool in the past in our examples/tutorials.
> > Difficult to erase all traces of it from the web.
> > It is better to remove the tool from the Solr distro, and, maybe,
> relocate
> > it to the sandbox/extras repo, from where a user can consciously choose
> to
> > download and use it if needed.
> >
> > If we have proper documentation, I don't see how anyone building POCs
> will
> > miss out on the post tool. Is there a use case that users can't achieve
> > easily through a documented method that the post tool caters to?
> >
> > On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org> wrote:
> >
> > > Ishan, your beliefs surprise me. Maybe I'm misunderstanding your point
> > of
> > > view... is the key word of your response "advertise" and we can then
> > debate
> > > what that means? In other words, are you saying bin/post (and other
> > things
> > > you don't think should be used in production) should not exist at all
> or
> > we
> > > should be very careful about how we "advertise" its existence? I think
> > > features that help people build POCs / explore / learn are good things,
> > > even if not used typically in production. At times we over-advertised
> > > features without enough disclaimers on the suitability of the feature
> in
> > > question.
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> > >
> > > On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
> > > ichattopadhyaya@gmail.com> wrote:
> > >
> > > > > Ishan, you did not give an argument for why you believe bin/post
> > should
> > > > go away. Do you feel it is better to document all the cURL commands
> to
> > be
> > > > more "standard"?
> > > >
> > > > Precisely! We should never advertise anything that shouldn't be used
> at
> > > > scale or in production. That includes the post tool, example modes of
> > > > startup, schemaless/data-driven etc.
> > > >
> > > > Curl is popular enough on Windows as well, through ports or WSL.
> There
> > > are
> > > > tools like Insomnia or Postman too. We should encourage their use,
> > rather
> > > > than promote our home grown tool by shipping oob and/or referring to
> it
> > > in
> > > > tutorials.
> > > >
> > > > On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
> > wrote:
> > > >
> > > > > *If* bin/solr is to subsume bin/post, I think it deserves its own
> > > issue;
> > > > > should not be a detail of SOLR-6994 -- "Implement Windows version
> of
> > > > > bin/post". Really; wow that'd be sneaky to do something so visible
> > to
> > > > > users / documentation under a OS/platform compatibility oriented
> > title.
> > > > >
> > > > > I am strongly not a fan of having bin/solr subsume bin/post.
> > bin/post
> > > is
> > > > > very distinct enough in purpose to remain separate. The stated
> > > > motivation
> > > > > seems out of convenience to Solr internal workings, which doesn't
> > serve
> > > > the
> > > > > user's best interests IMO.
> > > > >
> > > > > ~ David Smiley
> > > > > Apache Lucene/Solr Search Developer
> > > > > http://www.linkedin.com/in/davidwsmiley
> > > > >
> > > > >
> > > > > On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> > > > epugh@opensourceconnections.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > As part of SOLR-6994 I’m migrating the SimplePostTool to be part
> of
> > > > > > bin/solr commands.
> > > > > >
> > > > > >
> > > > > > We document a number of example use cases:
> > > > > >
> > > > > > * JSON file: bin/solr post -url
> > http://localhost:8983/wizbang/update
> > > > > > events.json
> > > > > > * XML files: bin/solr post -url
> > http://localhost:8983/records/update
> > > > > > article*.xml
> > > > > > * CSV file: bin/solr post -url
> > http://localhost:8983/signals/update
> > > > > > LATEST-signals.csv
> > > > > > * Directory of files: bin/solr post -url
> > > > > > http://localhost:8983/myfiles/update ~/Documents
> > > > > > * Web crawl: bin/solr post -url
> > > > > > http://localhost:8983/gettingstarted/update
> > https://solr.apache.org/
> > > > > > -recursive 1 -delay 1
> > > > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post
> -url
> > > > > > http://localhost:8983/my_collection/update -type
> application/json
> > > -out
> > > > > > yes -d
> > > > > > * Data as string: bin/solr post -url
> > > > > http://localhost:8983/signals/update
> > > > > > -type text/csv -out yes -d $'id,value\n1,0.47'
> > > > > >
> > > > > >
> > > > > > The last two, stdin and data as a string, feel rather obscure to
> > me,
> > > > and
> > > > > > I’d like to not port them over to being supported by the bin/solr
> > > post
> > > > > tool
> > > > > > equivalent. Thoughts?
> > > > > >
> > > > > > Eric
> > > > > > _______________________
> > > > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> > > 434.466.1467
> > > > |
> > > > > > http://www.opensourceconnections.com <
> > > > > > http://www.opensourceconnections.com/> | My Free/Busy <
> > > > > > http://tinyurl.com/eric-cal>
> > > > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> > > > > >
> > > > > >
> > > > > > This e-mail and all contents, including attachments, is
> considered
> > to
> > > > be
> > > > > > Company Confidential unless explicitly stated otherwise,
> regardless
> > > of
> > > > > > whether attachments are marked as such.
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--
Marcus Eagan
Re: Anyone using more obscure features of SimplePostTool?
Posted by David Smiley <ds...@apache.org>.
Glad to see we can limit this discussion to "bin/post" and not
other non-prod/maybe-prod things listed.
I think bin/post should either exist where it is or not exist; moving it to
the sandbox misses the point.
"curl" may very well be plenty adequate. FWIW your arguments work for me
to ditch bin/post, but it could be useful to hear from educators/trainers
as well. It's been a while since I could count myself as such.
BTW I know of at least one advantage over curl. I was once posting massive
many-gigabyte files to Solr in CSV (for a POC scenario). Curl wanted to
memory-map the whole thing and failed to get the RAM to do it. "bin/post"
streamed the whole thing quite fast. I know I made improvements to help it
in this specific regard. That said, if bin/post is mostly redundant, I
wouldn't want us to maintain bin/post forever *just* because of this.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Tue, May 23, 2023 at 2:05 AM Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:
> We have advertised the post tool in the past in our examples/tutorials.
> Difficult to erase all traces of it from the web.
> It is better to remove the tool from the Solr distro, and, maybe, relocate
> it to the sandbox/extras repo, from where a user can consciously choose to
> download and use it if needed.
>
> If we have proper documentation, I don't see how anyone building POCs will
> miss out on the post tool. Is there a use case that users can't achieve
> easily through a documented method that the post tool caters to?
>
> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org> wrote:
>
> > Ishan, your beliefs surprise me. Maybe I'm misunderstanding your point
> of
> > view... is the key word of your response "advertise" and we can then
> debate
> > what that means? In other words, are you saying bin/post (and other
> things
> > you don't think should be used in production) should not exist at all or
> we
> > should be very careful about how we "advertise" its existence? I think
> > features that help people build POCs / explore / learn are good things,
> > even if not used typically in production. At times we over-advertised
> > features without enough disclaimers on the suitability of the feature in
> > question.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
> > ichattopadhyaya@gmail.com> wrote:
> >
> > > > Ishan, you did not give an argument for why you believe bin/post
> should
> > > go away. Do you feel it is better to document all the cURL commands to
> be
> > > more "standard"?
> > >
> > > Precisely! We should never advertise anything that shouldn't be used at
> > > scale or in production. That includes the post tool, example modes of
> > > startup, schemaless/data-driven etc.
> > >
> > > Curl is popular enough on Windows as well, through ports or WSL. There
> > are
> > > tools like Insomnia or Postman too. We should encourage their use,
> rather
> > > than promote our home grown tool by shipping oob and/or referring to it
> > in
> > > tutorials.
> > >
> > > On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
> wrote:
> > >
> > > > *If* bin/solr is to subsume bin/post, I think it deserves its own
> > issue;
> > > > should not be a detail of SOLR-6994 -- "Implement Windows version of
> > > > bin/post". Really; wow that'd be sneaky to do something so visible
> to
> > > > users / documentation under a OS/platform compatibility oriented
> title.
> > > >
> > > > I am strongly not a fan of having bin/solr subsume bin/post.
> bin/post
> > is
> > > > very distinct enough in purpose to remain separate. The stated
> > > motivation
> > > > seems out of convenience to Solr internal workings, which doesn't
> serve
> > > the
> > > > user's best interests IMO.
> > > >
> > > > ~ David Smiley
> > > > Apache Lucene/Solr Search Developer
> > > > http://www.linkedin.com/in/davidwsmiley
> > > >
> > > >
> > > > On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> > > epugh@opensourceconnections.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > As part of SOLR-6994 I’m migrating the SimplePostTool to be part of
> > > > > bin/solr commands.
> > > > >
> > > > >
> > > > > We document a number of example use cases:
> > > > >
> > > > > * JSON file: bin/solr post -url
> http://localhost:8983/wizbang/update
> > > > > events.json
> > > > > * XML files: bin/solr post -url
> http://localhost:8983/records/update
> > > > > article*.xml
> > > > > * CSV file: bin/solr post -url
> http://localhost:8983/signals/update
> > > > > LATEST-signals.csv
> > > > > * Directory of files: bin/solr post -url
> > > > > http://localhost:8983/myfiles/update ~/Documents
> > > > > * Web crawl: bin/solr post -url
> > > > > http://localhost:8983/gettingstarted/update
> https://solr.apache.org/
> > > > > -recursive 1 -delay 1
> > > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> > > > > http://localhost:8983/my_collection/update -type application/json
> > -out
> > > > > yes -d
> > > > > * Data as string: bin/solr post -url
> > > > http://localhost:8983/signals/update
> > > > > -type text/csv -out yes -d $'id,value\n1,0.47'
> > > > >
> > > > >
> > > > > The last two, stdin and data as a string, feel rather obscure to
> me,
> > > and
> > > > > I’d like to not port them over to being supported by the bin/solr
> > post
> > > > tool
> > > > > equivalent. Thoughts?
> > > > >
> > > > > Eric
> > > > > _______________________
> > > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> > 434.466.1467
> > > |
> > > > > http://www.opensourceconnections.com <
> > > > > http://www.opensourceconnections.com/> | My Free/Busy <
> > > > > http://tinyurl.com/eric-cal>
> > > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> > > > >
> > > >
> > >
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> > > > >
> > > > >
> > > > > This e-mail and all contents, including attachments, is considered
> to
> > > be
> > > > > Company Confidential unless explicitly stated otherwise, regardless
> > of
> > > > > whether attachments are marked as such.
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Anyone using more obscure features of SimplePostTool?
Posted by Gus Heck <gu...@gmail.com>.
I say this despite the fact I know it will break some folks because anyone
relying on post.jar for a non-trivial use case will either have such a
horrible experience that they eventually move to elastic/opensearch or they
will write their own code anyway. The harm to future users is heavier.
On Tue, May 23, 2023 at 9:58 AM Gus Heck <gu...@gmail.com> wrote:
> I know of divisions of very large companies that have specifically done
> this, and even a product that wrote their "integrate with solr" suggesting
> to use post.jar and then sold to customers saying that they supported solr
> integration.
>
> post.jar should die. The "stream a large file" use case should be handled,
> but I think the tool should be just for that and documented as only for
> that use case.
>
> On Tue, May 23, 2023 at 2:16 AM Marcus Eagan <ma...@gmail.com>
> wrote:
>
>> I agree with Ishan on the post tool, specifically. The problem may be that
>> people have built other tools and integrations around it.
>>
>> Postman has accelerated cURL's growth for another class of users. Look at
>> all this content about cURL and Postman if you are unfamiliar
>> <
>> https://www.google.com/search?q=curl+usage+postman&oq=curl+usage+postman&aqs=chrome..69i57j0i22i30l2j0i390i650l3.3310j0j7&sourceid=chrome&ie=UTF-8
>> >
>> .
>>
>>
>> On Tue, May 23, 2023 at 2:10 AM Ishan Chattopadhyaya <
>> ichattopadhyaya@gmail.com> wrote:
>>
>> > In fact, doing these via curl will be more instructional to a user who
>> is
>> > using Solr for the first time. He/she will likely already know how curl
>> > works, so will not assume there's magic happening under the hood in the
>> > post tool.
>> >
>> > On Tue, 23 May 2023 at 11:38, Ishan Chattopadhyaya <
>> > ichattopadhyaya@gmail.com> wrote:
>> >
>> > > > * JSON file: bin/solr post -url
>> http://localhost:8983/wizbang/update
>> > > events.json
>> > > > * XML files: bin/solr post -url
>> http://localhost:8983/records/update
>> > > article*.xml
>> > > > * CSV file: bin/solr post -url http://localhost:8983/signals/update
>> > > LATEST-signals.csv
>> > > > * Directory of files: bin/solr post -url
>> > > http://localhost:8983/myfiles/update ~/Documents
>> > > > * Web crawl: bin/solr post -url
>> > > http://localhost:8983/gettingstarted/update https://solr.apache.org/
>> > > -recursive 1 -delay 1
>> > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
>> > > http://localhost:8983/my_collection/update -type application/json
>> -out
>> > > yes -d
>> > > > * Data as string: bin/solr post -url
>> > > http://localhost:8983/signals/update -type text/csv -out yes -d
>> > > $'id,value\n1,0.47'
>> > >
>> > > AFAICT, most of them can be achieved through bash/curl with one
>> liners.
>> > > Bash is supported on Windows, Mac OS and GNU/Linux. Am I missing
>> > something?
>> > >
>> > > On Tue, 23 May 2023 at 11:34, Ishan Chattopadhyaya <
>> > > ichattopadhyaya@gmail.com> wrote:
>> > >
>> > >> We have advertised the post tool in the past in our
>> examples/tutorials.
>> > >> Difficult to erase all traces of it from the web.
>> > >> It is better to remove the tool from the Solr distro, and, maybe,
>> > >> relocate it to the sandbox/extras repo, from where a user can
>> > consciously
>> > >> choose to download and use it if needed.
>> > >>
>> > >> If we have proper documentation, I don't see how anyone building POCs
>> > >> will miss out on the post tool. Is there a use case that users can't
>> > >> achieve easily through a documented method that the post tool caters
>> to?
>> > >>
>> > >> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org>
>> wrote:
>> > >>
>> > >>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your
>> point
>> > >>> of
>> > >>> view... is the key word of your response "advertise" and we can then
>> > >>> debate
>> > >>> what that means? In other words, are you saying bin/post (and other
>> > >>> things
>> > >>> you don't think should be used in production) should not exist at
>> all
>> > or
>> > >>> we
>> > >>> should be very careful about how we "advertise" its existence? I
>> think
>> > >>> features that help people build POCs / explore / learn are good
>> things,
>> > >>> even if not used typically in production. At times we
>> over-advertised
>> > >>> features without enough disclaimers on the suitability of the
>> feature
>> > in
>> > >>> question.
>> > >>>
>> > >>> ~ David Smiley
>> > >>> Apache Lucene/Solr Search Developer
>> > >>> http://www.linkedin.com/in/davidwsmiley
>> > >>>
>> > >>>
>> > >>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
>> > >>> ichattopadhyaya@gmail.com> wrote:
>> > >>>
>> > >>> > > Ishan, you did not give an argument for why you believe bin/post
>> > >>> should
>> > >>> > go away. Do you feel it is better to document all the cURL
>> commands
>> > to
>> > >>> be
>> > >>> > more "standard"?
>> > >>> >
>> > >>> > Precisely! We should never advertise anything that shouldn't be
>> used
>> > at
>> > >>> > scale or in production. That includes the post tool, example
>> modes of
>> > >>> > startup, schemaless/data-driven etc.
>> > >>> >
>> > >>> > Curl is popular enough on Windows as well, through ports or WSL.
>> > There
>> > >>> are
>> > >>> > tools like Insomnia or Postman too. We should encourage their use,
>> > >>> rather
>> > >>> > than promote our home grown tool by shipping oob and/or referring
>> to
>> > >>> it in
>> > >>> > tutorials.
>> > >>> >
>> > >>> > On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
>> > >>> wrote:
>> > >>> >
>> > >>> > > *If* bin/solr is to subsume bin/post, I think it deserves its
>> own
>> > >>> issue;
>> > >>> > > should not be a detail of SOLR-6994 -- "Implement Windows
>> version
>> > of
>> > >>> > > bin/post". Really; wow that'd be sneaky to do something so
>> visible
>> > >>> to
>> > >>> > > users / documentation under a OS/platform compatibility oriented
>> > >>> title.
>> > >>> > >
>> > >>> > > I am strongly not a fan of having bin/solr subsume bin/post.
>> > >>> bin/post is
>> > >>> > > very distinct enough in purpose to remain separate. The stated
>> > >>> > motivation
>> > >>> > > seems out of convenience to Solr internal workings, which
>> doesn't
>> > >>> serve
>> > >>> > the
>> > >>> > > user's best interests IMO.
>> > >>> > >
>> > >>> > > ~ David Smiley
>> > >>> > > Apache Lucene/Solr Search Developer
>> > >>> > > http://www.linkedin.com/in/davidwsmiley
>> > >>> > >
>> > >>> > >
>> > >>> > > On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
>> > >>> > epugh@opensourceconnections.com
>> > >>> > > >
>> > >>> > > wrote:
>> > >>> > >
>> > >>> > > > Hi all,
>> > >>> > > >
>> > >>> > > > As part of SOLR-6994 I’m migrating the SimplePostTool to be
>> part
>> > of
>> > >>> > > > bin/solr commands.
>> > >>> > > >
>> > >>> > > >
>> > >>> > > > We document a number of example use cases:
>> > >>> > > >
>> > >>> > > > * JSON file: bin/solr post -url
>> > >>> http://localhost:8983/wizbang/update
>> > >>> > > > events.json
>> > >>> > > > * XML files: bin/solr post -url
>> > >>> http://localhost:8983/records/update
>> > >>> > > > article*.xml
>> > >>> > > > * CSV file: bin/solr post -url
>> > >>> http://localhost:8983/signals/update
>> > >>> > > > LATEST-signals.csv
>> > >>> > > > * Directory of files: bin/solr post -url
>> > >>> > > > http://localhost:8983/myfiles/update ~/Documents
>> > >>> > > > * Web crawl: bin/solr post -url
>> > >>> > > > http://localhost:8983/gettingstarted/update
>> > >>> https://solr.apache.org/
>> > >>> > > > -recursive 1 -delay 1
>> > >>> > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post
>> > -url
>> > >>> > > > http://localhost:8983/my_collection/update -type
>> > application/json
>> > >>> -out
>> > >>> > > > yes -d
>> > >>> > > > * Data as string: bin/solr post -url
>> > >>> > > http://localhost:8983/signals/update
>> > >>> > > > -type text/csv -out yes -d $'id,value\n1,0.47'
>> > >>> > > >
>> > >>> > > >
>> > >>> > > > The last two, stdin and data as a string, feel rather
>> obscure to
>> > >>> me,
>> > >>> > and
>> > >>> > > > I’d like to not port them over to being supported by the
>> bin/solr
>> > >>> post
>> > >>> > > tool
>> > >>> > > > equivalent. Thoughts?
>> > >>> > > >
>> > >>> > > > Eric
>> > >>> > > > _______________________
>> > >>> > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
>> > >>> 434.466.1467
>> > >>> > |
>> > >>> > > > http://www.opensourceconnections.com <
>> > >>> > > > http://www.opensourceconnections.com/> | My Free/Busy <
>> > >>> > > > http://tinyurl.com/eric-cal>
>> > >>> > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
>> > >>> > > >
>> > >>> > >
>> > >>> >
>> > >>>
>> >
>> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
>> > >>> > > >
>> > >>> > > >
>> > >>> > > > This e-mail and all contents, including attachments, is
>> > considered
>> > >>> to
>> > >>> > be
>> > >>> > > > Company Confidential unless explicitly stated otherwise,
>> > >>> regardless of
>> > >>> > > > whether attachments are marked as such.
>> > >>> > > >
>> > >>> > > >
>> > >>> > >
>> > >>> >
>> > >>>
>> > >>
>> >
>>
>>
>> --
>> Marcus Eagan
>>
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Anyone using more obscure features of SimplePostTool?
Posted by Gus Heck <gu...@gmail.com>.
I know of divisions of very large companies that have specifically done
this, and even a product that wrote their "integrate with solr" suggesting
to use post.jar and then sold to customers saying that they supported solr
integration.
post.jar should die. The "stream a large file" use case should be handled,
but I think the tool should be just for that and documented as only for
that use case.
On Tue, May 23, 2023 at 2:16 AM Marcus Eagan <ma...@gmail.com> wrote:
> I agree with Ishan on the post tool, specifically. The problem may be that
> people have built other tools and integrations around it.
>
> Postman has accelerated cURL's growth for another class of users. Look at
> all this content about cURL and Postman if you are unfamiliar
> <
> https://www.google.com/search?q=curl+usage+postman&oq=curl+usage+postman&aqs=chrome..69i57j0i22i30l2j0i390i650l3.3310j0j7&sourceid=chrome&ie=UTF-8
> >
> .
>
>
> On Tue, May 23, 2023 at 2:10 AM Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
> > In fact, doing these via curl will be more instructional to a user who is
> > using Solr for the first time. He/she will likely already know how curl
> > works, so will not assume there's magic happening under the hood in the
> > post tool.
> >
> > On Tue, 23 May 2023 at 11:38, Ishan Chattopadhyaya <
> > ichattopadhyaya@gmail.com> wrote:
> >
> > > > * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
> > > events.json
> > > > * XML files: bin/solr post -url http://localhost:8983/records/update
> > > article*.xml
> > > > * CSV file: bin/solr post -url http://localhost:8983/signals/update
> > > LATEST-signals.csv
> > > > * Directory of files: bin/solr post -url
> > > http://localhost:8983/myfiles/update ~/Documents
> > > > * Web crawl: bin/solr post -url
> > > http://localhost:8983/gettingstarted/update https://solr.apache.org/
> > > -recursive 1 -delay 1
> > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> > > http://localhost:8983/my_collection/update -type application/json -out
> > > yes -d
> > > > * Data as string: bin/solr post -url
> > > http://localhost:8983/signals/update -type text/csv -out yes -d
> > > $'id,value\n1,0.47'
> > >
> > > AFAICT, most of them can be achieved through bash/curl with one liners.
> > > Bash is supported on Windows, Mac OS and GNU/Linux. Am I missing
> > something?
> > >
> > > On Tue, 23 May 2023 at 11:34, Ishan Chattopadhyaya <
> > > ichattopadhyaya@gmail.com> wrote:
> > >
> > >> We have advertised the post tool in the past in our
> examples/tutorials.
> > >> Difficult to erase all traces of it from the web.
> > >> It is better to remove the tool from the Solr distro, and, maybe,
> > >> relocate it to the sandbox/extras repo, from where a user can
> > consciously
> > >> choose to download and use it if needed.
> > >>
> > >> If we have proper documentation, I don't see how anyone building POCs
> > >> will miss out on the post tool. Is there a use case that users can't
> > >> achieve easily through a documented method that the post tool caters
> to?
> > >>
> > >> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org>
> wrote:
> > >>
> > >>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your
> point
> > >>> of
> > >>> view... is the key word of your response "advertise" and we can then
> > >>> debate
> > >>> what that means? In other words, are you saying bin/post (and other
> > >>> things
> > >>> you don't think should be used in production) should not exist at all
> > or
> > >>> we
> > >>> should be very careful about how we "advertise" its existence? I
> think
> > >>> features that help people build POCs / explore / learn are good
> things,
> > >>> even if not used typically in production. At times we
> over-advertised
> > >>> features without enough disclaimers on the suitability of the feature
> > in
> > >>> question.
> > >>>
> > >>> ~ David Smiley
> > >>> Apache Lucene/Solr Search Developer
> > >>> http://www.linkedin.com/in/davidwsmiley
> > >>>
> > >>>
> > >>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
> > >>> ichattopadhyaya@gmail.com> wrote:
> > >>>
> > >>> > > Ishan, you did not give an argument for why you believe bin/post
> > >>> should
> > >>> > go away. Do you feel it is better to document all the cURL commands
> > to
> > >>> be
> > >>> > more "standard"?
> > >>> >
> > >>> > Precisely! We should never advertise anything that shouldn't be
> used
> > at
> > >>> > scale or in production. That includes the post tool, example modes
> of
> > >>> > startup, schemaless/data-driven etc.
> > >>> >
> > >>> > Curl is popular enough on Windows as well, through ports or WSL.
> > There
> > >>> are
> > >>> > tools like Insomnia or Postman too. We should encourage their use,
> > >>> rather
> > >>> > than promote our home grown tool by shipping oob and/or referring
> to
> > >>> it in
> > >>> > tutorials.
> > >>> >
> > >>> > On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
> > >>> wrote:
> > >>> >
> > >>> > > *If* bin/solr is to subsume bin/post, I think it deserves its own
> > >>> issue;
> > >>> > > should not be a detail of SOLR-6994 -- "Implement Windows version
> > of
> > >>> > > bin/post". Really; wow that'd be sneaky to do something so
> visible
> > >>> to
> > >>> > > users / documentation under a OS/platform compatibility oriented
> > >>> title.
> > >>> > >
> > >>> > > I am strongly not a fan of having bin/solr subsume bin/post.
> > >>> bin/post is
> > >>> > > very distinct enough in purpose to remain separate. The stated
> > >>> > motivation
> > >>> > > seems out of convenience to Solr internal workings, which doesn't
> > >>> serve
> > >>> > the
> > >>> > > user's best interests IMO.
> > >>> > >
> > >>> > > ~ David Smiley
> > >>> > > Apache Lucene/Solr Search Developer
> > >>> > > http://www.linkedin.com/in/davidwsmiley
> > >>> > >
> > >>> > >
> > >>> > > On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> > >>> > epugh@opensourceconnections.com
> > >>> > > >
> > >>> > > wrote:
> > >>> > >
> > >>> > > > Hi all,
> > >>> > > >
> > >>> > > > As part of SOLR-6994 I’m migrating the SimplePostTool to be
> part
> > of
> > >>> > > > bin/solr commands.
> > >>> > > >
> > >>> > > >
> > >>> > > > We document a number of example use cases:
> > >>> > > >
> > >>> > > > * JSON file: bin/solr post -url
> > >>> http://localhost:8983/wizbang/update
> > >>> > > > events.json
> > >>> > > > * XML files: bin/solr post -url
> > >>> http://localhost:8983/records/update
> > >>> > > > article*.xml
> > >>> > > > * CSV file: bin/solr post -url
> > >>> http://localhost:8983/signals/update
> > >>> > > > LATEST-signals.csv
> > >>> > > > * Directory of files: bin/solr post -url
> > >>> > > > http://localhost:8983/myfiles/update ~/Documents
> > >>> > > > * Web crawl: bin/solr post -url
> > >>> > > > http://localhost:8983/gettingstarted/update
> > >>> https://solr.apache.org/
> > >>> > > > -recursive 1 -delay 1
> > >>> > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post
> > -url
> > >>> > > > http://localhost:8983/my_collection/update -type
> > application/json
> > >>> -out
> > >>> > > > yes -d
> > >>> > > > * Data as string: bin/solr post -url
> > >>> > > http://localhost:8983/signals/update
> > >>> > > > -type text/csv -out yes -d $'id,value\n1,0.47'
> > >>> > > >
> > >>> > > >
> > >>> > > > The last two, stdin and data as a string, feel rather obscure
> to
> > >>> me,
> > >>> > and
> > >>> > > > I’d like to not port them over to being supported by the
> bin/solr
> > >>> post
> > >>> > > tool
> > >>> > > > equivalent. Thoughts?
> > >>> > > >
> > >>> > > > Eric
> > >>> > > > _______________________
> > >>> > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> > >>> 434.466.1467
> > >>> > |
> > >>> > > > http://www.opensourceconnections.com <
> > >>> > > > http://www.opensourceconnections.com/> | My Free/Busy <
> > >>> > > > http://tinyurl.com/eric-cal>
> > >>> > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> > >>> > > >
> > >>> > > >
> > >>> > > > This e-mail and all contents, including attachments, is
> > considered
> > >>> to
> > >>> > be
> > >>> > > > Company Confidential unless explicitly stated otherwise,
> > >>> regardless of
> > >>> > > > whether attachments are marked as such.
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> >
>
>
> --
> Marcus Eagan
>
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Anyone using more obscure features of SimplePostTool?
Posted by Marcus Eagan <ma...@gmail.com>.
I agree with Ishan on the post tool, specifically. The problem may be that
people have built other tools and integrations around it.
Postman has accelerated cURL's growth for another class of users. Look at
all this content about cURL and Postman if you are unfamiliar
<https://www.google.com/search?q=curl+usage+postman&oq=curl+usage+postman&aqs=chrome..69i57j0i22i30l2j0i390i650l3.3310j0j7&sourceid=chrome&ie=UTF-8>
.
On Tue, May 23, 2023 at 2:10 AM Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:
> In fact, doing these via curl will be more instructional to a user who is
> using Solr for the first time. He/she will likely already know how curl
> works, so will not assume there's magic happening under the hood in the
> post tool.
>
> On Tue, 23 May 2023 at 11:38, Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
> > > * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
> > events.json
> > > * XML files: bin/solr post -url http://localhost:8983/records/update
> > article*.xml
> > > * CSV file: bin/solr post -url http://localhost:8983/signals/update
> > LATEST-signals.csv
> > > * Directory of files: bin/solr post -url
> > http://localhost:8983/myfiles/update ~/Documents
> > > * Web crawl: bin/solr post -url
> > http://localhost:8983/gettingstarted/update https://solr.apache.org/
> > -recursive 1 -delay 1
> > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> > http://localhost:8983/my_collection/update -type application/json -out
> > yes -d
> > > * Data as string: bin/solr post -url
> > http://localhost:8983/signals/update -type text/csv -out yes -d
> > $'id,value\n1,0.47'
> >
> > AFAICT, most of them can be achieved through bash/curl with one liners.
> > Bash is supported on Windows, Mac OS and GNU/Linux. Am I missing
> something?
> >
> > On Tue, 23 May 2023 at 11:34, Ishan Chattopadhyaya <
> > ichattopadhyaya@gmail.com> wrote:
> >
> >> We have advertised the post tool in the past in our examples/tutorials.
> >> Difficult to erase all traces of it from the web.
> >> It is better to remove the tool from the Solr distro, and, maybe,
> >> relocate it to the sandbox/extras repo, from where a user can
> consciously
> >> choose to download and use it if needed.
> >>
> >> If we have proper documentation, I don't see how anyone building POCs
> >> will miss out on the post tool. Is there a use case that users can't
> >> achieve easily through a documented method that the post tool caters to?
> >>
> >> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org> wrote:
> >>
> >>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your point
> >>> of
> >>> view... is the key word of your response "advertise" and we can then
> >>> debate
> >>> what that means? In other words, are you saying bin/post (and other
> >>> things
> >>> you don't think should be used in production) should not exist at all
> or
> >>> we
> >>> should be very careful about how we "advertise" its existence? I think
> >>> features that help people build POCs / explore / learn are good things,
> >>> even if not used typically in production. At times we over-advertised
> >>> features without enough disclaimers on the suitability of the feature
> in
> >>> question.
> >>>
> >>> ~ David Smiley
> >>> Apache Lucene/Solr Search Developer
> >>> http://www.linkedin.com/in/davidwsmiley
> >>>
> >>>
> >>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
> >>> ichattopadhyaya@gmail.com> wrote:
> >>>
> >>> > > Ishan, you did not give an argument for why you believe bin/post
> >>> should
> >>> > go away. Do you feel it is better to document all the cURL commands
> to
> >>> be
> >>> > more "standard"?
> >>> >
> >>> > Precisely! We should never advertise anything that shouldn't be used
> at
> >>> > scale or in production. That includes the post tool, example modes of
> >>> > startup, schemaless/data-driven etc.
> >>> >
> >>> > Curl is popular enough on Windows as well, through ports or WSL.
> There
> >>> are
> >>> > tools like Insomnia or Postman too. We should encourage their use,
> >>> rather
> >>> > than promote our home grown tool by shipping oob and/or referring to
> >>> it in
> >>> > tutorials.
> >>> >
> >>> > On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
> >>> wrote:
> >>> >
> >>> > > *If* bin/solr is to subsume bin/post, I think it deserves its own
> >>> issue;
> >>> > > should not be a detail of SOLR-6994 -- "Implement Windows version
> of
> >>> > > bin/post". Really; wow that'd be sneaky to do something so visible
> >>> to
> >>> > > users / documentation under a OS/platform compatibility oriented
> >>> title.
> >>> > >
> >>> > > I am strongly not a fan of having bin/solr subsume bin/post.
> >>> bin/post is
> >>> > > very distinct enough in purpose to remain separate. The stated
> >>> > motivation
> >>> > > seems out of convenience to Solr internal workings, which doesn't
> >>> serve
> >>> > the
> >>> > > user's best interests IMO.
> >>> > >
> >>> > > ~ David Smiley
> >>> > > Apache Lucene/Solr Search Developer
> >>> > > http://www.linkedin.com/in/davidwsmiley
> >>> > >
> >>> > >
> >>> > > On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> >>> > epugh@opensourceconnections.com
> >>> > > >
> >>> > > wrote:
> >>> > >
> >>> > > > Hi all,
> >>> > > >
> >>> > > > As part of SOLR-6994 I’m migrating the SimplePostTool to be part
> of
> >>> > > > bin/solr commands.
> >>> > > >
> >>> > > >
> >>> > > > We document a number of example use cases:
> >>> > > >
> >>> > > > * JSON file: bin/solr post -url
> >>> http://localhost:8983/wizbang/update
> >>> > > > events.json
> >>> > > > * XML files: bin/solr post -url
> >>> http://localhost:8983/records/update
> >>> > > > article*.xml
> >>> > > > * CSV file: bin/solr post -url
> >>> http://localhost:8983/signals/update
> >>> > > > LATEST-signals.csv
> >>> > > > * Directory of files: bin/solr post -url
> >>> > > > http://localhost:8983/myfiles/update ~/Documents
> >>> > > > * Web crawl: bin/solr post -url
> >>> > > > http://localhost:8983/gettingstarted/update
> >>> https://solr.apache.org/
> >>> > > > -recursive 1 -delay 1
> >>> > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post
> -url
> >>> > > > http://localhost:8983/my_collection/update -type
> application/json
> >>> -out
> >>> > > > yes -d
> >>> > > > * Data as string: bin/solr post -url
> >>> > > http://localhost:8983/signals/update
> >>> > > > -type text/csv -out yes -d $'id,value\n1,0.47'
> >>> > > >
> >>> > > >
> >>> > > > The last two, stdin and data as a string, feel rather obscure to
> >>> me,
> >>> > and
> >>> > > > I’d like to not port them over to being supported by the bin/solr
> >>> post
> >>> > > tool
> >>> > > > equivalent. Thoughts?
> >>> > > >
> >>> > > > Eric
> >>> > > > _______________________
> >>> > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> >>> 434.466.1467
> >>> > |
> >>> > > > http://www.opensourceconnections.com <
> >>> > > > http://www.opensourceconnections.com/> | My Free/Busy <
> >>> > > > http://tinyurl.com/eric-cal>
> >>> > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> >>> > > >
> >>> > >
> >>> >
> >>>
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> >>> > > >
> >>> > > >
> >>> > > > This e-mail and all contents, including attachments, is
> considered
> >>> to
> >>> > be
> >>> > > > Company Confidential unless explicitly stated otherwise,
> >>> regardless of
> >>> > > > whether attachments are marked as such.
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
>
--
Marcus Eagan
Re: Anyone using more obscure features of SimplePostTool?
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
In fact, doing these via curl will be more instructional to a user who is
using Solr for the first time. He/she will likely already know how curl
works, so will not assume there's magic happening under the hood in the
post tool.
On Tue, 23 May 2023 at 11:38, Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:
> > * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
> events.json
> > * XML files: bin/solr post -url http://localhost:8983/records/update
> article*.xml
> > * CSV file: bin/solr post -url http://localhost:8983/signals/update
> LATEST-signals.csv
> > * Directory of files: bin/solr post -url
> http://localhost:8983/myfiles/update ~/Documents
> > * Web crawl: bin/solr post -url
> http://localhost:8983/gettingstarted/update https://solr.apache.org/
> -recursive 1 -delay 1
> > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> http://localhost:8983/my_collection/update -type application/json -out
> yes -d
> > * Data as string: bin/solr post -url
> http://localhost:8983/signals/update -type text/csv -out yes -d
> $'id,value\n1,0.47'
>
> AFAICT, most of them can be achieved through bash/curl with one liners.
> Bash is supported on Windows, Mac OS and GNU/Linux. Am I missing something?
>
> On Tue, 23 May 2023 at 11:34, Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
>> We have advertised the post tool in the past in our examples/tutorials.
>> Difficult to erase all traces of it from the web.
>> It is better to remove the tool from the Solr distro, and, maybe,
>> relocate it to the sandbox/extras repo, from where a user can consciously
>> choose to download and use it if needed.
>>
>> If we have proper documentation, I don't see how anyone building POCs
>> will miss out on the post tool. Is there a use case that users can't
>> achieve easily through a documented method that the post tool caters to?
>>
>> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org> wrote:
>>
>>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your point
>>> of
>>> view... is the key word of your response "advertise" and we can then
>>> debate
>>> what that means? In other words, are you saying bin/post (and other
>>> things
>>> you don't think should be used in production) should not exist at all or
>>> we
>>> should be very careful about how we "advertise" its existence? I think
>>> features that help people build POCs / explore / learn are good things,
>>> even if not used typically in production. At times we over-advertised
>>> features without enough disclaimers on the suitability of the feature in
>>> question.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
>>> ichattopadhyaya@gmail.com> wrote:
>>>
>>> > > Ishan, you did not give an argument for why you believe bin/post
>>> should
>>> > go away. Do you feel it is better to document all the cURL commands to
>>> be
>>> > more "standard"?
>>> >
>>> > Precisely! We should never advertise anything that shouldn't be used at
>>> > scale or in production. That includes the post tool, example modes of
>>> > startup, schemaless/data-driven etc.
>>> >
>>> > Curl is popular enough on Windows as well, through ports or WSL. There
>>> are
>>> > tools like Insomnia or Postman too. We should encourage their use,
>>> rather
>>> > than promote our home grown tool by shipping oob and/or referring to
>>> it in
>>> > tutorials.
>>> >
>>> > On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org>
>>> wrote:
>>> >
>>> > > *If* bin/solr is to subsume bin/post, I think it deserves its own
>>> issue;
>>> > > should not be a detail of SOLR-6994 -- "Implement Windows version of
>>> > > bin/post". Really; wow that'd be sneaky to do something so visible
>>> to
>>> > > users / documentation under a OS/platform compatibility oriented
>>> title.
>>> > >
>>> > > I am strongly not a fan of having bin/solr subsume bin/post.
>>> bin/post is
>>> > > very distinct enough in purpose to remain separate. The stated
>>> > motivation
>>> > > seems out of convenience to Solr internal workings, which doesn't
>>> serve
>>> > the
>>> > > user's best interests IMO.
>>> > >
>>> > > ~ David Smiley
>>> > > Apache Lucene/Solr Search Developer
>>> > > http://www.linkedin.com/in/davidwsmiley
>>> > >
>>> > >
>>> > > On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
>>> > epugh@opensourceconnections.com
>>> > > >
>>> > > wrote:
>>> > >
>>> > > > Hi all,
>>> > > >
>>> > > > As part of SOLR-6994 I’m migrating the SimplePostTool to be part of
>>> > > > bin/solr commands.
>>> > > >
>>> > > >
>>> > > > We document a number of example use cases:
>>> > > >
>>> > > > * JSON file: bin/solr post -url
>>> http://localhost:8983/wizbang/update
>>> > > > events.json
>>> > > > * XML files: bin/solr post -url
>>> http://localhost:8983/records/update
>>> > > > article*.xml
>>> > > > * CSV file: bin/solr post -url
>>> http://localhost:8983/signals/update
>>> > > > LATEST-signals.csv
>>> > > > * Directory of files: bin/solr post -url
>>> > > > http://localhost:8983/myfiles/update ~/Documents
>>> > > > * Web crawl: bin/solr post -url
>>> > > > http://localhost:8983/gettingstarted/update
>>> https://solr.apache.org/
>>> > > > -recursive 1 -delay 1
>>> > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
>>> > > > http://localhost:8983/my_collection/update -type application/json
>>> -out
>>> > > > yes -d
>>> > > > * Data as string: bin/solr post -url
>>> > > http://localhost:8983/signals/update
>>> > > > -type text/csv -out yes -d $'id,value\n1,0.47'
>>> > > >
>>> > > >
>>> > > > The last two, stdin and data as a string, feel rather obscure to
>>> me,
>>> > and
>>> > > > I’d like to not port them over to being supported by the bin/solr
>>> post
>>> > > tool
>>> > > > equivalent. Thoughts?
>>> > > >
>>> > > > Eric
>>> > > > _______________________
>>> > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
>>> 434.466.1467
>>> > |
>>> > > > http://www.opensourceconnections.com <
>>> > > > http://www.opensourceconnections.com/> | My Free/Busy <
>>> > > > http://tinyurl.com/eric-cal>
>>> > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
>>> > > >
>>> > >
>>> >
>>> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
>>> > > >
>>> > > >
>>> > > > This e-mail and all contents, including attachments, is considered
>>> to
>>> > be
>>> > > > Company Confidential unless explicitly stated otherwise,
>>> regardless of
>>> > > > whether attachments are marked as such.
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
>>
Re: Anyone using more obscure features of SimplePostTool?
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
> * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
events.json
> * XML files: bin/solr post -url http://localhost:8983/records/update
article*.xml
> * CSV file: bin/solr post -url http://localhost:8983/signals/update
LATEST-signals.csv
> * Directory of files: bin/solr post -url
http://localhost:8983/myfiles/update ~/Documents
> * Web crawl: bin/solr post -url
http://localhost:8983/gettingstarted/update https://solr.apache.org/
-recursive 1 -delay 1
> * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
http://localhost:8983/my_collection/update -type application/json -out yes
-d
> * Data as string: bin/solr post -url http://localhost:8983/signals/update
-type text/csv -out yes -d $'id,value\n1,0.47'
AFAICT, most of them can be achieved through bash/curl with one liners.
Bash is supported on Windows, Mac OS and GNU/Linux. Am I missing something?
On Tue, 23 May 2023 at 11:34, Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:
> We have advertised the post tool in the past in our examples/tutorials.
> Difficult to erase all traces of it from the web.
> It is better to remove the tool from the Solr distro, and, maybe, relocate
> it to the sandbox/extras repo, from where a user can consciously choose to
> download and use it if needed.
>
> If we have proper documentation, I don't see how anyone building POCs will
> miss out on the post tool. Is there a use case that users can't achieve
> easily through a documented method that the post tool caters to?
>
> On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org> wrote:
>
>> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your point of
>> view... is the key word of your response "advertise" and we can then
>> debate
>> what that means? In other words, are you saying bin/post (and other
>> things
>> you don't think should be used in production) should not exist at all or
>> we
>> should be very careful about how we "advertise" its existence? I think
>> features that help people build POCs / explore / learn are good things,
>> even if not used typically in production. At times we over-advertised
>> features without enough disclaimers on the suitability of the feature in
>> question.
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
>> ichattopadhyaya@gmail.com> wrote:
>>
>> > > Ishan, you did not give an argument for why you believe bin/post
>> should
>> > go away. Do you feel it is better to document all the cURL commands to
>> be
>> > more "standard"?
>> >
>> > Precisely! We should never advertise anything that shouldn't be used at
>> > scale or in production. That includes the post tool, example modes of
>> > startup, schemaless/data-driven etc.
>> >
>> > Curl is popular enough on Windows as well, through ports or WSL. There
>> are
>> > tools like Insomnia or Postman too. We should encourage their use,
>> rather
>> > than promote our home grown tool by shipping oob and/or referring to it
>> in
>> > tutorials.
>> >
>> > On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org> wrote:
>> >
>> > > *If* bin/solr is to subsume bin/post, I think it deserves its own
>> issue;
>> > > should not be a detail of SOLR-6994 -- "Implement Windows version of
>> > > bin/post". Really; wow that'd be sneaky to do something so visible to
>> > > users / documentation under a OS/platform compatibility oriented
>> title.
>> > >
>> > > I am strongly not a fan of having bin/solr subsume bin/post.
>> bin/post is
>> > > very distinct enough in purpose to remain separate. The stated
>> > motivation
>> > > seems out of convenience to Solr internal workings, which doesn't
>> serve
>> > the
>> > > user's best interests IMO.
>> > >
>> > > ~ David Smiley
>> > > Apache Lucene/Solr Search Developer
>> > > http://www.linkedin.com/in/davidwsmiley
>> > >
>> > >
>> > > On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
>> > epugh@opensourceconnections.com
>> > > >
>> > > wrote:
>> > >
>> > > > Hi all,
>> > > >
>> > > > As part of SOLR-6994 I’m migrating the SimplePostTool to be part of
>> > > > bin/solr commands.
>> > > >
>> > > >
>> > > > We document a number of example use cases:
>> > > >
>> > > > * JSON file: bin/solr post -url
>> http://localhost:8983/wizbang/update
>> > > > events.json
>> > > > * XML files: bin/solr post -url
>> http://localhost:8983/records/update
>> > > > article*.xml
>> > > > * CSV file: bin/solr post -url http://localhost:8983/signals/update
>> > > > LATEST-signals.csv
>> > > > * Directory of files: bin/solr post -url
>> > > > http://localhost:8983/myfiles/update ~/Documents
>> > > > * Web crawl: bin/solr post -url
>> > > > http://localhost:8983/gettingstarted/update
>> https://solr.apache.org/
>> > > > -recursive 1 -delay 1
>> > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
>> > > > http://localhost:8983/my_collection/update -type application/json
>> -out
>> > > > yes -d
>> > > > * Data as string: bin/solr post -url
>> > > http://localhost:8983/signals/update
>> > > > -type text/csv -out yes -d $'id,value\n1,0.47'
>> > > >
>> > > >
>> > > > The last two, stdin and data as a string, feel rather obscure to
>> me,
>> > and
>> > > > I’d like to not port them over to being supported by the bin/solr
>> post
>> > > tool
>> > > > equivalent. Thoughts?
>> > > >
>> > > > Eric
>> > > > _______________________
>> > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
>> 434.466.1467
>> > |
>> > > > http://www.opensourceconnections.com <
>> > > > http://www.opensourceconnections.com/> | My Free/Busy <
>> > > > http://tinyurl.com/eric-cal>
>> > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
>> > > >
>> > >
>> >
>> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
>> > > >
>> > > >
>> > > > This e-mail and all contents, including attachments, is considered
>> to
>> > be
>> > > > Company Confidential unless explicitly stated otherwise, regardless
>> of
>> > > > whether attachments are marked as such.
>> > > >
>> > > >
>> > >
>> >
>>
>
Re: Anyone using more obscure features of SimplePostTool?
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
We have advertised the post tool in the past in our examples/tutorials.
Difficult to erase all traces of it from the web.
It is better to remove the tool from the Solr distro, and, maybe, relocate
it to the sandbox/extras repo, from where a user can consciously choose to
download and use it if needed.
If we have proper documentation, I don't see how anyone building POCs will
miss out on the post tool. Is there a use case that users can't achieve
easily through a documented method that the post tool caters to?
On Tue, 23 May 2023 at 11:25, David Smiley <ds...@apache.org> wrote:
> Ishan, your beliefs surprise me. Maybe I'm misunderstanding your point of
> view... is the key word of your response "advertise" and we can then debate
> what that means? In other words, are you saying bin/post (and other things
> you don't think should be used in production) should not exist at all or we
> should be very careful about how we "advertise" its existence? I think
> features that help people build POCs / explore / learn are good things,
> even if not used typically in production. At times we over-advertised
> features without enough disclaimers on the suitability of the feature in
> question.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
> ichattopadhyaya@gmail.com> wrote:
>
> > > Ishan, you did not give an argument for why you believe bin/post should
> > go away. Do you feel it is better to document all the cURL commands to be
> > more "standard"?
> >
> > Precisely! We should never advertise anything that shouldn't be used at
> > scale or in production. That includes the post tool, example modes of
> > startup, schemaless/data-driven etc.
> >
> > Curl is popular enough on Windows as well, through ports or WSL. There
> are
> > tools like Insomnia or Postman too. We should encourage their use, rather
> > than promote our home grown tool by shipping oob and/or referring to it
> in
> > tutorials.
> >
> > On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org> wrote:
> >
> > > *If* bin/solr is to subsume bin/post, I think it deserves its own
> issue;
> > > should not be a detail of SOLR-6994 -- "Implement Windows version of
> > > bin/post". Really; wow that'd be sneaky to do something so visible to
> > > users / documentation under a OS/platform compatibility oriented title.
> > >
> > > I am strongly not a fan of having bin/solr subsume bin/post. bin/post
> is
> > > very distinct enough in purpose to remain separate. The stated
> > motivation
> > > seems out of convenience to Solr internal workings, which doesn't serve
> > the
> > > user's best interests IMO.
> > >
> > > ~ David Smiley
> > > Apache Lucene/Solr Search Developer
> > > http://www.linkedin.com/in/davidwsmiley
> > >
> > >
> > > On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> > epugh@opensourceconnections.com
> > > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > As part of SOLR-6994 I’m migrating the SimplePostTool to be part of
> > > > bin/solr commands.
> > > >
> > > >
> > > > We document a number of example use cases:
> > > >
> > > > * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
> > > > events.json
> > > > * XML files: bin/solr post -url http://localhost:8983/records/update
> > > > article*.xml
> > > > * CSV file: bin/solr post -url http://localhost:8983/signals/update
> > > > LATEST-signals.csv
> > > > * Directory of files: bin/solr post -url
> > > > http://localhost:8983/myfiles/update ~/Documents
> > > > * Web crawl: bin/solr post -url
> > > > http://localhost:8983/gettingstarted/update https://solr.apache.org/
> > > > -recursive 1 -delay 1
> > > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> > > > http://localhost:8983/my_collection/update -type application/json
> -out
> > > > yes -d
> > > > * Data as string: bin/solr post -url
> > > http://localhost:8983/signals/update
> > > > -type text/csv -out yes -d $'id,value\n1,0.47'
> > > >
> > > >
> > > > The last two, stdin and data as a string, feel rather obscure to me,
> > and
> > > > I’d like to not port them over to being supported by the bin/solr
> post
> > > tool
> > > > equivalent. Thoughts?
> > > >
> > > > Eric
> > > > _______________________
> > > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> 434.466.1467
> > |
> > > > http://www.opensourceconnections.com <
> > > > http://www.opensourceconnections.com/> | My Free/Busy <
> > > > http://tinyurl.com/eric-cal>
> > > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> > > >
> > >
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> > > >
> > > >
> > > > This e-mail and all contents, including attachments, is considered to
> > be
> > > > Company Confidential unless explicitly stated otherwise, regardless
> of
> > > > whether attachments are marked as such.
> > > >
> > > >
> > >
> >
>
Re: Anyone using more obscure features of SimplePostTool?
Posted by David Smiley <ds...@apache.org>.
Ishan, your beliefs surprise me. Maybe I'm misunderstanding your point of
view... is the key word of your response "advertise" and we can then debate
what that means? In other words, are you saying bin/post (and other things
you don't think should be used in production) should not exist at all or we
should be very careful about how we "advertise" its existence? I think
features that help people build POCs / explore / learn are good things,
even if not used typically in production. At times we over-advertised
features without enough disclaimers on the suitability of the feature in
question.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Mon, May 22, 2023 at 10:09 PM Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:
> > Ishan, you did not give an argument for why you believe bin/post should
> go away. Do you feel it is better to document all the cURL commands to be
> more "standard"?
>
> Precisely! We should never advertise anything that shouldn't be used at
> scale or in production. That includes the post tool, example modes of
> startup, schemaless/data-driven etc.
>
> Curl is popular enough on Windows as well, through ports or WSL. There are
> tools like Insomnia or Postman too. We should encourage their use, rather
> than promote our home grown tool by shipping oob and/or referring to it in
> tutorials.
>
> On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org> wrote:
>
> > *If* bin/solr is to subsume bin/post, I think it deserves its own issue;
> > should not be a detail of SOLR-6994 -- "Implement Windows version of
> > bin/post". Really; wow that'd be sneaky to do something so visible to
> > users / documentation under a OS/platform compatibility oriented title.
> >
> > I am strongly not a fan of having bin/solr subsume bin/post. bin/post is
> > very distinct enough in purpose to remain separate. The stated
> motivation
> > seems out of convenience to Solr internal workings, which doesn't serve
> the
> > user's best interests IMO.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Wed, May 17, 2023 at 7:33 AM Eric Pugh <
> epugh@opensourceconnections.com
> > >
> > wrote:
> >
> > > Hi all,
> > >
> > > As part of SOLR-6994 I’m migrating the SimplePostTool to be part of
> > > bin/solr commands.
> > >
> > >
> > > We document a number of example use cases:
> > >
> > > * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
> > > events.json
> > > * XML files: bin/solr post -url http://localhost:8983/records/update
> > > article*.xml
> > > * CSV file: bin/solr post -url http://localhost:8983/signals/update
> > > LATEST-signals.csv
> > > * Directory of files: bin/solr post -url
> > > http://localhost:8983/myfiles/update ~/Documents
> > > * Web crawl: bin/solr post -url
> > > http://localhost:8983/gettingstarted/update https://solr.apache.org/
> > > -recursive 1 -delay 1
> > > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> > > http://localhost:8983/my_collection/update -type application/json -out
> > > yes -d
> > > * Data as string: bin/solr post -url
> > http://localhost:8983/signals/update
> > > -type text/csv -out yes -d $'id,value\n1,0.47'
> > >
> > >
> > > The last two, stdin and data as a string, feel rather obscure to me,
> and
> > > I’d like to not port them over to being supported by the bin/solr post
> > tool
> > > equivalent. Thoughts?
> > >
> > > Eric
> > > _______________________
> > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> |
> > > http://www.opensourceconnections.com <
> > > http://www.opensourceconnections.com/> | My Free/Busy <
> > > http://tinyurl.com/eric-cal>
> > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> > >
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> > >
> > >
> > > This e-mail and all contents, including attachments, is considered to
> be
> > > Company Confidential unless explicitly stated otherwise, regardless of
> > > whether attachments are marked as such.
> > >
> > >
> >
>
Re: Anyone using more obscure features of SimplePostTool?
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
> Ishan, you did not give an argument for why you believe bin/post should
go away. Do you feel it is better to document all the cURL commands to be
more "standard"?
Precisely! We should never advertise anything that shouldn't be used at
scale or in production. That includes the post tool, example modes of
startup, schemaless/data-driven etc.
Curl is popular enough on Windows as well, through ports or WSL. There are
tools like Insomnia or Postman too. We should encourage their use, rather
than promote our home grown tool by shipping oob and/or referring to it in
tutorials.
On Tue, 23 May, 2023, 3:26 am David Smiley, <ds...@apache.org> wrote:
> *If* bin/solr is to subsume bin/post, I think it deserves its own issue;
> should not be a detail of SOLR-6994 -- "Implement Windows version of
> bin/post". Really; wow that'd be sneaky to do something so visible to
> users / documentation under a OS/platform compatibility oriented title.
>
> I am strongly not a fan of having bin/solr subsume bin/post. bin/post is
> very distinct enough in purpose to remain separate. The stated motivation
> seems out of convenience to Solr internal workings, which doesn't serve the
> user's best interests IMO.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Wed, May 17, 2023 at 7:33 AM Eric Pugh <epugh@opensourceconnections.com
> >
> wrote:
>
> > Hi all,
> >
> > As part of SOLR-6994 I’m migrating the SimplePostTool to be part of
> > bin/solr commands.
> >
> >
> > We document a number of example use cases:
> >
> > * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
> > events.json
> > * XML files: bin/solr post -url http://localhost:8983/records/update
> > article*.xml
> > * CSV file: bin/solr post -url http://localhost:8983/signals/update
> > LATEST-signals.csv
> > * Directory of files: bin/solr post -url
> > http://localhost:8983/myfiles/update ~/Documents
> > * Web crawl: bin/solr post -url
> > http://localhost:8983/gettingstarted/update https://solr.apache.org/
> > -recursive 1 -delay 1
> > * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> > http://localhost:8983/my_collection/update -type application/json -out
> > yes -d
> > * Data as string: bin/solr post -url
> http://localhost:8983/signals/update
> > -type text/csv -out yes -d $'id,value\n1,0.47'
> >
> >
> > The last two, stdin and data as a string, feel rather obscure to me, and
> > I’d like to not port them over to being supported by the bin/solr post
> tool
> > equivalent. Thoughts?
> >
> > Eric
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> > http://www.opensourceconnections.com <
> > http://www.opensourceconnections.com/> | My Free/Busy <
> > http://tinyurl.com/eric-cal>
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> >
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> >
> >
> > This e-mail and all contents, including attachments, is considered to be
> > Company Confidential unless explicitly stated otherwise, regardless of
> > whether attachments are marked as such.
> >
> >
>
Re: Anyone using more obscure features of SimplePostTool?
Posted by David Smiley <ds...@apache.org>.
*If* bin/solr is to subsume bin/post, I think it deserves its own issue;
should not be a detail of SOLR-6994 -- "Implement Windows version of
bin/post". Really; wow that'd be sneaky to do something so visible to
users / documentation under a OS/platform compatibility oriented title.
I am strongly not a fan of having bin/solr subsume bin/post. bin/post is
very distinct enough in purpose to remain separate. The stated motivation
seems out of convenience to Solr internal workings, which doesn't serve the
user's best interests IMO.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Wed, May 17, 2023 at 7:33 AM Eric Pugh <ep...@opensourceconnections.com>
wrote:
> Hi all,
>
> As part of SOLR-6994 I’m migrating the SimplePostTool to be part of
> bin/solr commands.
>
>
> We document a number of example use cases:
>
> * JSON file: bin/solr post -url http://localhost:8983/wizbang/update
> events.json
> * XML files: bin/solr post -url http://localhost:8983/records/update
> article*.xml
> * CSV file: bin/solr post -url http://localhost:8983/signals/update
> LATEST-signals.csv
> * Directory of files: bin/solr post -url
> http://localhost:8983/myfiles/update ~/Documents
> * Web crawl: bin/solr post -url
> http://localhost:8983/gettingstarted/update https://solr.apache.org/
> -recursive 1 -delay 1
> * Standard input (stdin): echo '{commit: {}}' | bin/solr post -url
> http://localhost:8983/my_collection/update -type application/json -out
> yes -d
> * Data as string: bin/solr post -url http://localhost:8983/signals/update
> -type text/csv -out yes -d $'id,value\n1,0.47'
>
>
> The last two, stdin and data as a string, feel rather obscure to me, and
> I’d like to not port them over to being supported by the bin/solr post tool
> equivalent. Thoughts?
>
> Eric
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>