You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by Jason Gerlowski <ge...@gmail.com> on 2022/06/16 11:03:40 UTC
v2 API "Broader Changes" and Next Steps
Hi all,
I've been working over the last few months to get our "v2" API into
shape to eventually replace (and allow us to sunset) v1. At a high
level the two biggest next-steps are to (1) make whatever changes we'd
like to the v2 endpoints while they're still "experimental", and then
(2) start using them across Solr (Admin UI, SolrJ, etc.)
The big question in my mind is: what all should change in the v2
endpoints as part of item #1 above. Do people like the current v2
design enough to move forward as-is (or with some minor tweaking)? Or
is there interest in seeing a larger re-alignment around other design
principles (e.g. REST, OpenAPI, etc.) while v2 is still
"experimental"?
My impression from past discussions is that there's interest in seeing
broader changes to v2 (see [1] and [2] below). I tend to agree and
I'm willing to head up that effort. But I wanted to double-check
first that there was consensus on the overall idea of "broader
changes" before I start filing JIRAs and working out specifics.
Assuming consensus, my next step will be to write up the current v2
endpoints and some proposed changes in a spreadsheet that we can
iterate on. (Feedback on specific APIs, changes, etc. definitely
welcome at that point once I get there.)
Best,
Jason
[1] "Should v2 API be experimental" -
https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
[2] "Solr 9.0 Release Blockers"" -
https://lists.apache.org/thread/m7k2gvgxldkns7jqjnw1ghhqx7s3tpl1
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Eric Pugh <ep...@opensourceconnections.com>.
The direction that we are headed for how you interact w Solr is via the
admin UI. I am thinking of the security and schema designer additions for
example. And some outstanding pr’s to add support for paramsets to the
Admin UI.
Personally I always found using the browser to be very clunky and dangerous
for issuing anything beyond the simplest commands.
We should strive for better tooling, either admin UI like we have today
covering new features, or maybe include a nice api execution UI like kibana
does? A stepping stone to that would be if our api are OpenApi compliant
and Solr exposed a swagger type UI.
On Thu, Jun 16, 2022 at 9:07 AM Gus Heck <gu...@gmail.com> wrote:
> I'm all for standardizing on v2 (or something like it) but one key concern
> I have is that when the only access I have to a client's server is via a
> web browser, possibly from a machine they control and on which I can't
> install tools, I'd like the only barrier to my running necessary admin
> commands is their (hopefully) configured security controls
> (RBAC/JWT/whatever). It's a loss of functionality if a REST client program,
> plugin or curl is *required*. Those tools are good things, but the ability
> to fully control solr directly from a browser (if properly authenticated)
> is a good feature we shouldn't lose.
>
> Presently JSON based commands can be issued in the "documents" page, but
> some attention should be given to make sure we don't lose or even maybe
> enhance/rename that interface. Obviously, in the near future V1 will
> remain, but if deprecated, the implication is that someday (10.0? 11.0? it
> might go away).
>
> On Thu, Jun 16, 2022 at 10:30 AM Jan Høydahl <ja...@cominvent.com>
> wrote:
>
>> V2 has existed since v6.5 and has been documented in ref guide evner
>> since. In 7.0 release announcement we said:
>>
>>
>> -
>>
>> The new v2 API, exposed at /api/ and also supported via SolrJ, is now
>> the preferred API, but /solr/ continues to work.
>>
>> We have been showing v2 api commands in tabs for e.g. config api since
>> 7.4. So users have been encouraged to switch for at least 3-4 years.
>>
>> Jan Høydahl
>>
>> 16. jun. 2022 kl. 16:16 skrev Eric Pugh <epugh@opensourceconnections.com
>> >:
>>
>>
>> I don’t have the link, but I think we agree that v2 *could* evolve, hence
>> the whole experimental thing.
>>
>> I worry that the efforts of maintaining three versions of the api will
>> stall us. Especially if the changes we want to make aren’t wholesale
>> changes. Now if we introduced a graphql based api, that might justify
>> the v3 tag.
>>
>> My gut feeling is that no one uses v2 yet, so the end user impact is
>> minimal.
>>
>> Maybe run a quick survey to decide that?
>>
>> On Thu, Jun 16, 2022 at 5:41 AM Jan Høydahl <ja...@cominvent.com>
>> wrote:
>>
>>> HI,
>>>
>>> Thanks for having the guts to start such a journey!
>>> V2 has been around for several major releases and documented (it's in
>>> v6.6 guide https://solr.apache.org/guide/6_6/v2-api.html) long enough
>>> that we must assume a certain user base already.
>>> The "experimental" warning in RefGuide is not prominent between 6.6 and
>>> 8.11, looks like it was added in the 9.0 guide.
>>> Thus I'm not sure we should act as if the entire v2 design is up for
>>> change.
>>>
>>> So perhaps the next-gen, OpenApi style APIs should be v3 and then
>>> deprecate both v1 and v2 once the coverage is sufficient? I assume v3 could
>>> still stand on v2's sholders?
>>> Or if you plan to target a v2 rewrite for 10.0 only then I suppose it
>>> could be a rewrite of v2 since it will be in a major version, but then we
>>> need to know we can complete the work by then.
>>>
>>> Jan
>>>
>>> 16. jun. 2022 kl. 13:03 skrev Jason Gerlowski <ge...@gmail.com>:
>>>
>>> Hi all,
>>>
>>> I've been working over the last few months to get our "v2" API into
>>> shape to eventually replace (and allow us to sunset) v1. At a high
>>> level the two biggest next-steps are to (1) make whatever changes we'd
>>> like to the v2 endpoints while they're still "experimental", and then
>>> (2) start using them across Solr (Admin UI, SolrJ, etc.)
>>>
>>> The big question in my mind is: what all should change in the v2
>>> endpoints as part of item #1 above. Do people like the current v2
>>> design enough to move forward as-is (or with some minor tweaking)? Or
>>> is there interest in seeing a larger re-alignment around other design
>>> principles (e.g. REST, OpenAPI, etc.) while v2 is still
>>> "experimental"?
>>>
>>> My impression from past discussions is that there's interest in seeing
>>> broader changes to v2 (see [1] and [2] below). I tend to agree and
>>> I'm willing to head up that effort. But I wanted to double-check
>>> first that there was consensus on the overall idea of "broader
>>> changes" before I start filing JIRAs and working out specifics.
>>>
>>> Assuming consensus, my next step will be to write up the current v2
>>> endpoints and some proposed changes in a spreadsheet that we can
>>> iterate on. (Feedback on specific APIs, changes, etc. definitely
>>> welcome at that point once I get there.)
>>>
>>> Best,
>>>
>>> Jason
>>>
>>> [1] "Should v2 API be experimental" -
>>> https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
>>> [2] "Solr 9.0 Release Blockers"" -
>>> https://lists.apache.org/thread/m7k2gvgxldkns7jqjnw1ghhqx7s3tpl1
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>
>>>
>>>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
Re: v2 API "Broader Changes" and Next Steps
Posted by Eric Pugh <ep...@opensourceconnections.com>.
I really appreciate the perspective you bring Shawn on helping users...
we should think about how to make it easy for a user to run a command...
maybe an explicit admin UI for running arbitrary commands int he same way
we use the browser???
On Fri, Jun 17, 2022 at 1:35 PM Shawn Heisey <ap...@elyograg.org> wrote:
> On 6/17/2022 12:15 PM, Jason Gerlowski wrote:
> > Hey Shawn, Gus: I agree that "just sharing a URL" is dead simple and
> > can help for less-sophisticated users, or in more locked-down (i.e.
> > dev tool-less) environments. But I agree with Noble's point: using
> > the full range of HTTP methods is about as uniquitous "standard
> > practice" as it gets. We might help less-technically-saavy users by
> > writing a GET-only API, but that would end up being less intuitive for
> > the majority of users who are familiar with and expect that standard
> > practice.
>
> I have some mixed feelings about making it impossible to take actions
> with a GET. For index updates, I don't have a problem with those
> requiring POST, mostly because that is required now with v1. Same for
> the Config API or the Schema API. What you're saying about properly
> using all the HTTP methods makes sense and required if we want the API
> to be something modern like REST.
>
> One way could be to have all "manual" browser-based requests actually go
> through the admin UI. So clicking a button in the admin UI would take
> you to a URL that if repeated, would take exactly the same action, and
> that would be the URL that gets shared to demonstrate things. Similar
> to what happens when using the "query" interface in the admin UI now.
>
> The admin UI code running in the browser would translate those requests
> into the appropriate v2 call, including a POST body if that is needed.
> I am imagining all the required parameters being after the # in the URL,
> which only the browser ever sees.
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
Re: v2 API "Broader Changes" and Next Steps
Posted by Jason Gerlowski <ge...@gmail.com>.
> I hope I am not misunderstood ... I absolutely think we should fully embrace proper HTTP verb usage in v2 and document it extensively
Oh. I was misunderstanding you apparently then. Great! My mistake
on the misunderstanding; I appreciate the clarification.
I do think that the "Swagger UI" meets a lot of the goals you mention
about having a web UI that allows you to craft "just a URL" for the
situations you mentioned. If it doesn't, I totally see what you'd
value in having an Admin UI screen for that specifically.
Best,
Jason
On Fri, Jun 17, 2022 at 9:37 PM Shawn Heisey <ap...@elyograg.org> wrote:
>
> On 6/17/22 18:21, Noble Paul wrote:
>
> > Is there a need to make it easier to invoke POST/PUT/DELETE methods ?
> > absolutely yes.
> >
> > But, that should be a separate effort. Most likely , we would keep the
> > V1 API around for a while and users could resort to those APIs if they
> > wish to. However, the discussion is around whether we should migrate
> > our documentation and clients to use V2 as a de-facto interface
>
> I hope I am not misunderstood.
>
> I absolutely think we should fully embrace proper HTTP verb usage in v2
> and document it extensively. And I think the API should use request
> bodies wherever that is appropriate, regardless of whether that impacts
> the ability for people to craft URLs for a browser using the v2 API
> directly.
>
> I just want to be sure that users still have a way to send somebody a
> URL that accomplishes certain tasks right in the browser. The more I
> think about it, the more I like the idea of the admin UI being a vehicle
> to make this possible. I don't think we need to necessarily expose ALL
> of the API in this way. I like the idea Eric came up with for a page in
> the UI that allows arbitrary requests. My two cents on that: Have it
> ask for an HTTP verb, a URL path, provide a way to add URL parameters,
> and a place to put a body for the request if the verb is a type that
> generally allows a body. Maybe have some buttons that pre-fill the page
> for certain types of requests.
>
> As an example that the admin UI has right now, this is the URL I ended
> up on after I asked the admin UI to do a query on the Solr server that I
> manage for myself:
>
> http://localhost:8983/solr/#/dovecot/query?q=body:solr&q.op=OR&indent=true&facet=true&facet.field=user
>
> The admin UI takes that and converts it to the following, executes it,
> and shows the results in the UI:
>
> http://localhost:8983/solr/dovecot/select?facet.field=user&facet=true&indent=true&q.op=OR&q=body%3Asolr
>
> I am pretty sure this happens entirely in the javascript that the
> browser runs.
>
> Something similar could be done for any other request that users might
> commonly want to share as a URL.
>
> Implementing this certainly won't be a small undertaking. I am willing
> to help, but I have to caution that my html/css/javascript skills are
> not extensive.
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Shawn Heisey <ap...@elyograg.org>.
On 6/17/22 18:21, Noble Paul wrote:
> Is there a need to make it easier to invoke POST/PUT/DELETE methods ?
> absolutely yes.
>
> But, that should be a separate effort. Most likely , we would keep the
> V1 API around for a while and users could resort to those APIs if they
> wish to. However, the discussion is around whether we should migrate
> our documentation and clients to use V2 as a de-facto interface
I hope I am not misunderstood.
I absolutely think we should fully embrace proper HTTP verb usage in v2
and document it extensively. And I think the API should use request
bodies wherever that is appropriate, regardless of whether that impacts
the ability for people to craft URLs for a browser using the v2 API
directly.
I just want to be sure that users still have a way to send somebody a
URL that accomplishes certain tasks right in the browser. The more I
think about it, the more I like the idea of the admin UI being a vehicle
to make this possible. I don't think we need to necessarily expose ALL
of the API in this way. I like the idea Eric came up with for a page in
the UI that allows arbitrary requests. My two cents on that: Have it
ask for an HTTP verb, a URL path, provide a way to add URL parameters,
and a place to put a body for the request if the verb is a type that
generally allows a body. Maybe have some buttons that pre-fill the page
for certain types of requests.
As an example that the admin UI has right now, this is the URL I ended
up on after I asked the admin UI to do a query on the Solr server that I
manage for myself:
http://localhost:8983/solr/#/dovecot/query?q=body:solr&q.op=OR&indent=true&facet=true&facet.field=user
The admin UI takes that and converts it to the following, executes it,
and shows the results in the UI:
http://localhost:8983/solr/dovecot/select?facet.field=user&facet=true&indent=true&q.op=OR&q=body%3Asolr
I am pretty sure this happens entirely in the javascript that the
browser runs.
Something similar could be done for any other request that users might
commonly want to share as a URL.
Implementing this certainly won't be a small undertaking. I am willing
to help, but I have to caution that my html/css/javascript skills are
not extensive.
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Noble Paul <no...@gmail.com>.
@Shawn
Is there a need to make it easier to invoke POST/PUT/DELETE methods ?
absolutely yes.
But, that should be a separate effort. Most likely , we would keep the V1
API around for a while and users could resort to those APIs if they wish
to. However, the discussion is around whether we should migrate our
documentation and clients to use V2 as a de-facto interface
On Sat, Jun 18, 2022 at 5:35 AM Shawn Heisey <ap...@elyograg.org> wrote:
> On 6/17/2022 12:15 PM, Jason Gerlowski wrote:
> > Hey Shawn, Gus: I agree that "just sharing a URL" is dead simple and
> > can help for less-sophisticated users, or in more locked-down (i.e.
> > dev tool-less) environments. But I agree with Noble's point: using
> > the full range of HTTP methods is about as uniquitous "standard
> > practice" as it gets. We might help less-technically-saavy users by
> > writing a GET-only API, but that would end up being less intuitive for
> > the majority of users who are familiar with and expect that standard
> > practice.
>
> I have some mixed feelings about making it impossible to take actions
> with a GET. For index updates, I don't have a problem with those
> requiring POST, mostly because that is required now with v1. Same for
> the Config API or the Schema API. What you're saying about properly
> using all the HTTP methods makes sense and required if we want the API
> to be something modern like REST.
>
> One way could be to have all "manual" browser-based requests actually go
> through the admin UI. So clicking a button in the admin UI would take
> you to a URL that if repeated, would take exactly the same action, and
> that would be the URL that gets shared to demonstrate things. Similar
> to what happens when using the "query" interface in the admin UI now.
>
> The admin UI code running in the browser would translate those requests
> into the appropriate v2 call, including a POST body if that is needed.
> I am imagining all the required parameters being after the # in the URL,
> which only the browser ever sees.
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
--
-----------------------------------------------------
Noble Paul
Re: v2 API "Broader Changes" and Next Steps
Posted by Shawn Heisey <ap...@elyograg.org>.
On 6/17/2022 12:15 PM, Jason Gerlowski wrote:
> Hey Shawn, Gus: I agree that "just sharing a URL" is dead simple and
> can help for less-sophisticated users, or in more locked-down (i.e.
> dev tool-less) environments. But I agree with Noble's point: using
> the full range of HTTP methods is about as uniquitous "standard
> practice" as it gets. We might help less-technically-saavy users by
> writing a GET-only API, but that would end up being less intuitive for
> the majority of users who are familiar with and expect that standard
> practice.
I have some mixed feelings about making it impossible to take actions
with a GET. For index updates, I don't have a problem with those
requiring POST, mostly because that is required now with v1. Same for
the Config API or the Schema API. What you're saying about properly
using all the HTTP methods makes sense and required if we want the API
to be something modern like REST.
One way could be to have all "manual" browser-based requests actually go
through the admin UI. So clicking a button in the admin UI would take
you to a URL that if repeated, would take exactly the same action, and
that would be the URL that gets shared to demonstrate things. Similar
to what happens when using the "query" interface in the admin UI now.
The admin UI code running in the browser would translate those requests
into the appropriate v2 call, including a POST body if that is needed.
I am imagining all the required parameters being after the # in the URL,
which only the browser ever sees.
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Eric Pugh <ep...@opensourceconnections.com>.
As far as CORS goes, I think you are right technically…. It’s it’s own thing. Maybe I think of it as part of API from the perspective that supporting CORS makes it easier to integrate with the API from other systems. Kind of like supporting Swagger etc…
> On Jul 14, 2022, at 11:35 AM, Houston Putman <ho...@apache.org> wrote:
>
> I have added some comments to the API spreadsheet, but really like the
> direction.
>
> Thanks for putting all of this together, it couldn't have been easy!
>
> - Houston
>
> On Thu, Jul 14, 2022 at 7:00 AM Jason Gerlowski <ge...@gmail.com>
> wrote:
>
>> Hey all,
>>
>> Wanted to give another "plug" to the spreadsheet of proposed v2 API
>> changes, in case folks missed it the first time around. Please take a
>> look and review whenever you get a chance!
>>
>>
>> https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
>>
>>> Love to see CORS support added
>>
>> I'm not a CORS expert, so maybe I'm misunderstanding it, but I always
>> thought CORS was a security feature that was somewhat independent from
>> an APIs shape/design? Do endpoints need to be designed "for" CORs in
>> some way I'm missing? Just trying to understand if and how it'd
>> dovetail with the v2 effort here.
>>
>> Best,
>>
>> Jason
>>
>> On Thu, Jul 7, 2022 at 8:20 AM Eric Pugh
>> <ep...@opensourceconnections.com> wrote:
>>>
>>> Love to see CORS support added ;-)
>>>
>>>
>>> On Jul 6, 2022, at 9:45 AM, Jason Gerlowski <ge...@gmail.com>
>> wrote:
>>>
>>> [Jan] I think it is better for the project to evolve and fix this
>>>
>>>
>>> Glad to hear it; sorry for the confusion if I misunderstood your
>> concerns!
>>>
>>> Well in that case it sounds like there's general support for the idea
>>> of broader changes to the v2 API, and no categorical objections
>>> (albeit a few concerns about helping users upgrade, etc.)
>>>
>>> Of course, there'll need to be a good bit of discussion still around
>>> what specific changes to make. REST and OpenAPI support are the two
>>> things that've come up repeatedly in past discussions, so I've gone
>>> ahead and put together a Google Sheet with first-drafts of the changes
>>> each API would need if we go in that direction. I've attached the
>>> sheet to SOLR-15871 ("Cosmetic and consistency improvements for the v2
>>> API") and linked it below. Hopefully that'll be a good way to
>>> kickstart the discussion.
>>>
>>>
>> https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
>>>
>>> Thanks all for the feedback so far!
>>>
>>> Best,
>>>
>>> Jason
>>>
>>> On Tue, Jun 21, 2022 at 4:25 AM Jan Høydahl <ja...@cominvent.com>
>> wrote:
>>>
>>>
>>> I'd love to find a way to
>>> address your concerns and still evolve v2 without backcompat, if we
>>> can.
>>>
>>>
>>> I just wanted to highlight that some users may be using v2 without
>> realizing it was experimental due to the back-and-forth communication we
>> have had on this.
>>> Personally I think it is better for the project to evolve and fix this,
>> even if that means we'll put extra migration effort on some v2 users in
>> minor releases. We'd of course need to clearly mark such changes so it
>> won't come as a surprise.
>>>
>>> Jan
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>
>>>
>>> _______________________
>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
>> http://www.opensourceconnections.com | My Free/Busy
>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>>> This e-mail and all contents, including attachments, is considered to be
>> Company Confidential unless explicitly stated otherwise, regardless of
>> whether attachments are marked as such.
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>>
>>
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: v2 API "Broader Changes" and Next Steps
Posted by Houston Putman <ho...@apache.org>.
I have added some comments to the API spreadsheet, but really like the
direction.
Thanks for putting all of this together, it couldn't have been easy!
- Houston
On Thu, Jul 14, 2022 at 7:00 AM Jason Gerlowski <ge...@gmail.com>
wrote:
> Hey all,
>
> Wanted to give another "plug" to the spreadsheet of proposed v2 API
> changes, in case folks missed it the first time around. Please take a
> look and review whenever you get a chance!
>
>
> https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
>
> > Love to see CORS support added
>
> I'm not a CORS expert, so maybe I'm misunderstanding it, but I always
> thought CORS was a security feature that was somewhat independent from
> an APIs shape/design? Do endpoints need to be designed "for" CORs in
> some way I'm missing? Just trying to understand if and how it'd
> dovetail with the v2 effort here.
>
> Best,
>
> Jason
>
> On Thu, Jul 7, 2022 at 8:20 AM Eric Pugh
> <ep...@opensourceconnections.com> wrote:
> >
> > Love to see CORS support added ;-)
> >
> >
> > On Jul 6, 2022, at 9:45 AM, Jason Gerlowski <ge...@gmail.com>
> wrote:
> >
> > [Jan] I think it is better for the project to evolve and fix this
> >
> >
> > Glad to hear it; sorry for the confusion if I misunderstood your
> concerns!
> >
> > Well in that case it sounds like there's general support for the idea
> > of broader changes to the v2 API, and no categorical objections
> > (albeit a few concerns about helping users upgrade, etc.)
> >
> > Of course, there'll need to be a good bit of discussion still around
> > what specific changes to make. REST and OpenAPI support are the two
> > things that've come up repeatedly in past discussions, so I've gone
> > ahead and put together a Google Sheet with first-drafts of the changes
> > each API would need if we go in that direction. I've attached the
> > sheet to SOLR-15871 ("Cosmetic and consistency improvements for the v2
> > API") and linked it below. Hopefully that'll be a good way to
> > kickstart the discussion.
> >
> >
> https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
> >
> > Thanks all for the feedback so far!
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Jun 21, 2022 at 4:25 AM Jan Høydahl <ja...@cominvent.com>
> wrote:
> >
> >
> > I'd love to find a way to
> > address your concerns and still evolve v2 without backcompat, if we
> > can.
> >
> >
> > I just wanted to highlight that some users may be using v2 without
> realizing it was experimental due to the back-and-forth communication we
> have had on this.
> > Personally I think it is better for the project to evolve and fix this,
> even if that means we'll put extra migration effort on some v2 users in
> minor releases. We'd of course need to clearly mark such changes so it
> won't come as a surprise.
> >
> > Jan
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
> >
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com | My Free/Busy
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
Re: v2 API "Broader Changes" and Next Steps
Posted by Marcus Eagan <ma...@gmail.com>.
Really happy to see everyone's interest in supporting beginning users. I
often feel protecting them is one really important aspect of supporting
them.
There is a mechanism for disabling the Admin API by default. In SOLR-14014
<https://issues.apache.org/jira/browse/SOLR-14014> I added a flag for the
start command. If you'd want it as a configuration option, that would be a
new discussion but not a ton of code. It's an interesting thought, and I
definitely want to do everything I can to support the evolution to a
single API (v2) as the standard.
Best,
Marcus
On Sat, Jul 23, 2022 at 10:38 AM Shawn Heisey <ap...@elyograg.org> wrote:
> On 7/23/2022 9:41 AM, Gus Heck wrote:
> > There is something of a risk to have a server that accepts non safe get
> > requests. All these requests are contrary to the HTTP specification (
> > https://www.rfc-editor.org/rfc/rfc9110.html#section-9.2.1).
>
> If the only way to make certain things happen with a GET is via the
> admin UI, then I think we can close that vector by turning the admin UI
> off by default and requiring a config option to enable it, probably in
> solr.xml. I think the configs we include should NOT have that option
> enabled.
>
> As you know, every capability that the admin UI has can be accomplished
> directly. Maybe not with just a browser, but any dedicated attacker
> won't limit themselves to a browser.
>
> to another discussion (for a different email thread): Standardizing ALL
> configs to the same format. We currently have a mix of xml, json, and
> properties.
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
--
Marcus Eagan
Re: v2 API "Broader Changes" and Next Steps
Posted by Jason Gerlowski <ge...@gmail.com>.
> A crawler ... could do very bad things if [unleashed on] a wiki with instructions on how to maintain their deployed solr,
What a terrifyingly imaginative example haha!
Thanks all for the pointers and ideas! Gus - I definitely see the
value in making better use of the servlet/filter abstractions to
organize v1/v2 code. I'm not sure the "endpoint reshaping" work I'm
setting out to do will take me close to that code at all, but I'll
keep it in mind in case I find overlap we can take advantage of.
It sounds like there's general consensus around improving the v2 API
surface. To summarize some of the points discussed:
1. We are interested in improving v2.
2. REST (including more meaningful use of HTTP verbs and delineation
of "read" vs "write" APIs) and OpenAPI compatibility as organizing
principles.
3. Changes can largely be made "in place" without needing a whole new
"v3" or strict concern for backcompat. (Though in some places
preserving backcompat will make sense or even be required to support
rolling upgrades, etc.)
4. It seems like people are mostly OK with the specific changes
proposed in https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
. (Or at least, there haven't been a ton of inline comments there?
That seems like enough agreement for me to start working on the
specific changes proposed in the spreadsheet. I've got an operator
release on my plate for this week, but I'll start on API changes as
soon as that's wrapped up. (That'll give people more time to give
feedback on the proposed changes as well).
Thanks all!
Jason
On Sat, Jul 23, 2022 at 9:11 PM Gus Heck <gu...@gmail.com> wrote:
>
> On Sat, Jul 23, 2022 at 1:38 PM Shawn Heisey <ap...@elyograg.org> wrote:
>
> > On 7/23/2022 9:41 AM, Gus Heck wrote:
> > > There is something of a risk to have a server that accepts non safe get
> > > requests. All these requests are contrary to the HTTP specification (
> > > https://www.rfc-editor.org/rfc/rfc9110.html#section-9.2.1).
> >
> > If the only way to make certain things happen with a GET is via the
> > admin UI, then I think we can close that vector by turning the admin UI
> > off by default and requiring a config option to enable it, probably in
> > solr.xml.
>
>
> The admin UI is definitely not required presently to make
> persistent changes to solr. The the V1 api is get request based. Only if
> one were to turn off v1 (i.e. deconfigure it's servlet in a hypothetical
> future) would that become true. If we were to try to turn various versions
> of the API's or feature areas on and off it would imply yet another if
> check for yet another system property or some such that runs on every
> request. Much better if the inherent dispatch of the server started the
> request off in a servlet focused on that feature (or returned 404).
>
> As I think was mentioned previously in another discussion where I brought
> this up we do have some difficulty with the order of some of our api
> commands because the collection is part of the url ahead of the verb that
> would normally denote the servlet i.e. we have /solr/collection1/update
> instead of /solr/update/collection1 so some URL remapping for back compat
> might become necessary, but that's an easily solved problem I think.
> (jetty.xml is already using the rewrite handler for other things, so that
> might be a place to start)
>
> -Gus
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Gus Heck <gu...@gmail.com>.
On Sat, Jul 23, 2022 at 1:38 PM Shawn Heisey <ap...@elyograg.org> wrote:
> On 7/23/2022 9:41 AM, Gus Heck wrote:
> > There is something of a risk to have a server that accepts non safe get
> > requests. All these requests are contrary to the HTTP specification (
> > https://www.rfc-editor.org/rfc/rfc9110.html#section-9.2.1).
>
> If the only way to make certain things happen with a GET is via the
> admin UI, then I think we can close that vector by turning the admin UI
> off by default and requiring a config option to enable it, probably in
> solr.xml.
The admin UI is definitely not required presently to make
persistent changes to solr. The the V1 api is get request based. Only if
one were to turn off v1 (i.e. deconfigure it's servlet in a hypothetical
future) would that become true. If we were to try to turn various versions
of the API's or feature areas on and off it would imply yet another if
check for yet another system property or some such that runs on every
request. Much better if the inherent dispatch of the server started the
request off in a servlet focused on that feature (or returned 404).
As I think was mentioned previously in another discussion where I brought
this up we do have some difficulty with the order of some of our api
commands because the collection is part of the url ahead of the verb that
would normally denote the servlet i.e. we have /solr/collection1/update
instead of /solr/update/collection1 so some URL remapping for back compat
might become necessary, but that's an easily solved problem I think.
(jetty.xml is already using the rewrite handler for other things, so that
might be a place to start)
-Gus
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: v2 API "Broader Changes" and Next Steps
Posted by Shawn Heisey <ap...@elyograg.org>.
On 7/23/2022 9:41 AM, Gus Heck wrote:
> There is something of a risk to have a server that accepts non safe get
> requests. All these requests are contrary to the HTTP specification (
> https://www.rfc-editor.org/rfc/rfc9110.html#section-9.2.1).
If the only way to make certain things happen with a GET is via the
admin UI, then I think we can close that vector by turning the admin UI
off by default and requiring a config option to enable it, probably in
solr.xml. I think the configs we include should NOT have that option
enabled.
As you know, every capability that the admin UI has can be accomplished
directly. Maybe not with just a browser, but any dedicated attacker
won't limit themselves to a browser.
to another discussion (for a different email thread): Standardizing ALL
configs to the same format. We currently have a mix of xml, json, and
properties.
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Gus Heck <gu...@gmail.com>.
Lost track of this convo for a while... but basically like what I read. I
like that we've acknowledged the positive role of the UI, and I also very
much like the use of http verbs for v2. The primary concern for me is that
I can do most things via a browser connection to solr, so a kibana-like ui
or even something simpler enhancing what we have on the Documents page
could be sufficient. I find less use for sending folks URLs, but that point
makes good sense too. Couple thoughts:
There is something of a risk to have a server that accepts non safe get
requests. All these requests are contrary to the HTTP specification (
https://www.rfc-editor.org/rfc/rfc9110.html#section-9.2.1). A crawler or
browser accelerator that might prefetch pages could do very bad things if
unleashed on an intranet where there was (for example) a wiki with
instructions on how to maintain their deployed solr, despite only issuing
get requests. So I also think it is a good idea if unsafe gets can be
turned off for folks who wish to guard against this (say on prod machines).
Another thought I would like to float is that the current paradigm is that
every api (v1, v2, query, json query, export, streaming, etc) is piped
through our single servlet filter. If we are going to make serious changes
to v2 (or do a v3) we might endeavor to put the new version into a servlet
(which will have dedicated methods for verbs anyway). My changes to move
CoreContainer into a servlet listener should make it possible to get a
reference to CC in the servlet's init() method. There is probably a bunch
of code that needs to be extracted to a reusable location so that v2 and v1
can continue to share appropriate code. But this would be a step towards
simplifying the architecture so that functionality is organized into
servlet classes of related logic and data rather than if-blocks and methods
sharing all state in a few tightly coupled classes.
It would be cool if (as one possible example) someday one had the option to
build an index and then run a solr instance with the admin/update entirely
disabled (i.e. servlets commented out of the web.xml and not even loaded).
This would provide a read-only server without needing authentication.
-Gus
On Wed, Jul 20, 2022 at 6:59 PM Jan Høydahl <ja...@cominvent.com> wrote:
> I'm still on holiday, but just chiming in to say I like and support the
> direction here. I'm impressed by the structured and holistic approach on
> this.
> Also agree it is far better to have a consistent "new" v2 api in 9.x or
> 10.0 at the expense of back-compat for existing v2 users -- maintaining
> three versions is not going to happen.
>
> BTW: OpenAPI is the new name for what we earlier referred to as the
> Swagger specification. The name Swagger is still the brand name for some
> OpenAPI tooling - see
> https://swagger.io/blog/api-strategy/difference-between-swagger-and-openapi/
>
> Jan
>
> > 15. jul. 2022 kl. 05:17 skrev David Smiley <ds...@apache.org>:
> >
> > On Thu, Jul 14, 2022 at 4:31 PM Jason Gerlowski <ge...@gmail.com>
> > wrote:
> >
> >>> I think it would be best to tackle one limited area first so we can see
> >> it in practice both at the surface and implementation.
> >>
> >> I think that makes sense, assuming that by "tackling" you mean
> >> updating the API path/verb/etc and moving the "real" logic over to the
> >> v2 class? I'd rather we not take a "limited area" approach to getting
> >> consensus around the API endpoint design itself, as IMO that'd produce
> >> a less cohesive result. I don't think that was what you meant, but
> >> just double-checking.
> >>
> >
> > Sure -- API first. There may be some implementation realities that cause
> > us to reconsider choices but we can take that as it happens.
> >
> > ~ David
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: v2 API "Broader Changes" and Next Steps
Posted by Jan Høydahl <ja...@cominvent.com>.
I'm still on holiday, but just chiming in to say I like and support the direction here. I'm impressed by the structured and holistic approach on this.
Also agree it is far better to have a consistent "new" v2 api in 9.x or 10.0 at the expense of back-compat for existing v2 users -- maintaining three versions is not going to happen.
BTW: OpenAPI is the new name for what we earlier referred to as the Swagger specification. The name Swagger is still the brand name for some OpenAPI tooling - see https://swagger.io/blog/api-strategy/difference-between-swagger-and-openapi/
Jan
> 15. jul. 2022 kl. 05:17 skrev David Smiley <ds...@apache.org>:
>
> On Thu, Jul 14, 2022 at 4:31 PM Jason Gerlowski <ge...@gmail.com>
> wrote:
>
>>> I think it would be best to tackle one limited area first so we can see
>> it in practice both at the surface and implementation.
>>
>> I think that makes sense, assuming that by "tackling" you mean
>> updating the API path/verb/etc and moving the "real" logic over to the
>> v2 class? I'd rather we not take a "limited area" approach to getting
>> consensus around the API endpoint design itself, as IMO that'd produce
>> a less cohesive result. I don't think that was what you meant, but
>> just double-checking.
>>
>
> Sure -- API first. There may be some implementation realities that cause
> us to reconsider choices but we can take that as it happens.
>
> ~ David
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by David Smiley <ds...@apache.org>.
On Thu, Jul 14, 2022 at 4:31 PM Jason Gerlowski <ge...@gmail.com>
wrote:
> > I think it would be best to tackle one limited area first so we can see
> it in practice both at the surface and implementation.
>
> I think that makes sense, assuming that by "tackling" you mean
> updating the API path/verb/etc and moving the "real" logic over to the
> v2 class? I'd rather we not take a "limited area" approach to getting
> consensus around the API endpoint design itself, as IMO that'd produce
> a less cohesive result. I don't think that was what you meant, but
> just double-checking.
>
Sure -- API first. There may be some implementation realities that cause
us to reconsider choices but we can take that as it happens.
~ David
Re: v2 API "Broader Changes" and Next Steps
Posted by Jason Gerlowski <ge...@gmail.com>.
Hey, thanks for chiming in David, Houston, Eric! I'm glad to get a
few affirmations on the general direction/approach of the design.
Some responses (mostly agreement) in-line.
> I strongly prefer to move our functionality to implement the new API as at its core/directly, and then have v1 be a shim on top
Agree completely. That's the best way I can think of to make sure
that v2 doesn't fall behind v1: make v1 the shim and put the "real"
logic in v2. That was a primary motivation in tackling the move away
from the apispec framework - annotated Java classes give us a place to
put logic that we didn't have when everything was JSON. I haven't
started this effort yet, but plan on tracking it on SOLR-15736.
> I think it would be best to tackle one limited area first so we can see it in practice both at the surface and implementation.
I think that makes sense, assuming that by "tackling" you mean
updating the API path/verb/etc and moving the "real" logic over to the
v2 class? I'd rather we not take a "limited area" approach to getting
consensus around the API endpoint design itself, as IMO that'd produce
a less cohesive result. I don't think that was what you meant, but
just double-checking.
Best,
Jason
On Thu, Jul 14, 2022 at 1:00 PM David Smiley <ds...@apache.org> wrote:
>
> I did look over it twice now, here & there. Boy, there is a big API
> surface area to Solr! Thanks for working on this huge task Jason!
>
> I think it would be best to tackle one limited area first so we can see it
> in practice both at the surface and implementation. From an implementation
> perspective, I strongly prefer to move our functionality to implement the
> new API as at its core/directly, and then have v1 be a shim on top. This
> will lead to new/clean code in our future with legacy concerns off to the
> side that will be easy to jettison some day. I think the v2 we have today
> didn't take that approach.
>
> I'm not familiar with OpenAPI, but based on the conversations here, I love
> the prospect of having our users in various languages more easily talk to
> Solr. Maybe it could be used in Solrj to reduce code maintenance as well.
> Overall, similar to Swagger, which I have used in a POC and immediately
> loved that I had a simple UI that I didn't have to write/maintain.
>
> Let's not constrain ourselves with supporting the current v2 API on Solr
> 10! Doing so increases the risk that this API change won't even happen in
> the first place because it's too hard.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Thu, Jul 14, 2022 at 7:00 AM Jason Gerlowski <ge...@gmail.com>
> wrote:
>
> > Hey all,
> >
> > Wanted to give another "plug" to the spreadsheet of proposed v2 API
> > changes, in case folks missed it the first time around. Please take a
> > look and review whenever you get a chance!
> >
> >
> > https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
> >
> > > Love to see CORS support added
> >
> > I'm not a CORS expert, so maybe I'm misunderstanding it, but I always
> > thought CORS was a security feature that was somewhat independent from
> > an APIs shape/design? Do endpoints need to be designed "for" CORs in
> > some way I'm missing? Just trying to understand if and how it'd
> > dovetail with the v2 effort here.
> >
> > Best,
> >
> > Jason
> >
> > On Thu, Jul 7, 2022 at 8:20 AM Eric Pugh
> > <ep...@opensourceconnections.com> wrote:
> > >
> > > Love to see CORS support added ;-)
> > >
> > >
> > > On Jul 6, 2022, at 9:45 AM, Jason Gerlowski <ge...@gmail.com>
> > wrote:
> > >
> > > [Jan] I think it is better for the project to evolve and fix this
> > >
> > >
> > > Glad to hear it; sorry for the confusion if I misunderstood your
> > concerns!
> > >
> > > Well in that case it sounds like there's general support for the idea
> > > of broader changes to the v2 API, and no categorical objections
> > > (albeit a few concerns about helping users upgrade, etc.)
> > >
> > > Of course, there'll need to be a good bit of discussion still around
> > > what specific changes to make. REST and OpenAPI support are the two
> > > things that've come up repeatedly in past discussions, so I've gone
> > > ahead and put together a Google Sheet with first-drafts of the changes
> > > each API would need if we go in that direction. I've attached the
> > > sheet to SOLR-15871 ("Cosmetic and consistency improvements for the v2
> > > API") and linked it below. Hopefully that'll be a good way to
> > > kickstart the discussion.
> > >
> > >
> > https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
> > >
> > > Thanks all for the feedback so far!
> > >
> > > Best,
> > >
> > > Jason
> > >
> > > On Tue, Jun 21, 2022 at 4:25 AM Jan Høydahl <ja...@cominvent.com>
> > wrote:
> > >
> > >
> > > I'd love to find a way to
> > > address your concerns and still evolve v2 without backcompat, if we
> > > can.
> > >
> > >
> > > I just wanted to highlight that some users may be using v2 without
> > realizing it was experimental due to the back-and-forth communication we
> > have had on this.
> > > Personally I think it is better for the project to evolve and fix this,
> > even if that means we'll put extra migration effort on some v2 users in
> > minor releases. We'd of course need to clearly mark such changes so it
> > won't come as a surprise.
> > >
> > > Jan
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > For additional commands, e-mail: dev-help@solr.apache.org
> > >
> > >
> > > _______________________
> > > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> > http://www.opensourceconnections.com | My Free/Busy
> > > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > > This e-mail and all contents, including attachments, is considered to be
> > Company Confidential unless explicitly stated otherwise, regardless of
> > whether attachments are marked as such.
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by David Smiley <ds...@apache.org>.
I did look over it twice now, here & there. Boy, there is a big API
surface area to Solr! Thanks for working on this huge task Jason!
I think it would be best to tackle one limited area first so we can see it
in practice both at the surface and implementation. From an implementation
perspective, I strongly prefer to move our functionality to implement the
new API as at its core/directly, and then have v1 be a shim on top. This
will lead to new/clean code in our future with legacy concerns off to the
side that will be easy to jettison some day. I think the v2 we have today
didn't take that approach.
I'm not familiar with OpenAPI, but based on the conversations here, I love
the prospect of having our users in various languages more easily talk to
Solr. Maybe it could be used in Solrj to reduce code maintenance as well.
Overall, similar to Swagger, which I have used in a POC and immediately
loved that I had a simple UI that I didn't have to write/maintain.
Let's not constrain ourselves with supporting the current v2 API on Solr
10! Doing so increases the risk that this API change won't even happen in
the first place because it's too hard.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Thu, Jul 14, 2022 at 7:00 AM Jason Gerlowski <ge...@gmail.com>
wrote:
> Hey all,
>
> Wanted to give another "plug" to the spreadsheet of proposed v2 API
> changes, in case folks missed it the first time around. Please take a
> look and review whenever you get a chance!
>
>
> https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
>
> > Love to see CORS support added
>
> I'm not a CORS expert, so maybe I'm misunderstanding it, but I always
> thought CORS was a security feature that was somewhat independent from
> an APIs shape/design? Do endpoints need to be designed "for" CORs in
> some way I'm missing? Just trying to understand if and how it'd
> dovetail with the v2 effort here.
>
> Best,
>
> Jason
>
> On Thu, Jul 7, 2022 at 8:20 AM Eric Pugh
> <ep...@opensourceconnections.com> wrote:
> >
> > Love to see CORS support added ;-)
> >
> >
> > On Jul 6, 2022, at 9:45 AM, Jason Gerlowski <ge...@gmail.com>
> wrote:
> >
> > [Jan] I think it is better for the project to evolve and fix this
> >
> >
> > Glad to hear it; sorry for the confusion if I misunderstood your
> concerns!
> >
> > Well in that case it sounds like there's general support for the idea
> > of broader changes to the v2 API, and no categorical objections
> > (albeit a few concerns about helping users upgrade, etc.)
> >
> > Of course, there'll need to be a good bit of discussion still around
> > what specific changes to make. REST and OpenAPI support are the two
> > things that've come up repeatedly in past discussions, so I've gone
> > ahead and put together a Google Sheet with first-drafts of the changes
> > each API would need if we go in that direction. I've attached the
> > sheet to SOLR-15871 ("Cosmetic and consistency improvements for the v2
> > API") and linked it below. Hopefully that'll be a good way to
> > kickstart the discussion.
> >
> >
> https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
> >
> > Thanks all for the feedback so far!
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Jun 21, 2022 at 4:25 AM Jan Høydahl <ja...@cominvent.com>
> wrote:
> >
> >
> > I'd love to find a way to
> > address your concerns and still evolve v2 without backcompat, if we
> > can.
> >
> >
> > I just wanted to highlight that some users may be using v2 without
> realizing it was experimental due to the back-and-forth communication we
> have had on this.
> > Personally I think it is better for the project to evolve and fix this,
> even if that means we'll put extra migration effort on some v2 users in
> minor releases. We'd of course need to clearly mark such changes so it
> won't come as a surprise.
> >
> > Jan
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
> >
> > _______________________
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com | My Free/Busy
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
Re: v2 API "Broader Changes" and Next Steps
Posted by Jason Gerlowski <ge...@gmail.com>.
Hey all,
Wanted to give another "plug" to the spreadsheet of proposed v2 API
changes, in case folks missed it the first time around. Please take a
look and review whenever you get a chance!
https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
> Love to see CORS support added
I'm not a CORS expert, so maybe I'm misunderstanding it, but I always
thought CORS was a security feature that was somewhat independent from
an APIs shape/design? Do endpoints need to be designed "for" CORs in
some way I'm missing? Just trying to understand if and how it'd
dovetail with the v2 effort here.
Best,
Jason
On Thu, Jul 7, 2022 at 8:20 AM Eric Pugh
<ep...@opensourceconnections.com> wrote:
>
> Love to see CORS support added ;-)
>
>
> On Jul 6, 2022, at 9:45 AM, Jason Gerlowski <ge...@gmail.com> wrote:
>
> [Jan] I think it is better for the project to evolve and fix this
>
>
> Glad to hear it; sorry for the confusion if I misunderstood your concerns!
>
> Well in that case it sounds like there's general support for the idea
> of broader changes to the v2 API, and no categorical objections
> (albeit a few concerns about helping users upgrade, etc.)
>
> Of course, there'll need to be a good bit of discussion still around
> what specific changes to make. REST and OpenAPI support are the two
> things that've come up repeatedly in past discussions, so I've gone
> ahead and put together a Google Sheet with first-drafts of the changes
> each API would need if we go in that direction. I've attached the
> sheet to SOLR-15871 ("Cosmetic and consistency improvements for the v2
> API") and linked it below. Hopefully that'll be a good way to
> kickstart the discussion.
>
> https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
>
> Thanks all for the feedback so far!
>
> Best,
>
> Jason
>
> On Tue, Jun 21, 2022 at 4:25 AM Jan Høydahl <ja...@cominvent.com> wrote:
>
>
> I'd love to find a way to
> address your concerns and still evolve v2 without backcompat, if we
> can.
>
>
> I just wanted to highlight that some users may be using v2 without realizing it was experimental due to the back-and-forth communication we have had on this.
> Personally I think it is better for the project to evolve and fix this, even if that means we'll put extra migration effort on some v2 users in minor releases. We'd of course need to clearly mark such changes so it won't come as a surprise.
>
> Jan
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Eric Pugh <ep...@opensourceconnections.com>.
Love to see CORS support added ;-)
> On Jul 6, 2022, at 9:45 AM, Jason Gerlowski <ge...@gmail.com> wrote:
>
>> [Jan] I think it is better for the project to evolve and fix this
>
> Glad to hear it; sorry for the confusion if I misunderstood your concerns!
>
> Well in that case it sounds like there's general support for the idea
> of broader changes to the v2 API, and no categorical objections
> (albeit a few concerns about helping users upgrade, etc.)
>
> Of course, there'll need to be a good bit of discussion still around
> what specific changes to make. REST and OpenAPI support are the two
> things that've come up repeatedly in past discussions, so I've gone
> ahead and put together a Google Sheet with first-drafts of the changes
> each API would need if we go in that direction. I've attached the
> sheet to SOLR-15871 ("Cosmetic and consistency improvements for the v2
> API") and linked it below. Hopefully that'll be a good way to
> kickstart the discussion.
>
> https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
>
> Thanks all for the feedback so far!
>
> Best,
>
> Jason
>
> On Tue, Jun 21, 2022 at 4:25 AM Jan Høydahl <ja...@cominvent.com> wrote:
>>
>> I'd love to find a way to
>> address your concerns and still evolve v2 without backcompat, if we
>> can.
>>
>>
>> I just wanted to highlight that some users may be using v2 without realizing it was experimental due to the back-and-forth communication we have had on this.
>> Personally I think it is better for the project to evolve and fix this, even if that means we'll put extra migration effort on some v2 users in minor releases. We'd of course need to clearly mark such changes so it won't come as a surprise.
>>
>> Jan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com <http://www.opensourceconnections.com/> | My Free/Busy <http://tinyurl.com/eric-cal>
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
Re: v2 API "Broader Changes" and Next Steps
Posted by Jason Gerlowski <ge...@gmail.com>.
> [Jan] I think it is better for the project to evolve and fix this
Glad to hear it; sorry for the confusion if I misunderstood your concerns!
Well in that case it sounds like there's general support for the idea
of broader changes to the v2 API, and no categorical objections
(albeit a few concerns about helping users upgrade, etc.)
Of course, there'll need to be a good bit of discussion still around
what specific changes to make. REST and OpenAPI support are the two
things that've come up repeatedly in past discussions, so I've gone
ahead and put together a Google Sheet with first-drafts of the changes
each API would need if we go in that direction. I've attached the
sheet to SOLR-15871 ("Cosmetic and consistency improvements for the v2
API") and linked it below. Hopefully that'll be a good way to
kickstart the discussion.
https://docs.google.com/spreadsheets/d/1HAoBBFPpSiT8mJmgNZKkZAPwfCfPvlc08m5jz3fQBpA/edit?usp=sharing
Thanks all for the feedback so far!
Best,
Jason
On Tue, Jun 21, 2022 at 4:25 AM Jan Høydahl <ja...@cominvent.com> wrote:
>
> I'd love to find a way to
> address your concerns and still evolve v2 without backcompat, if we
> can.
>
>
> I just wanted to highlight that some users may be using v2 without realizing it was experimental due to the back-and-forth communication we have had on this.
> Personally I think it is better for the project to evolve and fix this, even if that means we'll put extra migration effort on some v2 users in minor releases. We'd of course need to clearly mark such changes so it won't come as a surprise.
>
> Jan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Jan Høydahl <ja...@cominvent.com>.
> I'd love to find a way to
> address your concerns and still evolve v2 without backcompat, if we
> can.
I just wanted to highlight that some users may be using v2 without realizing it was experimental due to the back-and-forth communication we have had on this.
Personally I think it is better for the project to evolve and fix this, even if that means we'll put extra migration effort on some v2 users in minor releases. We'd of course need to clearly mark such changes so it won't come as a surprise.
Jan
Re: v2 API "Broader Changes" and Next Steps
Posted by Jason Gerlowski <ge...@gmail.com>.
Hey all,
Thanks for the quick responses! Overall it sounds like no one objects
to the idea of broader changes necessarily, but that there are some
concerns about the timing/mechanics (backcompat, target release, etc.)
and specific API design (use of HTTP verbs, "URL-bar" friendliness,
etc). Hopefully that's a fair summary? A few thoughts on each topic
below:
> V2 has been around for several major releases and documented [...] long enough that we must assume a certain user base already [...] Thus I'm not sure we should act as if the entire v2 design is up for change
I totally understand your concern about burning our users Jan, but I
think Eric's response is spot-on: the ability to evolve v2 without
backcompat constraints was an explicit rationale in the "Should v2 API
be Experimental" thread where we agreed to the designation. What's the
purpose of the 'experimental' label, if not to free us up to make
changes outside of major-versions (and warn users of that
possibility)?
As I said then: "we don't have any means to know for sure how many
users v2 has today. But if we judge from the few signals we do have,
the number must be very small. e.g. The last user-list email that
mentions a v2 API path is [...] from May of 2018! Potential
backcompat breaks might inconvenience that small set of users, but
that inconvenience would be vastly outweighed by the benefit to all
our users of getting a cleaner, more consistent API out sooner."
If the root of your concern is that there might be a chunk of longtime
v2 users who haven't seen the shift to 'experimental': is there a
documentation/communication change that would assuage your concerns
there? A thread on 'announce@' highlighting the change maybe? Or a
survey in 'users@' to identify whether there is a v2 user-base out
there as you suspect? Something else? I'd love to find a way to
address your concerns and still evolve v2 without backcompat, if we
can.
> It's REALLY nice to be able to try things out with a browser ... [users] may have absolutely no idea how to use something like curl
Hey Shawn, Gus: I agree that "just sharing a URL" is dead simple and
can help for less-sophisticated users, or in more locked-down (i.e.
dev tool-less) environments. But I agree with Noble's point: using
the full range of HTTP methods is about as uniquitous "standard
practice" as it gets. We might help less-technically-saavy users by
writing a GET-only API, but that would end up being less intuitive for
the majority of users who are familiar with and expect that standard
practice.
That said: if we pursue OpenAPI (which, spoilers: I'd like to) then we
would get a nice (e.g.) Swagger UI for free, which should make it easy
for any user to send any request with "just a browser". Would that
address your concerns? [2]
Again, thanks for all the feedback!
Best,
Jason
[1] https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
[2] https://swagger.io/tools/swagger-ui/
On Thu, Jun 16, 2022 at 9:09 PM Noble Paul <no...@gmail.com> wrote:
>
> @Shawn
>
> So, if we move towards an HTTP POST , PUT or DELETE based commands we will never be able to send a link to others over text format because they work only on HTTP GET. I think this is an artificial limitation that we are going to impose on anyone designing an API standard and against the widely adopted standards in the industry.
>
> So, if we decide to use all the possible verbs of HTTP, users will have to either use curL or the admin UI for most commands . However some of the GET based V2 APIs (there are a lot of them) will continue to work . Please keep in mind that the GET based APIs are READ apis and the POST/PUT/DELETE APIs result in modifying the state of the cluster/node. I believe it is a security vulnerability if we let GET operations perform write operations and we should get rid of them ASAP
>
>
>
> On Fri, Jun 17, 2022 at 5:12 AM Shawn Heisey <ap...@elyograg.org> wrote:
>>
>> On 6/16/22 09:07, Gus Heck wrote:
>>
>> > I'm all for standardizing on v2 (or something like it) but one key
>> > concern I have is that when the only access I have to a client's
>> > server is via a web browser, possibly from a machine they control and
>> > on which I can't install tools, I'd like the only barrier to my
>> > running necessary admin commands is their (hopefully) configured
>> > security controls (RBAC/JWT/whatever). It's a loss of functionality if
>> > a REST client program, plugin or curl is *required*. Those tools are
>> > good things, but the ability to fully control solr directly from a
>> > browser (if properly authenticated) is a good feature we shouldn't lose.
>>
>> I agree with this.
>>
>> It's REALLY nice to be able to try things out with a browser, or to
>> issue infrequently-used admin requests with a browser. Or to send
>> somebody a URL with a note that says "This is what I am thinking." The
>> v1 API makes this possible, and I have abused it in this way a LOT. I
>> know someone is going to say "sending a body is trivial with curl." But
>> the person I am sending the message to may have absolutely no idea how
>> to use something like curl, and they may be on a stock windows setup
>> that doesn't have any of those cool tools available.
>>
>> I'm OK with such fiddling happening via the admin UI ... but I don't
>> think the admin UI is as feature-complete as it needs to be for an API
>> that *requires* a body in the request to work. And it's very important
>> from my perspective that I can send a URL to someone that demonstrates
>> how to do something that will ultimately happen with a client that knows
>> how to send request bodies.
>>
>> Thanks,
>> Shawn
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>>
>
>
> --
> -----------------------------------------------------
> Noble Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Noble Paul <no...@gmail.com>.
@Shawn
So, if we move towards an HTTP POST , PUT or DELETE based commands we will
never be able to send a link to others over text format because they work
only on HTTP GET. I think this is an artificial limitation that we are
going to impose on anyone designing an API standard and against the widely
adopted standards in the industry.
So, if we decide to use all the possible verbs of HTTP, users will have to
either use curL or the admin UI for most commands . However some of the GET
based V2 APIs (there are a lot of them) will continue to work . Please keep
in mind that the GET based APIs are READ apis and the POST/PUT/DELETE APIs
result in modifying the state of the cluster/node. I believe it is a
security vulnerability if we let GET operations perform write operations
and we should get rid of them ASAP
On Fri, Jun 17, 2022 at 5:12 AM Shawn Heisey <ap...@elyograg.org> wrote:
> On 6/16/22 09:07, Gus Heck wrote:
>
> > I'm all for standardizing on v2 (or something like it) but one key
> > concern I have is that when the only access I have to a client's
> > server is via a web browser, possibly from a machine they control and
> > on which I can't install tools, I'd like the only barrier to my
> > running necessary admin commands is their (hopefully) configured
> > security controls (RBAC/JWT/whatever). It's a loss of functionality if
> > a REST client program, plugin or curl is *required*. Those tools are
> > good things, but the ability to fully control solr directly from a
> > browser (if properly authenticated) is a good feature we shouldn't lose.
>
> I agree with this.
>
> It's REALLY nice to be able to try things out with a browser, or to
> issue infrequently-used admin requests with a browser. Or to send
> somebody a URL with a note that says "This is what I am thinking." The
> v1 API makes this possible, and I have abused it in this way a LOT. I
> know someone is going to say "sending a body is trivial with curl." But
> the person I am sending the message to may have absolutely no idea how
> to use something like curl, and they may be on a stock windows setup
> that doesn't have any of those cool tools available.
>
> I'm OK with such fiddling happening via the admin UI ... but I don't
> think the admin UI is as feature-complete as it needs to be for an API
> that *requires* a body in the request to work. And it's very important
> from my perspective that I can send a URL to someone that demonstrates
> how to do something that will ultimately happen with a client that knows
> how to send request bodies.
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
--
-----------------------------------------------------
Noble Paul
Re: v2 API "Broader Changes" and Next Steps
Posted by Shawn Heisey <ap...@elyograg.org>.
On 6/16/22 09:07, Gus Heck wrote:
> I'm all for standardizing on v2 (or something like it) but one key
> concern I have is that when the only access I have to a client's
> server is via a web browser, possibly from a machine they control and
> on which I can't install tools, I'd like the only barrier to my
> running necessary admin commands is their (hopefully) configured
> security controls (RBAC/JWT/whatever). It's a loss of functionality if
> a REST client program, plugin or curl is *required*. Those tools are
> good things, but the ability to fully control solr directly from a
> browser (if properly authenticated) is a good feature we shouldn't lose.
I agree with this.
It's REALLY nice to be able to try things out with a browser, or to
issue infrequently-used admin requests with a browser. Or to send
somebody a URL with a note that says "This is what I am thinking." The
v1 API makes this possible, and I have abused it in this way a LOT. I
know someone is going to say "sending a body is trivial with curl." But
the person I am sending the message to may have absolutely no idea how
to use something like curl, and they may be on a stock windows setup
that doesn't have any of those cool tools available.
I'm OK with such fiddling happening via the admin UI ... but I don't
think the admin UI is as feature-complete as it needs to be for an API
that *requires* a body in the request to work. And it's very important
from my perspective that I can send a URL to someone that demonstrates
how to do something that will ultimately happen with a client that knows
how to send request bodies.
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org
Re: v2 API "Broader Changes" and Next Steps
Posted by Gus Heck <gu...@gmail.com>.
I'm all for standardizing on v2 (or something like it) but one key concern
I have is that when the only access I have to a client's server is via a
web browser, possibly from a machine they control and on which I can't
install tools, I'd like the only barrier to my running necessary admin
commands is their (hopefully) configured security controls
(RBAC/JWT/whatever). It's a loss of functionality if a REST client program,
plugin or curl is *required*. Those tools are good things, but the ability
to fully control solr directly from a browser (if properly authenticated)
is a good feature we shouldn't lose.
Presently JSON based commands can be issued in the "documents" page, but
some attention should be given to make sure we don't lose or even maybe
enhance/rename that interface. Obviously, in the near future V1 will
remain, but if deprecated, the implication is that someday (10.0? 11.0? it
might go away).
On Thu, Jun 16, 2022 at 10:30 AM Jan Høydahl <ja...@cominvent.com> wrote:
> V2 has existed since v6.5 and has been documented in ref guide evner
> since. In 7.0 release announcement we said:
>
>
> -
>
> The new v2 API, exposed at /api/ and also supported via SolrJ, is now
> the preferred API, but /solr/ continues to work.
>
> We have been showing v2 api commands in tabs for e.g. config api since
> 7.4. So users have been encouraged to switch for at least 3-4 years.
>
> Jan Høydahl
>
> 16. jun. 2022 kl. 16:16 skrev Eric Pugh <ep...@opensourceconnections.com>:
>
>
> I don’t have the link, but I think we agree that v2 *could* evolve, hence
> the whole experimental thing.
>
> I worry that the efforts of maintaining three versions of the api will
> stall us. Especially if the changes we want to make aren’t wholesale
> changes. Now if we introduced a graphql based api, that might justify
> the v3 tag.
>
> My gut feeling is that no one uses v2 yet, so the end user impact is
> minimal.
>
> Maybe run a quick survey to decide that?
>
> On Thu, Jun 16, 2022 at 5:41 AM Jan Høydahl <ja...@cominvent.com> wrote:
>
>> HI,
>>
>> Thanks for having the guts to start such a journey!
>> V2 has been around for several major releases and documented (it's in
>> v6.6 guide https://solr.apache.org/guide/6_6/v2-api.html) long enough
>> that we must assume a certain user base already.
>> The "experimental" warning in RefGuide is not prominent between 6.6 and
>> 8.11, looks like it was added in the 9.0 guide.
>> Thus I'm not sure we should act as if the entire v2 design is up for
>> change.
>>
>> So perhaps the next-gen, OpenApi style APIs should be v3 and then
>> deprecate both v1 and v2 once the coverage is sufficient? I assume v3 could
>> still stand on v2's sholders?
>> Or if you plan to target a v2 rewrite for 10.0 only then I suppose it
>> could be a rewrite of v2 since it will be in a major version, but then we
>> need to know we can complete the work by then.
>>
>> Jan
>>
>> 16. jun. 2022 kl. 13:03 skrev Jason Gerlowski <ge...@gmail.com>:
>>
>> Hi all,
>>
>> I've been working over the last few months to get our "v2" API into
>> shape to eventually replace (and allow us to sunset) v1. At a high
>> level the two biggest next-steps are to (1) make whatever changes we'd
>> like to the v2 endpoints while they're still "experimental", and then
>> (2) start using them across Solr (Admin UI, SolrJ, etc.)
>>
>> The big question in my mind is: what all should change in the v2
>> endpoints as part of item #1 above. Do people like the current v2
>> design enough to move forward as-is (or with some minor tweaking)? Or
>> is there interest in seeing a larger re-alignment around other design
>> principles (e.g. REST, OpenAPI, etc.) while v2 is still
>> "experimental"?
>>
>> My impression from past discussions is that there's interest in seeing
>> broader changes to v2 (see [1] and [2] below). I tend to agree and
>> I'm willing to head up that effort. But I wanted to double-check
>> first that there was consensus on the overall idea of "broader
>> changes" before I start filing JIRAs and working out specifics.
>>
>> Assuming consensus, my next step will be to write up the current v2
>> endpoints and some proposed changes in a spreadsheet that we can
>> iterate on. (Feedback on specific APIs, changes, etc. definitely
>> welcome at that point once I get there.)
>>
>> Best,
>>
>> Jason
>>
>> [1] "Should v2 API be experimental" -
>> https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
>> [2] "Solr 9.0 Release Blockers"" -
>> https://lists.apache.org/thread/m7k2gvgxldkns7jqjnw1ghhqx7s3tpl1
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>>
>>
>>
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: v2 API "Broader Changes" and Next Steps
Posted by Jan Høydahl <ja...@cominvent.com>.
V2 has existed since v6.5 and has been documented in ref guide evner since. In 7.0 release announcement we said:
The new v2 API, exposed at /api/ and also supported via SolrJ, is now the preferred API, but /solr/ continues to work.
We have been showing v2 api commands in tabs for e.g. config api since 7.4. So users have been encouraged to switch for at least 3-4 years.
Jan Høydahl
> 16. jun. 2022 kl. 16:16 skrev Eric Pugh <ep...@opensourceconnections.com>:
>
>
> I don’t have the link, but I think we agree that v2 *could* evolve, hence the whole experimental thing.
>
> I worry that the efforts of maintaining three versions of the api will stall us. Especially if the changes we want to make aren’t wholesale changes. Now if we introduced a graphql based api, that might justify the v3 tag.
>
> My gut feeling is that no one uses v2 yet, so the end user impact is minimal.
>
> Maybe run a quick survey to decide that?
>
>> On Thu, Jun 16, 2022 at 5:41 AM Jan Høydahl <ja...@cominvent.com> wrote:
>> HI,
>>
>> Thanks for having the guts to start such a journey!
>> V2 has been around for several major releases and documented (it's in v6.6 guide https://solr.apache.org/guide/6_6/v2-api.html) long enough that we must assume a certain user base already.
>> The "experimental" warning in RefGuide is not prominent between 6.6 and 8.11, looks like it was added in the 9.0 guide.
>> Thus I'm not sure we should act as if the entire v2 design is up for change.
>>
>> So perhaps the next-gen, OpenApi style APIs should be v3 and then deprecate both v1 and v2 once the coverage is sufficient? I assume v3 could still stand on v2's sholders?
>> Or if you plan to target a v2 rewrite for 10.0 only then I suppose it could be a rewrite of v2 since it will be in a major version, but then we need to know we can complete the work by then.
>>
>> Jan
>>
>>> 16. jun. 2022 kl. 13:03 skrev Jason Gerlowski <ge...@gmail.com>:
>>>
>>> Hi all,
>>>
>>> I've been working over the last few months to get our "v2" API into
>>> shape to eventually replace (and allow us to sunset) v1. At a high
>>> level the two biggest next-steps are to (1) make whatever changes we'd
>>> like to the v2 endpoints while they're still "experimental", and then
>>> (2) start using them across Solr (Admin UI, SolrJ, etc.)
>>>
>>> The big question in my mind is: what all should change in the v2
>>> endpoints as part of item #1 above. Do people like the current v2
>>> design enough to move forward as-is (or with some minor tweaking)? Or
>>> is there interest in seeing a larger re-alignment around other design
>>> principles (e.g. REST, OpenAPI, etc.) while v2 is still
>>> "experimental"?
>>>
>>> My impression from past discussions is that there's interest in seeing
>>> broader changes to v2 (see [1] and [2] below). I tend to agree and
>>> I'm willing to head up that effort. But I wanted to double-check
>>> first that there was consensus on the overall idea of "broader
>>> changes" before I start filing JIRAs and working out specifics.
>>>
>>> Assuming consensus, my next step will be to write up the current v2
>>> endpoints and some proposed changes in a spreadsheet that we can
>>> iterate on. (Feedback on specific APIs, changes, etc. definitely
>>> welcome at that point once I get there.)
>>>
>>> Best,
>>>
>>> Jason
>>>
>>> [1] "Should v2 API be experimental" -
>>> https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
>>> [2] "Solr 9.0 Release Blockers"" -
>>> https://lists.apache.org/thread/m7k2gvgxldkns7jqjnw1ghhqx7s3tpl1
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>
>>
Re: v2 API "Broader Changes" and Next Steps
Posted by Eric Pugh <ep...@opensourceconnections.com>.
I don’t have the link, but I think we agree that v2 *could* evolve, hence
the whole experimental thing.
I worry that the efforts of maintaining three versions of the api will
stall us. Especially if the changes we want to make aren’t wholesale
changes. Now if we introduced a graphql based api, that might justify
the v3 tag.
My gut feeling is that no one uses v2 yet, so the end user impact is
minimal.
Maybe run a quick survey to decide that?
On Thu, Jun 16, 2022 at 5:41 AM Jan Høydahl <ja...@cominvent.com> wrote:
> HI,
>
> Thanks for having the guts to start such a journey!
> V2 has been around for several major releases and documented (it's in v6.6
> guide https://solr.apache.org/guide/6_6/v2-api.html) long enough that we
> must assume a certain user base already.
> The "experimental" warning in RefGuide is not prominent between 6.6 and
> 8.11, looks like it was added in the 9.0 guide.
> Thus I'm not sure we should act as if the entire v2 design is up for
> change.
>
> So perhaps the next-gen, OpenApi style APIs should be v3 and then
> deprecate both v1 and v2 once the coverage is sufficient? I assume v3 could
> still stand on v2's sholders?
> Or if you plan to target a v2 rewrite for 10.0 only then I suppose it
> could be a rewrite of v2 since it will be in a major version, but then we
> need to know we can complete the work by then.
>
> Jan
>
> 16. jun. 2022 kl. 13:03 skrev Jason Gerlowski <ge...@gmail.com>:
>
> Hi all,
>
> I've been working over the last few months to get our "v2" API into
> shape to eventually replace (and allow us to sunset) v1. At a high
> level the two biggest next-steps are to (1) make whatever changes we'd
> like to the v2 endpoints while they're still "experimental", and then
> (2) start using them across Solr (Admin UI, SolrJ, etc.)
>
> The big question in my mind is: what all should change in the v2
> endpoints as part of item #1 above. Do people like the current v2
> design enough to move forward as-is (or with some minor tweaking)? Or
> is there interest in seeing a larger re-alignment around other design
> principles (e.g. REST, OpenAPI, etc.) while v2 is still
> "experimental"?
>
> My impression from past discussions is that there's interest in seeing
> broader changes to v2 (see [1] and [2] below). I tend to agree and
> I'm willing to head up that effort. But I wanted to double-check
> first that there was consensus on the overall idea of "broader
> changes" before I start filing JIRAs and working out specifics.
>
> Assuming consensus, my next step will be to write up the current v2
> endpoints and some proposed changes in a spreadsheet that we can
> iterate on. (Feedback on specific APIs, changes, etc. definitely
> welcome at that point once I get there.)
>
> Best,
>
> Jason
>
> [1] "Should v2 API be experimental" -
> https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
> [2] "Solr 9.0 Release Blockers"" -
> https://lists.apache.org/thread/m7k2gvgxldkns7jqjnw1ghhqx7s3tpl1
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
>
Re: v2 API "Broader Changes" and Next Steps
Posted by Jan Høydahl <ja...@cominvent.com>.
HI,
Thanks for having the guts to start such a journey!
V2 has been around for several major releases and documented (it's in v6.6 guide https://solr.apache.org/guide/6_6/v2-api.html <https://solr.apache.org/guide/6_6/v2-api.html>) long enough that we must assume a certain user base already.
The "experimental" warning in RefGuide is not prominent between 6.6 and 8.11, looks like it was added in the 9.0 guide.
Thus I'm not sure we should act as if the entire v2 design is up for change.
So perhaps the next-gen, OpenApi style APIs should be v3 and then deprecate both v1 and v2 once the coverage is sufficient? I assume v3 could still stand on v2's sholders?
Or if you plan to target a v2 rewrite for 10.0 only then I suppose it could be a rewrite of v2 since it will be in a major version, but then we need to know we can complete the work by then.
Jan
> 16. jun. 2022 kl. 13:03 skrev Jason Gerlowski <ge...@gmail.com>:
>
> Hi all,
>
> I've been working over the last few months to get our "v2" API into
> shape to eventually replace (and allow us to sunset) v1. At a high
> level the two biggest next-steps are to (1) make whatever changes we'd
> like to the v2 endpoints while they're still "experimental", and then
> (2) start using them across Solr (Admin UI, SolrJ, etc.)
>
> The big question in my mind is: what all should change in the v2
> endpoints as part of item #1 above. Do people like the current v2
> design enough to move forward as-is (or with some minor tweaking)? Or
> is there interest in seeing a larger re-alignment around other design
> principles (e.g. REST, OpenAPI, etc.) while v2 is still
> "experimental"?
>
> My impression from past discussions is that there's interest in seeing
> broader changes to v2 (see [1] and [2] below). I tend to agree and
> I'm willing to head up that effort. But I wanted to double-check
> first that there was consensus on the overall idea of "broader
> changes" before I start filing JIRAs and working out specifics.
>
> Assuming consensus, my next step will be to write up the current v2
> endpoints and some proposed changes in a spreadsheet that we can
> iterate on. (Feedback on specific APIs, changes, etc. definitely
> welcome at that point once I get there.)
>
> Best,
>
> Jason
>
> [1] "Should v2 API be experimental" -
> https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
> [2] "Solr 9.0 Release Blockers"" -
> https://lists.apache.org/thread/m7k2gvgxldkns7jqjnw1ghhqx7s3tpl1
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
Re: v2 API "Broader Changes" and Next Steps
Posted by Noble Paul <no...@gmail.com>.
Thanks Jason for spearheading this effort.
I'll be biased because I built it. So, I refrain from a vote. However, I
shall contribute constructively to the discussion
On Thu, Jun 16, 2022 at 9:05 PM Jason Gerlowski <ge...@gmail.com>
wrote:
> Hi all,
>
> I've been working over the last few months to get our "v2" API into
> shape to eventually replace (and allow us to sunset) v1. At a high
> level the two biggest next-steps are to (1) make whatever changes we'd
> like to the v2 endpoints while they're still "experimental", and then
> (2) start using them across Solr (Admin UI, SolrJ, etc.)
>
> The big question in my mind is: what all should change in the v2
> endpoints as part of item #1 above. Do people like the current v2
> design enough to move forward as-is (or with some minor tweaking)? Or
> is there interest in seeing a larger re-alignment around other design
> principles (e.g. REST, OpenAPI, etc.) while v2 is still
> "experimental"?
>
> My impression from past discussions is that there's interest in seeing
> broader changes to v2 (see [1] and [2] below). I tend to agree and
> I'm willing to head up that effort. But I wanted to double-check
> first that there was consensus on the overall idea of "broader
> changes" before I start filing JIRAs and working out specifics.
>
> Assuming consensus, my next step will be to write up the current v2
> endpoints and some proposed changes in a spreadsheet that we can
> iterate on. (Feedback on specific APIs, changes, etc. definitely
> welcome at that point once I get there.)
>
> Best,
>
> Jason
>
> [1] "Should v2 API be experimental" -
> https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
> [2] "Solr 9.0 Release Blockers"" -
> https://lists.apache.org/thread/m7k2gvgxldkns7jqjnw1ghhqx7s3tpl1
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>
--
-----------------------------------------------------
Noble Paul
Re: v2 API "Broader Changes" and Next Steps
Posted by Mike Drob <md...@mdrob.com>.
I suspect that #2 will inspire #1, once we start trying to use it
everywhere in SolrJ and other areas we’ll see patterns and antipatterns
emerge and guide us on what the API should be.
On Thu, Jun 16, 2022 at 6:05 AM Jason Gerlowski <ge...@gmail.com>
wrote:
> Hi all,
>
> I've been working over the last few months to get our "v2" API into
> shape to eventually replace (and allow us to sunset) v1. At a high
> level the two biggest next-steps are to (1) make whatever changes we'd
> like to the v2 endpoints while they're still "experimental", and then
> (2) start using them across Solr (Admin UI, SolrJ, etc.)
>
> The big question in my mind is: what all should change in the v2
> endpoints as part of item #1 above. Do people like the current v2
> design enough to move forward as-is (or with some minor tweaking)? Or
> is there interest in seeing a larger re-alignment around other design
> principles (e.g. REST, OpenAPI, etc.) while v2 is still
> "experimental"?
>
> My impression from past discussions is that there's interest in seeing
> broader changes to v2 (see [1] and [2] below). I tend to agree and
> I'm willing to head up that effort. But I wanted to double-check
> first that there was consensus on the overall idea of "broader
> changes" before I start filing JIRAs and working out specifics.
>
> Assuming consensus, my next step will be to write up the current v2
> endpoints and some proposed changes in a spreadsheet that we can
> iterate on. (Feedback on specific APIs, changes, etc. definitely
> welcome at that point once I get there.)
>
> Best,
>
> Jason
>
> [1] "Should v2 API be experimental" -
> https://lists.apache.org/thread/t342hl7lvt5b4qmb5vrrh7tvmdjlbb22
> [2] "Solr 9.0 Release Blockers"" -
> https://lists.apache.org/thread/m7k2gvgxldkns7jqjnw1ghhqx7s3tpl1
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>