You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "gerlowskija (via GitHub)" <gi...@apache.org> on 2023/06/02 15:02:57 UTC
[GitHub] [solr] gerlowskija opened a new pull request, #1681: SOLR-16835: Generate Python client from OpenAPI spec
gerlowskija opened a new pull request, #1681:
URL: https://github.com/apache/solr/pull/1681
https://issues.apache.org/jira/browse/SOLR-16835
# Description
Currently, Solr only offers one "first-party" client, SolrJ, which can only be used within JVM environments. This is obviously limiting for our users.
Historically, keeping client bindings in multiple languages up to date has been too daunting for the project to undertake. But now that Solr produces an OpenAPI spec for its upcoming "v2" API, we have the ability to generate client bindings from that for a variety of languages with very little additional maintenance cost.
# Solution
This PR uses the 'openapi-generator' gradle plugin to create a Python client based on our OpenAPI spec. A few details:
- `./gradlew openApiValidate` to generate the OpenAPI spec and ensure it's valid to use for code-generation.
- `./gradlew openApiGenerate` to generate the Python client
- Generation uses a template that we can modify if we'd like to. (Currently this PR just uses template unmodified.)
- Once created, client is available at `solr/core/build/generated/python` and can be installed to the user's Python environment with `python setup.py install --user`
- Generated code uses urllib3 to send requests (though that's abstracted away entirely afaict)
- The organization (and to an extent, quality) of the generated code depends in part on some OpenAPI annotations that we don't use across-the-board currently. This PR adds those annotations to a few Java classes so the generated client is a little easier to demo. We already have other JIRA tickets for doing this more comprehensively though.
See below for a snippet that uses the generated client to create a collection:
```
import solr
from solr.api import collection_api
from solr.model.create_collection_request_body import CreateCollectionRequestBody
configuration = solr.Configuration(host = "http://localhost:8983/api")
with solr.ApiClient(configuration) as api_client:
api_instance = collection_api.CollectionApi(api_client)
create_params = CreateCollectionRequestBody(
name = "somecollection",
num_shards = 1
)
try:
api_response = api_instance.create_collection(create_collection_request_body = create_params)
pprint(api_response)
except solr.ApiException as e:
print("Exception when calling CollectionApi->create_collection: %s\n" % e)
```
# Tests
The code-generator has an option to generate tests for the generated client code. I've disabled that currently, trusting the generator to do its thing, but we can enable that if it's something we'd feel more comfortable using.
Other than those generated tests, this has mostly been manual testing on my part.
# Checklist
Please review the following and check all that apply:
- [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability.
- [x] I have created a Jira issue and added the issue ID to my pull request title.
- [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended)
- [x] I have developed this patch against the `main` branch.
- [ ] I have run `./gradlew check`.
- [ ] I have added tests for my changes.
- [ ] I have added documentation for the [Reference Guide](https://github.com/apache/solr/tree/main/solr/solr-ref-guide)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by Jason Gerlowski <ge...@gmail.com>.
Good idea! To be honest I thought I already had done that awhile back, but
I can't find the email. So maybe it never made it out of my drafts folder
: /
In any case I just started a new thread on users@ about this, titled:
"Feedback/Review Requested: 'Official' Solr Python Client". Hopefully
that'll garner the PR a few knowledgeable eyes.
Best,
Jason
On Sat, Oct 28, 2023 at 11:06 AM Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:
> Jason, maybe asking in solr users list, inviting python developers for help
> might be better for visibility?
>
> On Fri, 27 Oct, 2023, 7:37 pm gerlowskija (via GitHub), <gi...@apache.org>
> wrote:
>
> >
> > gerlowskija commented on PR #1681:
> > URL: https://github.com/apache/solr/pull/1681#issuecomment-1782979188
> >
> > Quick update on the status of this PR:
> >
> > The Java client generation I mentioned above has been merged for some
> > time now. So there's no "hard" blocker for getting this merged.
> >
> > My main concern at this point is that I don't know the Python
> ecosystem
> > well enough to weigh in on what folks want from a Python client. Maybe
> > OpenAPI's default Python template is good enough without any tweaking,
> but
> > I'm reluctant to assume that and commit absent feedback from someone that
> > actually knows and uses Python.
> >
> > If anyone out there has Python experience or currently uses a Python
> > client, could really use your input!
> >
> >
> > --
> > This is an automated message from the Apache Git Service.
> > To respond to the message, please log on to GitHub and use the
> > URL above to go to the specific comment.
> >
> > To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
> >
> > For queries about this service, please contact Infrastructure at:
> > users@infra.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
> > For additional commands, e-mail: issues-help@solr.apache.org
> >
> >
>
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by Ishan Chattopadhyaya <ic...@gmail.com>.
Jason, maybe asking in solr users list, inviting python developers for help
might be better for visibility?
On Fri, 27 Oct, 2023, 7:37 pm gerlowskija (via GitHub), <gi...@apache.org>
wrote:
>
> gerlowskija commented on PR #1681:
> URL: https://github.com/apache/solr/pull/1681#issuecomment-1782979188
>
> Quick update on the status of this PR:
>
> The Java client generation I mentioned above has been merged for some
> time now. So there's no "hard" blocker for getting this merged.
>
> My main concern at this point is that I don't know the Python ecosystem
> well enough to weigh in on what folks want from a Python client. Maybe
> OpenAPI's default Python template is good enough without any tweaking, but
> I'm reluctant to assume that and commit absent feedback from someone that
> actually knows and uses Python.
>
> If anyone out there has Python experience or currently uses a Python
> client, could really use your input!
>
>
> --
> This is an automated message from the Apache Git Service.
> To respond to the message, please log on to GitHub and use the
> URL above to go to the specific comment.
>
> To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
>
> For queries about this service, please contact Infrastructure at:
> users@infra.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
> For additional commands, e-mail: issues-help@solr.apache.org
>
>
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija merged PR #1681:
URL: https://github.com/apache/solr/pull/1681
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] gerlowskija commented on pull request #1681: SOLR-16835: Generate Python client from OpenAPI spec
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1573892626
I've left this as a "draft" to signal that I don't intend to merge it imminently, but this is largely ready to go if anyone wants to review or pull it down to try out. I'd really love some feedback if anyone has a chance to "kick the tires" on their own.
In terms of "timeline" for merging: there's another ticket similar to this one that's targeting Java code generation, and I'd like to make a little progress there before I merge here. (I don't want to do anything in this PR that makes Java-generation harder, or vice versa.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1834334142
Alright! #2079 has been merged, giving our generated clients a way to make rudimentary queries. With that in place, this PR now switches the ltr `train_and_upload_demo_model.py` example script to use the new client. The resulting Python code looks really nice IMO!
Now that we've finally gotten some dog-fooding around the new client, I've taken this PR out of "Draft" mode. Hopefully seeing the client in use will help generate some ideas and feedback. Pending additional feedback though, I'll aim to merge this sometime next week!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "MarcusSorealheis (via GitHub)" <gi...@apache.org>.
MarcusSorealheis commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1788342857
> I think we should call this client the 'Solr Admin Client' considering that's what it enables users to do.
In the package name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] stillalex commented on a diff in pull request #1681: SOLR-16835: Generate Python client from OpenAPI spec
Posted by "stillalex (via GitHub)" <gi...@apache.org>.
stillalex commented on code in PR #1681:
URL: https://github.com/apache/solr/pull/1681#discussion_r1221721126
##########
solr/core/build.gradle:
##########
@@ -29,6 +30,26 @@ resolve {
prettyPrint = true
}
+openApiValidate {
+ inputSpec = file("build/generated/openapi/openapi.json").toString()
+ recommend = true
+}
+tasks.withType(org.openapitools.generator.gradle.plugin.tasks.ValidateTask) {
+ dependsOn(resolve)
+}
+
+openApiGenerate {
+ generatorName = "python"
+ inputSpec = file("build/generated/openapi/openapi.json").toString()
+ outputDir = file("build/generated/python").toString()
Review Comment:
very minor suggestion here
```
outputDir = file("build/generated/$generatorName").toString()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on code in PR #1681:
URL: https://github.com/apache/solr/pull/1681#discussion_r1411269237
##########
solr/modules/ltr/example/train_and_upload_demo_model.py:
##########
@@ -8,8 +8,15 @@
from optparse import OptionParser
-solrQueryUrl = ""
+import solr
+from solr.api import querying_api
+def setupSolrClient(host, port):
+ '''Configures the Solr client with the specified Solr host/port'''
+ solr_client_config = solr.Configuration(
+ host = "http://" + host + ":" + str(port) + "/api"
Review Comment:
I think that's the plan eventually, yep.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "madrob (via GitHub)" <gi...@apache.org>.
madrob commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1784393637
Would a python client make any of our bats tests easier possibly instead of complex curl commands?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1785895502
> I think we should call this client the 'Solr Admin Client' considering that's what it enables users to do.
The APIs currently covered by our OAS are all admin APIs, sure. But that's a point-in-time reality that's likely to change as more APIs are converted to JAX-RS and added to our OAS. I definitely don't want folks that come here today to be confused when they can't (yet) find the Python method to run a query, etc. But I'm also pretty leery of over-specifying and then being stuck with a name that's misleading in the other direction.
Wdyt about calling out the current limitations with a big bold message at the top of the client's README, or some other doc-based solution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on code in PR #1681:
URL: https://github.com/apache/solr/pull/1681#discussion_r1411268892
##########
solr/modules/ltr/example/README.md:
##########
@@ -33,7 +33,15 @@ Please refer to the Solr Reference Guide's section on [Learning To Rank](https:/
`ln -s /Users/YourNameHere/Downloads/liblinear-2.1 ./modules/ltr/example/liblinear`
-3. Extract features, train a reranking model, and deploy it to Solr.
+3. Prepare your Python3 environment to run the training script.
+
+ `./gradlew solr:modules:ltr:syncPythonClientSourceCode`
+
+ `pushd solrclient && python3 setup.py install --user && popd`
Review Comment:
Ah yes, they're a fun little builtin!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on code in PR #1681:
URL: https://github.com/apache/solr/pull/1681#discussion_r1411268892
##########
solr/modules/ltr/example/README.md:
##########
@@ -33,7 +33,15 @@ Please refer to the Solr Reference Guide's section on [Learning To Rank](https:/
`ln -s /Users/YourNameHere/Downloads/liblinear-2.1 ./modules/ltr/example/liblinear`
-3. Extract features, train a reranking model, and deploy it to Solr.
+3. Prepare your Python3 environment to run the training script.
+
+ `./gradlew solr:modules:ltr:syncPythonClientSourceCode`
+
+ `pushd solrclient && python3 setup.py install --user && popd`
Review Comment:
Ah yes, they're a fun little builtin! Maybe you're right about them being too obscure though...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1810571548
> In the package name?
I won't put words in his mouth, but I interpreted Anshum's comment as being more about documentation and messaging around the client in general. As it stands today, folks that come in expecting a full querying/indexing client **are** going to be disappointed.
----
A small general update on this:
With the JS client PR (#2050) merged, I've got more time and attention to hopefully move this forward. My first step is to find a way to dogfood the client, ideally in Python code we already have. We have a surprising number of small Python scripts in our codebase (see below), if any of these are making Admin APIs calls they might be a good place to get some dog-fooding in. (This would require script users to run `gradle` to generate the client prior to running their desired script - but for dev-scripts this is probably a small enough hurdle - WDYT?)
```
$ find . -name "*.py" | grep -v nodejs | grep -v build
./solr/example/films/vectors/create_dataset.py
./solr/example/films/vectors/films.py
./solr/example/films/vectors/create_model.py
./solr/modules/ltr/example/libsvm_formatter.py
./solr/modules/ltr/example/train_and_upload_demo_model.py
./dev-tools/scripts/githubPRs.py
./dev-tools/scripts/smokeTestRelease.py
./dev-tools/scripts/refguide/gen-refguide-redirects.py
./dev-tools/scripts/addVersion.py
./dev-tools/scripts/scriptutil.py
./dev-tools/scripts/scaffoldNewModule.py
./dev-tools/scripts/releasedJirasRegex.py
./dev-tools/scripts/releaseWizard.py
./dev-tools/scripts/addDepsToChanges.py
./dev-tools/scripts/checkJavadocLinks.py
./dev-tools/scripts/reproduceJenkinsFailures.py
./dev-tools/scripts/create_line_file_docs.py
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1785810470
> > Up to this point I kindof thought the BATS framework was targeting bash specifically, but you two are the most familiar so if you think it'd be a decent fit I'm happy to give it a shot...
>
> Yeah, there is a niceness about just using bash in the BATS framework, and I'm not sure any of the curl stuff is that bad... I can definitely see some awesome integration tests using python, but that would target other stuff maybe?
> > Up to this point I kindof thought the BATS framework was targeting bash specifically, but you two are the most familiar so if you think it'd be a decent fit I'm happy to give it a shot...
>
> Yeah, there is a niceness about just using bash in the BATS framework, and I'm not sure any of the curl stuff is that bad... I can definitely see some awesome integration tests using python, but that would target other stuff maybe?
I could see a whole separate set of tests that demonstrate complex multi step processes (and are probably going to end up being brittle ;-( ) that we could add. For example, I added a .bats test that demonstrated running two SEPARATE SolrCloud instances, and using classic Replication to move data between them ;-). There are lots of things like splitting collections or using the REINDEXCOLLECTION command that might benefit by a separate "integration" test approach both to validate the code works, but also to show folks how to do these things. Heck, #1999 features a complex multi step bats test just to help me figure out how it would all work together ;-).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1786017548
> I've been putting off migrating anything to v2 in the AdminUI on the hope that somethign like a client would come along...
[Be careful what you wish for](https://github.com/apache/solr/pull/2050) @epugh 😛
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "HoustonPutman (via GitHub)" <gi...@apache.org>.
HoustonPutman commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1785715652
> Up to this point I kindof thought the BATS framework was targeting bash specifically, but you two are the most familiar so if you think it'd be a decent fit I'm happy to give it a shot...
Yeah, there is a niceness about just using bash in the BATS framework, and I'm not sure any of the curl stuff is that bad... I can definitely see some awesome integration tests using python, but that would target other stuff maybe?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1813223505
Probably our best bet for dogfooding the Python client is the `./solr/modules/ltr/example/train_and_upload_demo_model.py` script, which uses Python to send simple queries. This requires that the generated clients have some rudimentary support for querying, so I've taken a crack at adding this in #2079. That PR will block this one, unless others have thoughts around ways to validate or QA the generated Python client.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "anshumg (via GitHub)" <gi...@apache.org>.
anshumg commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1785855702
I think we should call this client the 'Solr Admin Client' considering that's what it enables users to do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1785811520
> > Here is another thought... What if we tried generating a Javascript client and using THAT in our admin UI?
>
> Very good ideas Eric. Using a generated JS client in AdminUI would give us a way to cut over from v1 to v2, and could be the start of a client that could be released as an independent artifact.
I've been putting off migrating anything to v2 in the AdminUI on the hope that somethign like a client would come along...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1784261686
> Here is another thought... What if we tried generating a Javascript client and using THAT in our admin UI?
Very good ideas Eric. Using a generated JS client in AdminUI would give us a way to cut over from v1 to v2, and could be the start of a client that could be released as an independent artifact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #1681:
URL: https://github.com/apache/solr/pull/1681#discussion_r1411167045
##########
solr/modules/ltr/example/README.md:
##########
@@ -33,7 +33,15 @@ Please refer to the Solr Reference Guide's section on [Learning To Rank](https:/
`ln -s /Users/YourNameHere/Downloads/liblinear-2.1 ./modules/ltr/example/liblinear`
-3. Extract features, train a reranking model, and deploy it to Solr.
+3. Prepare your Python3 environment to run the training script.
+
+ `./gradlew solr:modules:ltr:syncPythonClientSourceCode`
+
+ `pushd solrclient && python3 setup.py install --user && popd`
Review Comment:
pushd and popd are new to me! at first I thought it was somethign I needed to understand...
##########
solr/modules/ltr/example/train_and_upload_demo_model.py:
##########
@@ -8,8 +8,15 @@
from optparse import OptionParser
-solrQueryUrl = ""
+import solr
+from solr.api import querying_api
+def setupSolrClient(host, port):
+ '''Configures the Solr client with the specified Solr host/port'''
+ solr_client_config = solr.Configuration(
+ host = "http://" + host + ":" + str(port) + "/api"
Review Comment:
i guess at some point the /api will get dropped right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
[GitHub] [solr] stillalex commented on a diff in pull request #1681: SOLR-16835: Generate Python client from OpenAPI spec
Posted by "stillalex (via GitHub)" <gi...@apache.org>.
stillalex commented on code in PR #1681:
URL: https://github.com/apache/solr/pull/1681#discussion_r1221721126
##########
solr/core/build.gradle:
##########
@@ -29,6 +30,26 @@ resolve {
prettyPrint = true
}
+openApiValidate {
+ inputSpec = file("build/generated/openapi/openapi.json").toString()
+ recommend = true
+}
+tasks.withType(org.openapitools.generator.gradle.plugin.tasks.ValidateTask) {
+ dependsOn(resolve)
+}
+
+openApiGenerate {
+ generatorName = "python"
+ inputSpec = file("build/generated/openapi/openapi.json").toString()
+ outputDir = file("build/generated/python").toString()
Review Comment:
very minor suggestion here
```
outputDir = file("build/generated/$generatorName").toString()
```
[edit] actually nevermind, this is not a string! results are not what you'd expect. please ignore
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on code in PR #1681:
URL: https://github.com/apache/solr/pull/1681#discussion_r1415801779
##########
solr/modules/ltr/example/README.md:
##########
@@ -33,7 +33,15 @@ Please refer to the Solr Reference Guide's section on [Learning To Rank](https:/
`ln -s /Users/YourNameHere/Downloads/liblinear-2.1 ./modules/ltr/example/liblinear`
-3. Extract features, train a reranking model, and deploy it to Solr.
+3. Prepare your Python3 environment to run the training script.
+
+ `./gradlew solr:modules:ltr:syncPythonClientSourceCode`
+
+ `pushd solrclient && python3 setup.py install --user && popd`
Review Comment:
Alright, I've removed pushd/popd here as being perhaps too obscure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1782979188
Quick update on the status of this PR:
The Java client generation I mentioned above has been merged for some time now. So there's no "hard" blocker for getting this merged.
My main concern at this point is that I don't know the Python ecosystem well enough to weigh in on what folks want from a Python client. Maybe OpenAPI's default Python template is good enough without any tweaking, but I'm reluctant to assume that and commit absent feedback from someone that actually knows and uses Python.
If anyone out there has Python experience or currently uses a Python client, could really use your input!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1783330450
i wonder if there is anything we could do to "dogfood" the python client? Is there anything we wanted to build, like a script that interacts with Solr to do X that could be written in python, using this client library, to validate it....
A crazy thought would be to try and write a script that demonstrates "Doing something complex with Solr apis that are currently available in v2" using the client? Like, this file, but where appropriate we use the Python client?
https://github.com/apache/solr/blob/4e54090d2a10638dfd24311428084da56a043c89/solr/packaging/test/test_opennlp.bats
Here is another thought... What if we tried generating a Javascript client and using THAT in our admin UI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-16835: Generate Python client from OpenAPI spec [solr]
Posted by "gerlowskija (via GitHub)" <gi...@apache.org>.
gerlowskija commented on PR #1681:
URL: https://github.com/apache/solr/pull/1681#issuecomment-1785585637
> i wonder if there is anything we could do to "dogfood" the python client?
Yeah, it's still not ideal IMO but dogfooding might get us enough confidence to merge at least. If anyone has a good idea where dog-fooding might make sense, that'd be awesome. Curious to hear what @epugh thinks about @madrob 's suggestion above. Up to this point I kindof thought the BATS framework was targeting bash specifically, but you two are the most familiar so if you think it'd be a decent fit I'm happy to give it a shot...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org