You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "janhoy (via GitHub)" <gi...@apache.org> on 2026/09/24 18:47:15 UTC
[PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
janhoy opened a new pull request, #4943:
URL: https://github.com/apache/solr/pull/4943
https://issues.apache.org/jira/browse/SOLR-18479
# Description
Fields, dynamic fields and field types each have `PUT`/`DELETE` routes in the v2 Schema API. Copy-fields were the one resource reachable only through `/schema/bulk` — the routes existed but were commented out, with a TODO noting that copy-field rules have no name to address them by.
# Philosophy
**Address rules by their source field.** A copy-field rule has no name, but it does have a source, and `/copyfields/{sourceField}` is a well-defined resource: *the set of rules copying from that field*. Every verb acts on that set.
**The verb says what happens to the set.** `PUT` replaces it, `POST` adds to it, `DELETE` removes from it. That distinction matters because the underlying `add-copy-field` is purely additive and does **not** check for duplicates — re-adding an existing rule makes the source get copied twice at index time. So `PUT` clears before adding and `POST` skips destinations already present, which makes both safe to repeat. A client that times out never has to know whether its request landed.
**Diff under the schema update lock.** Working out which rules to drop or skip happens inside `SchemaManager`, in two internal-only ops (`upsert-copy-field`, `append-copy-field`) alongside the existing `upsert-field`. `doOperations` refreshes the schema under the lock and replays ops on retry, so a diff computed in the resource layer would go stale exactly when it matters.
# Endpoints
All under `/api/collections/{collection}/schema` (or `/cores/{core}`). Write operations require `schema-edit`, the read requires `schema-read`.
### `GET /copyfields/{sourceField}`
Lists the rules copying from `sourceField`, covering glob rules as well as explicit ones.
- **200** — `{"copyFields":[{"source":…,"dest":…,"maxChars":…}]}`; an empty list if the source has no rules.
> Preferred over `GET /copyfields?source.fl=…`, which remains the only way to filter by destination or by several sources at once.
Note the response is a list of individual rules, not the set-shaped body the write verbs take. That is deliberate: the shape is identical to the existing `/copyfields`, and `maxChars` is per rule, so `{"source":…,"destinations":[…]}` could not represent a source whose rules have differing limits.
### `PUT /copyfields/{sourceField}`
Sets the complete destination list for the source, removing any rule left out. Idempotent.
- **body** — `{"destinations":["a","b"],"maxChars":100}`. `destinations` is required; a bare string is accepted for a single destination, and `dest` is accepted as an alias because v1 commands deserialize through the same model. `maxChars` optional. An empty list clears the source.
- **200** — applied.
- **400** — `destinations` missing, a body `source` contradicting the path, or the schema rejects a rule.
### `POST /copyfields/{sourceField}`
Adds destinations, leaving the source's existing rules in place. Destinations already present are skipped, so repeating is harmless. Idempotent.
- **body / responses** — as `PUT`.
### `DELETE /copyfields/{sourceField}`
Removes every rule copying from the source.
- **200** — applied.
- **404** — the source has no copy-field rules.
### `DELETE /copyfields/{sourceField}/{destinationFields}`
Removes the named rules. `destinationFields` is one destination or a comma-separated list.
- **200** — applied.
- **400** — the segment names no destinations, or a named rule does not exist.
# Tests
`V2CopyFieldApiTest` covers all six routes: replace, append, idempotency of both, per-destination and bulk delete, the transactional rollback of a rejected replace, non-interference with other sources, and the error cases. Error-case unit coverage in `V2UpdateSchemaErrorCaseTests`. Ref guide updated. `./gradlew check` passes.
PR authored using Claude AI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on code in PR #4943:
URL: https://github.com/apache/solr/pull/4943#discussion_r4097825886
##########
solr/api/src/java/org/apache/solr/client/api/endpoint/UpdateSchemaApi.java:
##########
@@ -112,25 +113,54 @@ SolrJerseyResponse addFieldType(
SolrJerseyResponse deleteFieldType(@PathParam("fieldTypeName") String fieldTypeName)
throws Exception;
- // TODO Gah! Copyfields don't currently have names for us to create/delete by name in an API like
- // /schema/copyfields/{copyFieldName}...figure out how to address this in a way consistent with
- // all our other APIs. (In the meantime, the functionality is exposed through the bulk API at
- // least.
- // @POST
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(
- // summary = "Add a new copy-field with the specified name.",
- // tags = {"schema"})
- // SolrJerseyResponse addCopyField(@PathParam("copyFieldName") String copyFieldName,
- // @RequestBody SchemaChangeOperation.AddCopyField requestBody) throws Exception;
- //
- // @DELETE
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(summary = "Remove the copy-field with the specified name.", tags = {"schema"})
- // SolrJerseyResponse deleteCopyField(@PathParam("copyFieldName") String copyFieldName) throws
- // Exception;
+ // Copy-field rules have no name of their own, so unlike the other schema resources they are
+ // addressed by their source field. A single request therefore covers all of the rules that
+ // share that source.
+ @PUT
+ @Path("/copyfields/{sourceField}")
+ @StoreApiParameters
+ @Operation(
+ summary =
+ "Set the copy-field rules for the specified source field. The given destinations "
+ + "replace any rules that source already has.",
Review Comment:
A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #4943:
URL: https://github.com/apache/solr/pull/4943#issuecomment-5834288727
[eric_solr_18479_changes.patch](https://github.com/user-attachments/files/32657183/eric_solr_18479_changes.patch)
^^ is a patch file that converts to using strong typeing and updates the schema.js.....
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #4943:
URL: https://github.com/apache/solr/pull/4943#issuecomment-5826549816
is DELETE /api/collections/collName/schema/copyfields/srcField/dstFields using `dstFields` a bit not aligned with POST /api/collections/collName/schema/copyfields/srcField {"destinations":[]...} ? Should they both be `dstFields` ? not `destiations`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on code in PR #4943:
URL: https://github.com/apache/solr/pull/4943#discussion_r4097825886
##########
solr/api/src/java/org/apache/solr/client/api/endpoint/UpdateSchemaApi.java:
##########
@@ -112,25 +113,54 @@ SolrJerseyResponse addFieldType(
SolrJerseyResponse deleteFieldType(@PathParam("fieldTypeName") String fieldTypeName)
throws Exception;
- // TODO Gah! Copyfields don't currently have names for us to create/delete by name in an API like
- // /schema/copyfields/{copyFieldName}...figure out how to address this in a way consistent with
- // all our other APIs. (In the meantime, the functionality is exposed through the bulk API at
- // least.
- // @POST
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(
- // summary = "Add a new copy-field with the specified name.",
- // tags = {"schema"})
- // SolrJerseyResponse addCopyField(@PathParam("copyFieldName") String copyFieldName,
- // @RequestBody SchemaChangeOperation.AddCopyField requestBody) throws Exception;
- //
- // @DELETE
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(summary = "Remove the copy-field with the specified name.", tags = {"schema"})
- // SolrJerseyResponse deleteCopyField(@PathParam("copyFieldName") String copyFieldName) throws
- // Exception;
+ // Copy-field rules have no name of their own, so unlike the other schema resources they are
+ // addressed by their source field. A single request therefore covers all of the rules that
+ // share that source.
+ @PUT
+ @Path("/copyfields/{sourceField}")
+ @StoreApiParameters
+ @Operation(
+ summary =
+ "Set the copy-field rules for the specified source field. The given destinations "
+ + "replace any rules that source already has.",
Review Comment:
A disconnect with the PUT (and POST) body format is that all destinations share the same `maxChars`. For POST (additive), that can be defended by sending multiple POST requests, for each group of destinations that share same `maxChars`. But this won't work for PUT. An alternative is making the body an array of objects:
```json
[
{"destinations": ["foo"], "maxChars": 100}
]
```
We could also keep the 1:1 plain object format, and we could use `dest` instead of `destination` to stick with what v1 uses??
```json
[
{"dest": "foo", "maxChars": 100}
{"dest": "bar", "maxChars": 20}
]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #4943:
URL: https://github.com/apache/solr/pull/4943#discussion_r4105110613
##########
solr/core/src/test/org/apache/solr/handler/admin/api/V2CopyFieldApiTest.java:
##########
@@ -0,0 +1,317 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.handler.admin.api;
+
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import org.apache.solr.client.solrj.RemoteSolrException;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.client.solrj.request.V2Request;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.common.params.MapSolrParams;
+import org.apache.solr.common.util.NamedList;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+/** Tests the v2 copy-field endpoints, which address rules by their source field. */
+public class V2CopyFieldApiTest extends SolrCloudTestCase {
+
+ private static final String COLLECTION = "v2CopyFieldApiTest";
+ private static final String SOURCE = "cf_source";
+ private static final String DEST_ONE = "cf_dest_one";
+ private static final String DEST_TWO = "cf_dest_two";
+
+ @BeforeClass
+ public static void setupCluster() throws Exception {
+ System.setProperty("managed.schema.mutable", "true");
+ configureCluster(1).addConfig("conf", configset("cloud-managed")).configure();
+
+ CollectionAdminRequest.createCollection(COLLECTION, "conf", 1, 1)
+ .process(cluster.getSolrClient());
+ cluster.waitForActiveCollection(COLLECTION, 1, 1);
+
+ for (String field : List.of(SOURCE, DEST_ONE, DEST_TWO)) {
+ new V2Request.Builder(schemaPath("/fields/" + field))
Review Comment:
I was surprised to see this V2Request.builder used, as I didn't think that was our pattern. I checked and the pattern is like "new AliasesApi.DeleteAlias(aliasName)" in the tests.. Claude tells me we are missing the corresponding SolrJ methods... Not sure why yet.
##########
solr/solr-ref-guide/modules/indexing-guide/pages/schema-api.adoc:
##########
@@ -490,6 +490,7 @@ s|Required |Default: none
|===
+
A field or an array of fields to which the source field will be copied.
+In the v2 API this attribute is named `destinations`; `dest` is still accepted as an alias.
Review Comment:
i don't think at this point, with v2 being experimental, that we need to have back compat... adds confusion and we definitly don't do it in ohter places!
##########
solr/api/src/java/org/apache/solr/client/api/endpoint/UpdateSchemaApi.java:
##########
@@ -112,25 +113,54 @@ SolrJerseyResponse addFieldType(
SolrJerseyResponse deleteFieldType(@PathParam("fieldTypeName") String fieldTypeName)
throws Exception;
- // TODO Gah! Copyfields don't currently have names for us to create/delete by name in an API like
Review Comment:
Gah! ;-). Makes me smile this morning.
##########
solr/api/src/java/org/apache/solr/client/api/endpoint/UpdateSchemaApi.java:
##########
@@ -112,25 +113,54 @@ SolrJerseyResponse addFieldType(
SolrJerseyResponse deleteFieldType(@PathParam("fieldTypeName") String fieldTypeName)
throws Exception;
- // TODO Gah! Copyfields don't currently have names for us to create/delete by name in an API like
- // /schema/copyfields/{copyFieldName}...figure out how to address this in a way consistent with
- // all our other APIs. (In the meantime, the functionality is exposed through the bulk API at
- // least.
- // @POST
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(
- // summary = "Add a new copy-field with the specified name.",
- // tags = {"schema"})
- // SolrJerseyResponse addCopyField(@PathParam("copyFieldName") String copyFieldName,
- // @RequestBody SchemaChangeOperation.AddCopyField requestBody) throws Exception;
- //
- // @DELETE
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(summary = "Remove the copy-field with the specified name.", tags = {"schema"})
- // SolrJerseyResponse deleteCopyField(@PathParam("copyFieldName") String copyFieldName) throws
- // Exception;
+ // Copy-field rules have no name of their own, so unlike the other schema resources they are
+ // addressed by their source field. A single request therefore covers all of the rules that
+ // share that source.
+ @PUT
+ @Path("/copyfields/{sourceField}")
+ @StoreApiParameters
+ @Operation(
+ summary =
+ "Set the copy-field rules for the specified source field. The given destinations "
+ + "replace any rules that source already has.",
Review Comment:
Looking back above... The more fine grained "you must provide maxChars" seems totally fine and better than 1 value to set them all version! These apis aren't used much, it's not like this traffic is going to stress the network back and forth ;-).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on PR #4943:
URL: https://github.com/apache/solr/pull/4943#issuecomment-5845479565
Feel free to commit to this PR branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on code in PR #4943:
URL: https://github.com/apache/solr/pull/4943#discussion_r4104174753
##########
solr/api/src/java/org/apache/solr/client/api/endpoint/UpdateSchemaApi.java:
##########
@@ -112,25 +113,54 @@ SolrJerseyResponse addFieldType(
SolrJerseyResponse deleteFieldType(@PathParam("fieldTypeName") String fieldTypeName)
throws Exception;
- // TODO Gah! Copyfields don't currently have names for us to create/delete by name in an API like
- // /schema/copyfields/{copyFieldName}...figure out how to address this in a way consistent with
- // all our other APIs. (In the meantime, the functionality is exposed through the bulk API at
- // least.
- // @POST
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(
- // summary = "Add a new copy-field with the specified name.",
- // tags = {"schema"})
- // SolrJerseyResponse addCopyField(@PathParam("copyFieldName") String copyFieldName,
- // @RequestBody SchemaChangeOperation.AddCopyField requestBody) throws Exception;
- //
- // @DELETE
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(summary = "Remove the copy-field with the specified name.", tags = {"schema"})
- // SolrJerseyResponse deleteCopyField(@PathParam("copyFieldName") String copyFieldName) throws
- // Exception;
+ // Copy-field rules have no name of their own, so unlike the other schema resources they are
+ // addressed by their source field. A single request therefore covers all of the rules that
+ // share that source.
+ @PUT
+ @Path("/copyfields/{sourceField}")
+ @StoreApiParameters
+ @Operation(
+ summary =
+ "Set the copy-field rules for the specified source field. The given destinations "
+ + "replace any rules that source already has.",
Review Comment:
Or we could give up on fine grained v2 paths for anything but perhaps `GET <srcField>` and `DELETE <srcField>`, and resolve to the `/bulk` endpoint for most modifications of copyFields?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on PR #4943:
URL: https://github.com/apache/solr/pull/4943#issuecomment-5831882810
> is DELETE /api/collections/collName/schema/copyfields/srcField/dstFields using `dstFields` a bit not aligned with POST /api/collections/collName/schema/copyfields/srcField {"destinations":[]...} ? Should they both be `dstFields` ? not `destiations`?
The delete api operates on plain field-names only, the POST api needs a body.
Example use of DELETE, which would delete to copyField rules:
`DELETE /api/collections/collName/schema/copyfields/title/title_facet,title_sort`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-18479: Add v2 copy-field endpoints to the Schema API [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #4943:
URL: https://github.com/apache/solr/pull/4943#discussion_r4100965628
##########
solr/api/src/java/org/apache/solr/client/api/endpoint/UpdateSchemaApi.java:
##########
@@ -112,25 +113,54 @@ SolrJerseyResponse addFieldType(
SolrJerseyResponse deleteFieldType(@PathParam("fieldTypeName") String fieldTypeName)
throws Exception;
- // TODO Gah! Copyfields don't currently have names for us to create/delete by name in an API like
- // /schema/copyfields/{copyFieldName}...figure out how to address this in a way consistent with
- // all our other APIs. (In the meantime, the functionality is exposed through the bulk API at
- // least.
- // @POST
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(
- // summary = "Add a new copy-field with the specified name.",
- // tags = {"schema"})
- // SolrJerseyResponse addCopyField(@PathParam("copyFieldName") String copyFieldName,
- // @RequestBody SchemaChangeOperation.AddCopyField requestBody) throws Exception;
- //
- // @DELETE
- // @Path("/copyfields/{copyFieldName}")
- // @StoreApiParameters
- // @Operation(summary = "Remove the copy-field with the specified name.", tags = {"schema"})
- // SolrJerseyResponse deleteCopyField(@PathParam("copyFieldName") String copyFieldName) throws
- // Exception;
+ // Copy-field rules have no name of their own, so unlike the other schema resources they are
+ // addressed by their source field. A single request therefore covers all of the rules that
+ // share that source.
+ @PUT
+ @Path("/copyfields/{sourceField}")
+ @StoreApiParameters
+ @Operation(
+ summary =
+ "Set the copy-field rules for the specified source field. The given destinations "
+ + "replace any rules that source already has.",
Review Comment:
Let's figure out what we want for the next ten years, not the previous ten years. We want to make V2 better.. Not drag along V1 concepts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org