copyField dest is not an explicit field and doesn't match a dynamic field

[dmitri maziuk <dm...@gmail.com>]

Hi all,

trying to POST to .../update/json/docs, payload includes 
"DISPLAY_MAPPING" : "foo", and the result is a 500 with
```
null:org.apache.solr.common.SolrException: copyField dest 
:'doc.DISPLAY_MAPPING_str' is not an explicit field and doesn't match a 
dynamicField.
```

Which is fine as far as that goes: the only place this can come from is 
AddSchemaFields w/ the default mapping as:
```
<str name="valueClass">java.lang.String</str>
<str name="fieldType">text_general</str>
<lst name="copyField">
   <str name="dest>*_str</str>
   <int name="maxChars">256</int>
</lst>
```

Except that DISPLAY_MAPPING exists in the schema as 
org.apache.solr.schema.StrField and I can't figure out why it is 
triggering AddSchemaField on it in the first place.

Any suggestions?

TIA
Dima

SOLVED (-ish) Re: copyField dest is not an explicit field and doesn't match a dynamic field

[dmitri maziuk <dm...@gmail.com>]

On 2022-03-10 6:07 PM, Shawn Heisey wrote:

> I am not very familiar with the update processor that adds fields.  But 
> if I understand that correctly, it means that for any field matching 
> that specification, it's going to do a copyField

So from empirical observation,
- it does not add a copyField for the field that already exists in the 
schema, *but*
- it seems to check/validate the rule/schema *first* and decide whether 
it'll need to add the field later.

So it fails on the missing def'n even though the rule is never actually 
triggered.

It's arguably a bug: ideally it should check the whole thing up front at 
startup and refuse to run when the rules in solrconfig are inconsistent 
with the schema. Or it could fail to create the *_str field when it's 
actually needed -- the current behaviour is neither here nor there.

Thanks again,
Dima

Re: copyField dest is not an explicit field and doesn't match a dynamic field

[dmitri maziuk <dm...@gmail.com>]

On 2022-03-10 6:07 PM, Shawn Heisey wrote:

> I am not very familiar with the update processor that adds fields.  But 
> if I understand that correctly, it means that for any field matching 
> that specification, it's going to do a copyField

...

Thanks, I get that.

The comment (inherited from 6.x days) says it is for adding "unknown 
fields to the schema" and it looks like the comment is actually wrong: 
it seems to be adding _str for *every* field (or at least more than just 
that one: I managed to get trigger the same error on other fields). 
Which doesn't sound like a smart thing to do and actually doesn't make 
sense for the field that initially triggered this, and some others.

Looks like that default definition for '<dynamicField name="*_str"' 
somehow got lost in the 6.6->8.7 upgrade here. Or maybe got dropped 
deliberately.

Thanks again,
Dima

Re: copyField dest is not an explicit field and doesn't match a dynamic field

[Shawn Heisey <ap...@elyograg.org>]

On 3/10/2022 4:24 PM, dmitri maziuk wrote:
> Which is fine as far as that goes: the only place this can come from 
> is AddSchemaFields w/ the default mapping as:
> ```
> <str name="valueClass">java.lang.String</str>
> <str name="fieldType">text_general</str>
> <lst name="copyField">
>   <str name="dest>*_str</str>
>   <int name="maxChars">256</int>
> </lst>
> ```

I am not very familiar with the update processor that adds fields.  But 
if I understand that correctly, it means that for any field matching 
that specification, it's going to do a copyField, which means that if 
you have a string field named "DISPLAY_MAPPING" it's going to try to 
copy it to another field named "DISPLAY_MAPPING_str" ... and the error 
is saying that there's nothing in the schema that can handle the 
"DISPLAY_MAPPING_str" field.

You can either remove the copyField specification, or add something to 
the schema that will handle the destination field name.

The _default schema that comes with Solr contains the following, which 
would handle that:

<dynamicField name="*_str" type="strings" stored="false" 
docValues="true" indexed="false" useDocValuesAsStored="false"/>

Thanks,
Shawn