You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "Tim Grein (Jira)" <ji...@apache.org> on 2024/07/24 14:29:00 UTC
[jira] [Created] (CALCITE-6498) Elasticsearch nested mappings do not work
Tim Grein created CALCITE-6498:
----------------------------------
Summary: Elasticsearch nested mappings do not work
Key: CALCITE-6498
URL: https://issues.apache.org/jira/browse/CALCITE-6498
Project: Calcite
Issue Type: Bug
Components: elasticsearch-adapter
Reporter: Tim Grein
Assignee: Tim Grein
Fix For: 1.38.0
`EmbeddedElasticsearchPolicy#
applyMapping` tries to apply a nested mapping by adding another `properties` field inside a parent field mapping. In Elasticsearch you need to use `fields` rather than `properties` for a nested mapping to work.
I've tested this with ES 7 and ES 6 (only including ES 7 example here in the issue description; for ES 6 you need to wrap everything under "mappings" in "_doc"):
ES 7:
{code:java}
PUT /some_index
{
"mappings": {
"properties": {
"some_field": {
"type": "text",
"properties": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}{code}
will lead to the following error:
{code:java}
{
"error" : {
"root_cause" : [
{
"type" : "mapper_parsing_exception",
"reason" : "Mapping definition for [some_field] has unsupported parameters: [properties : {keyword={type=keyword}}]"
}
],
"type" : "mapper_parsing_exception",
"reason" : "Failed to parse mapping [_doc]: Mapping definition for [some_field] has unsupported parameters: [properties : {keyword={type=keyword}}]",
"caused_by" : {
"type" : "mapper_parsing_exception",
"reason" : "Mapping definition for [some_field] has unsupported parameters: [properties : {keyword={type=keyword}}]"
}
},
"status" : 400
} {code}
Successful request:
{code:java}
PUT /some_index
{
"mappings": {
"properties": {
"some_field": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
} {code}
You'll encounter this error also, if you adapt the test data in ElasticsearchAdapterTest to include nested fields:
{code:java}
@BeforeAll
public static void setupInstance() throws Exception {
// "city.keyword" is a nested field with type "keyword"
final Map<String, String> mapping =
ImmutableMap.of("city", "text", "city.keyword", "keyword", "state", "keyword", "pop", "long"); {code}
Error:
{code:java}
{...
{"type":"mapper_parsing_exception","reason":"unknown parameter [properties] on mapper [city] of type [text]"}
...} {code}
Looking at `ElasticsearchJson#visitMappingProperties` I assume this will lead to a similar issue, which I'll double-check.
This is related to https://issues.apache.org/jira/browse/CALCITE-3027 as you want to detect, if you perform a `LIKE` operator on a purely `text` mapped field, which will lead to weird semantics (`text` mapped fields are analyzed/broken up into several tokens), if you do not prevent it. Usually you have a nested `keyword` mapping for a `text` field, which you can fallback to. So this is rather important to work correctly overall and in the tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)