[jira] [Updated] (NIFI-6404) PutElasticsearchHttp: Remove _type as being compulsory

["Koji Kawamura (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-6404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Koji Kawamura updated NIFI-6404:
--------------------------------
    Fix Version/s: 1.13.0
       Resolution: Fixed
           Status: Resolved  (was: Patch Available)

> PutElasticsearchHttp: Remove _type as being compulsory
> ------------------------------------------------------
>
>                 Key: NIFI-6404
>                 URL: https://issues.apache.org/jira/browse/NIFI-6404
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>    Affects Versions: 1.10.0, 1.9.2
>         Environment: Elasticsearch 7.x
>            Reporter: David Vassallo
>            Assignee: Joseph Gresock
>            Priority: Major
>             Fix For: 1.13.0
>
>
> In ES 7.x and above, document "type" is no longer compulsory and in fact is deprecated. When using the 1.9.2 version of PutElasticsearchHttp with ES v7.2, it still works however you'll see the following HTTP in the response:
>  
> {{HTTP/1.1 200 OK}}
>  *{{Warning: 299 Elasticsearch-7.2.0-508c38a "[types removal] Specifying types in bulk requests is deprecated."}}*
>  {{content-type: application/json; charset=UTF-8}}
>  
> The fix is relatively straightforward:
>  * In *PutElasticserachHttp.java*, remove the requirement of a compulsory "Type" property:
> {code:java}
> public static final PropertyDescriptor TYPE = new PropertyDescriptor.Builder()
>  .name("put-es-type")
>  .displayName("Type")
>  .description("The type of this document (used by Elasticsearch < 7.0 for indexing and searching). Leave empty for ES >= 7.0") // <-
>  .required(false) // <--------- CHANGE
>  .expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
>  .addValidator(StandardValidators.NON_EMPTY_EL_VALIDATOR)
>  .build();
> {code}
>  
>  * In *AbstractElasticsearchHttpProcessor.java*, check for the presence of "docType". If not present, assume elasticsearch 7.x or above and omit from bulk API URL:
>  
> {code:java}
> protected void buildBulkCommand(StringBuilder sb, String index, String docType, String indexOp, String id, String jsonString) {
>     if (indexOp.equalsIgnoreCase("index")) {
>         sb.append("{\"index\": { \"_index\": \"");
>         sb.append(StringEscapeUtils.escapeJson(index));
>         if (!(StringUtils.isEmpty(docType) | docType == null)){ // <--------- CHANGE START
>             sb.append("\", \"_type\": \"");
>             sb.append(StringEscapeUtils.escapeJson(docType));
>             sb.append("\"");
>         }                                // <--------- CHANGE END
>         if (!StringUtils.isEmpty(id)) { 
>             sb.append(", \"_id\": \"");
>             sb.append(StringEscapeUtils.escapeJson(id));
>             sb.append("\"");
>         } 
>         sb.append("}}\n");
>         sb.append(jsonString);
>         sb.append("\n");
>     } else if (indexOp.equalsIgnoreCase("upsert") || indexOp.equalsIgnoreCase("update")) {
>         sb.append("{\"update\": { \"_index\": \"");
>         sb.append(StringEscapeUtils.escapeJson(index));
>         sb.append("\", \"_type\": \"");
>         sb.append(StringEscapeUtils.escapeJson(docType));
>         sb.append("\", \"_id\": \"");
>         sb.append(StringEscapeUtils.escapeJson(id));
>         sb.append("\" }\n");
>         sb.append("{\"doc\": ");
>         sb.append(jsonString);
>         sb.append(", \"doc_as_upsert\": ");
>         sb.append(indexOp.equalsIgnoreCase("upsert"));
>         sb.append(" }\n");
>     } else if (indexOp.equalsIgnoreCase("delete")) {
>         sb.append("{\"delete\": { \"_index\": \"");
>         sb.append(StringEscapeUtils.escapeJson(index));
>         sb.append("\", \"_type\": \"");
>         sb.append(StringEscapeUtils.escapeJson(docType));
>         sb.append("\", \"_id\": \"");
>         sb.append(StringEscapeUtils.escapeJson(id));
>         sb.append("\" }\n");
>     }
> }
> {code}
>  
>  * The *TestPutElasticsearchHttp.java* test file needs to be updated to reflect that now a requests without type is valid (it's currently marked as invalid)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)