[GitHub] [pulsar] longtengz opened a new issue #13173: Elasticsearch sink `keyIgnore` config support non-keyvalue schemas (i.e. avro and json)

[GitBox <gi...@apache.org>]

longtengz opened a new issue #13173:
URL: https://github.com/apache/pulsar/issues/13173


   **Is your enhancement request related to a problem? Please describe.**
   
   Currently, `keyIgnore` config only works when keyvalue schemas are used for a topic, because of this line below
   
   https://github.com/apache/pulsar/blob/ede95e9ba084c03822e40824c0ea41878f5fc102/pulsar-io/elastic-search/src/main/java/org/apache/pulsar/io/elasticsearch/ElasticSearchSink.java#L169
   
   **Describe the solution you'd like**
   
   I'd like avro and json schemas to work as well when `keyIgnore = false`, since to my understanding, currently the golang and python client does not support keyvalue schema. So I can manually set the key as a primitive string, and still use this `keyIgnore` config to have `_id` in ES set as the message key.
   
   I think the solution here is quite simple if I understand the code correctly here. 
   ```
   if keySchema != null {
       // use key schema to encode the key
   } else {
       // use the key directly
   }
   ``
   
   **Describe alternatives you've considered**
   A clear and concise description of any alternative solutions or features you've considered.
   
   **Additional context**
   Add any other context or screenshots about the feature request here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #13173: Elasticsearch sink `keyIgnore` config support non-keyvalue schemas (i.e. avro and json)

[GitBox <gi...@apache.org>]

github-actions[bot] commented on issue #13173:
URL: https://github.com/apache/pulsar/issues/13173#issuecomment-1051439792


   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org