You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "samz-github (via GitHub)" <gi...@apache.org> on 2024/09/27 07:09:08 UTC

[I] [Feature] Provide option to support auto-generate doc id by ES for insert trace record when use elasticsearch storage [skywalking]

samz-github opened a new issue, #12657:
URL: https://github.com/apache/skywalking/issues/12657

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Description
   
   My corporation elasticsearch DBA feedback to me that skywalking use custom doc id butnot auto-generated by ES cluster, and it makes ES performace not good. 
   I have checked storage-eleasticsearch-plugin source code.  Skywalking OAP query ES with  the doc id  for UI template. For query trace record, not using doc id, but other ES field, like segment_id or trace_id.  Can we not to generate custom doc id when insert  trace recod in ES? Instead, it just let ES auto generate doc id when inserting trace record.  
   
   `
   
       @Override
       public InsertRequest prepareBatchInsert(Model model, Record record) throws IOException {
           final ElasticSearchConverter.ToStorage toStorage = new ElasticSearchConverter.ToStorage(model.getName());
           storageBuilder.entity2Storage(record, toStorage);
           Map<String, Object> builder = IndexController.INSTANCE.appendTableColumn(model, toStorage.obtain());
           String modelName = TimeSeriesUtils.writeIndexName(model, record.getTimeBucket());
           String id = IndexController.INSTANCE.generateDocId(model, record.id().build());
           Optional<String> routingValue = RoutingUtils.getRoutingValue(model, toStorage);
           return getClient().prepareInsert(modelName, id, routingValue, builder);
       }
   `
   
   ### Use case
   
   It helps to improve ES performace when  insert and search  ES trace records, especially for huge trace records everyday . 
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a pull request to implement this on your own?
   
   - [X] Yes I am willing to submit a pull request on my own!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature] Provide option to support auto-generate doc id by ES for insert trace record when use elasticsearch storage [skywalking]

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #12657: [Feature] Provide option to support auto-generate doc id by ES  for insert trace record when use elasticsearch storage
URL: https://github.com/apache/skywalking/issues/12657


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature] Provide option to support auto-generate doc id by ES for insert trace record when use elasticsearch storage [skywalking]

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #12657:
URL: https://github.com/apache/skywalking/issues/12657#issuecomment-2378558680

   This is intentional. We controlled the ID to make sure it would be updated by doc ID, rather than query condition. This improves the performance.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature] Provide option to support auto-generate doc id by ES for insert trace record when use elasticsearch storage [skywalking]

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #12657: [Feature] Provide option to support auto-generate doc id by ES  for insert trace record when use elasticsearch storage
URL: https://github.com/apache/skywalking/issues/12657


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org