[GitHub] [skywalking] yaojingguo commented on issue #3778: Use the JavaAgent,Conflict with KafkaClient? (Magic v1 does not support record headers)

[GitBox <gi...@apache.org>]

yaojingguo commented on issue #3778:
URL: https://github.com/apache/skywalking/issues/3778#issuecomment-782833577


   A solution is to patch kafka-clients. Just commenting out the following code of [MemoryRecordsBuilder.java](https://github.com/apache/kafka/blob/2.3.1/clients/src/main/java/org/apache/kafka/common/record/MemoryRecordsBuilder.java). I use kafka-clients 2.3.1.
   
   ```
               if (magic < RecordBatch.MAGIC_VALUE_V2 && headers != null && headers.length > 0)
                   throw new IllegalArgumentException("Magic v" + magic + " does not support record headers");
   ```
   
   
   My first thought is to patch [KafkaProducerInterceptor](https://github.com/apache/skywalking/blob/v8.4.0/apm-sniffer/apm-sdk-plugin/kafka-plugin/src/main/java/org/apache/skywalking/apm/plugin/kafka/KafkaProducerInterceptor.java). [KafkaProducer](https://github.com/apache/kafka/blob/2.3.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java) has a private field  [apiVersions](https://kafka.apache.org/protocol#api_versions) which is updated to reflect Kafka server version before `KafkaProducer` sends its first message to Kafka server. However, `KafkaProducer` uses a separate thread to actually talk to a Kafka node. We don't know when `apiVersions` reflects the real version of a Kafka node. So this approach does not work out.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org