You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "chia7712 (via GitHub)" <gi...@apache.org> on 2024/03/20 11:04:33 UTC

Re: [PR] KAFKA-15807: Added support for compression of metrics (KIP-714) [kafka]

chia7712 commented on code in PR #15148:
URL: https://github.com/apache/kafka/pull/15148#discussion_r1531889679


##########
clients/src/main/java/org/apache/kafka/common/telemetry/internals/ClientTelemetryUtils.java:
##########
@@ -175,16 +182,44 @@ public static boolean validateRequiredResourceLabels(Map<String, String> metadat
     }
 
     public static CompressionType preferredCompressionType(List<CompressionType> acceptedCompressionTypes) {
-        // TODO: Support compression in client telemetry.
+        if (acceptedCompressionTypes != null && !acceptedCompressionTypes.isEmpty()) {
+            // Broker is providing the compression types in order of preference. Grab the
+            // first one.
+            return acceptedCompressionTypes.get(0);
+        }
         return CompressionType.NONE;
     }
 
     public static ByteBuffer compress(byte[] raw, CompressionType compressionType) {
-        // TODO: Support compression in client telemetry.
-        if (compressionType == CompressionType.NONE) {
-            return ByteBuffer.wrap(raw);
-        } else {
-            throw new UnsupportedOperationException("Compression is not supported");
+        try {
+            try (ByteBufferOutputStream compressedOut = new ByteBufferOutputStream(512)) {
+                try (OutputStream out = compressionType.wrapForOutput(compressedOut, RecordBatch.CURRENT_MAGIC_VALUE)) {
+                    out.write(raw);
+                    out.flush();
+                }
+                compressedOut.buffer().flip();
+                return ByteBuffer.wrap(Utils.toArray(compressedOut.buffer()));
+            }
+        } catch (IOException e) {
+            throw new KafkaException("Failed to compress metrics data", e);
+        }
+    }
+
+    public static ByteBuffer decompress(byte[] metrics, CompressionType compressionType) {
+        ByteBuffer data = ByteBuffer.wrap(metrics);
+        try (InputStream in = compressionType.wrapForInput(data, RecordBatch.CURRENT_MAGIC_VALUE, BufferSupplier.create());
+            ByteArrayOutputStream out = new ByteArrayOutputStream()) {
+
+            byte[] bytes = new byte[data.capacity() * 2];
+            int nRead;
+            while ((nRead = in.read(bytes, 0, bytes.length)) != -1) {
+                out.write(bytes, 0, nRead);
+            }
+
+            out.flush();
+            return ByteBuffer.wrap(out.toByteArray());

Review Comment:
   hi @apoorvmittal10 I have a question: Is it worth using `ByteBufferOutputStream` to replace `ByteArrayOutputStream`? We can avoid the array copy by taking buffer from `ByteBufferOutputStream` directly.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org