You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2023/10/02 13:28:03 UTC
[spark] branch master updated: [SPARK-45377][CORE] Handle InputStream in NettyLogger
This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new cdbb301143d [SPARK-45377][CORE] Handle InputStream in NettyLogger
cdbb301143d is described below
commit cdbb301143de2e9a0ea525d20867948f49863842
Author: Hasnain Lakhani <ha...@databricks.com>
AuthorDate: Mon Oct 2 08:27:50 2023 -0500
[SPARK-45377][CORE] Handle InputStream in NettyLogger
### What changes were proposed in this pull request?
Handle `InputStream`s in the `NettyLogger` so we can print out how many available bytes there are.
### Why are the changes needed?
As part of the SSL support we are going to transfer `InputStream`s via Netty, and this functionality makes it easy to see the size of the streams in the log at a glance.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
CI. Tested as part of the changes in https://github.com/apache/spark/pull/42685 which this is split out of, I observed the logs there.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #43165 from hasnain-db/spark-tls-netty-logger.
Authored-by: Hasnain Lakhani <ha...@databricks.com>
Signed-off-by: Sean Owen <sr...@gmail.com>
---
.../main/java/org/apache/spark/network/util/NettyLogger.java | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java b/common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java
index 9398726a926..f4c0df6239d 100644
--- a/common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java
+++ b/common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java
@@ -17,6 +17,9 @@
package org.apache.spark.network.util;
+import java.io.IOException;
+import java.io.InputStream;
+
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufHolder;
import io.netty.channel.ChannelHandlerContext;
@@ -42,6 +45,14 @@ public class NettyLogger {
} else if (arg instanceof ByteBufHolder) {
return format(ctx, eventName) + " " +
((ByteBufHolder) arg).content().readableBytes() + "B";
+ } else if (arg instanceof InputStream) {
+ int available = -1;
+ try {
+ available = ((InputStream) arg).available();
+ } catch (IOException ex) {
+ // Swallow, but return -1 to indicate an error happened
+ }
+ return format(ctx, eventName, arg) + " " + available + "B";
} else {
return super.format(ctx, eventName, arg);
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org