You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2022/01/13 09:25:18 UTC
[spark] branch master updated: [SPARK-37889][SQL] Replace Log4j2 MarkerFilter with RegexFilter
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8ed6144 [SPARK-37889][SQL] Replace Log4j2 MarkerFilter with RegexFilter
8ed6144 is described below
commit 8ed6144a6a599d913358206e84b726573d1f2f86
Author: Kent Yao <ya...@apache.org>
AuthorDate: Thu Jan 13 18:24:19 2022 +0900
[SPARK-37889][SQL] Replace Log4j2 MarkerFilter with RegexFilter
### What changes were proposed in this pull request?
Log4j2 MarkerFilter can not suppress
```
10:55:54.597 ERROR org.apache.thrift.server.TThreadPoolServer: Thrift error occurred during processing of message.
org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) ~[libthrift-0.12.0.jar:0.12.0]
```
We shall use RegexFilter instead
### Why are the changes needed?
bugfix
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
run `build/sbt -Phive -Phive-thriftserver "hive-thriftserver/testOnly *ThriftServerWithSparkContextInBinarySuite"` locally and checked
Closes #35186 from yaooqinn/SPARK-37889.
Authored-by: Kent Yao <ya...@apache.org>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
conf/log4j2.properties.template | 4 ++--
sql/hive-thriftserver/src/test/resources/log4j2.properties | 8 ++++----
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/conf/log4j2.properties.template b/conf/log4j2.properties.template
index 85b4f67..99f68a8 100644
--- a/conf/log4j2.properties.template
+++ b/conf/log4j2.properties.template
@@ -57,7 +57,7 @@ logger.FunctionRegistry.level = error
# For deploying Spark ThriftServer
# SPARK-34128: Suppress undesirable TTransportException warnings involved in THRIFT-4805
-appender.console.filter.1.type = MarkerFilter
-appender.console.filter.1.marker = Thrift error occurred during processing of message
+appender.console.filter.1.type = RegexFilter
+appender.console.filter.1.regex = .*Thrift error occurred during processing of message.*
appender.console.filter.1.onMatch = deny
appender.console.filter.1.onMismatch = neutral
diff --git a/sql/hive-thriftserver/src/test/resources/log4j2.properties b/sql/hive-thriftserver/src/test/resources/log4j2.properties
index 58e18af..939335b 100644
--- a/sql/hive-thriftserver/src/test/resources/log4j2.properties
+++ b/sql/hive-thriftserver/src/test/resources/log4j2.properties
@@ -33,8 +33,8 @@ appender.console.filter.1.a.type = ThresholdFilter
appender.console.filter.1.a.level = warn
# SPARK-34128: Suppress undesirable TTransportException warnings, due to THRIFT-4805
-appender.console.filter.1.b.type = MarkerFilter
-appender.console.filter.1.b.marker = Thrift error occurred during processing of message
+appender.console.filter.1.b.type = RegexFilter
+appender.console.filter.1.b.regex = .*Thrift error occurred during processing of message.*
appender.console.filter.1.b.onMatch = deny
appender.console.filter.1.b.onMismatch = neutral
@@ -47,8 +47,8 @@ appender.file.layout.pattern = %d{HH:mm:ss.SSS} %t %p %c{1}: %m%n
appender.file.filter.1.type = Filters
-appender.file.filter.1.a.type = MarkerFilter
-appender.file.filter.1.a.marker = Thrift error occurred during processing of message
+appender.file.filter.1.a.type = RegexFilter
+appender.file.filter.1.a.regx = .*Thrift error occurred during processing of message.*
appender.file.filter.1.a.onMatch = deny
appender.file.filter.1.a.onMismatch = neutral
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org