You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "panbingkun (via GitHub)" <gi...@apache.org> on 2024/04/08 08:35:40 UTC
[PR] [SPARK-47587][SQL] Hive module: Migrate logWarn with variables to structured logging framework [spark]
panbingkun opened a new pull request, #45927:
URL: https://github.com/apache/spark/pull/45927
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GA.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [WIP][SPARK-47587][SQL] Hive module: Migrate logWarn with variables to structured logging framework [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #45927:
URL: https://github.com/apache/spark/pull/45927#discussion_r1560342791
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala:
##########
@@ -104,21 +105,28 @@ private[hive] class HiveMetastoreCatalog(sparkSession: SparkSession) extends Log
None
}
case _ =>
- logWarning(s"Table $tableIdentifier should be stored as $expectedFileFormat. " +
- s"However, we are getting a ${relation.fileFormat} from the metastore cache. " +
- "This cached entry will be invalidated.")
+ logWarningUnexpectedFileFormat(tableIdentifier, expectedFileFormat,
+ relation.fileFormat.toString)
catalogProxy.invalidateCachedTable(tableIdentifier)
None
}
case other =>
- logWarning(s"Table $tableIdentifier should be stored as $expectedFileFormat. " +
- s"However, we are getting a $other from the metastore cache. " +
- "This cached entry will be invalidated.")
+ logWarningUnexpectedFileFormat(tableIdentifier, expectedFileFormat, other.toString)
catalogProxy.invalidateCachedTable(tableIdentifier)
None
}
}
+ private def logWarningUnexpectedFileFormat(
Review Comment:
Extract a `common` method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47587][SQL] Hive module: Migrate logWarn with variables to structured logging framework [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on PR #45927:
URL: https://github.com/apache/spark/pull/45927#issuecomment-2048804834
cc @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47587][SQL] Hive module: Migrate logWarn with variables to structured logging framework [spark]
Posted by "gengliangwang (via GitHub)" <gi...@apache.org>.
gengliangwang commented on PR #45927:
URL: https://github.com/apache/spark/pull/45927#issuecomment-2048956649
Thanks, merging to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [WIP][SPARK-47587][SQL] Hive module: Migrate logWarn with variables to structured logging framework [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #45927:
URL: https://github.com/apache/spark/pull/45927#discussion_r1560341095
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -229,8 +230,8 @@ private[hive] class HiveClientImpl(
case e: Exception if causedByThrift(e) =>
caughtException = e
logWarning(
- "HiveClient got thrift exception, destroying client and retrying " +
- s"(${retryLimit - numTries} tries remaining)", e)
+ log"HiveClient got thrift exception, destroying client and retrying " +
+ log"${MDC(RETRY_COUNT, numTries)} times", e)
Review Comment:
Although there is a `slight difference` in the meaning expressed in these two places, `one` is how many times are `left` and the `other` is how many times have been `retried`. In order to be consistent with `other parts of Spark`, adjustments have been made here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47587][SQL] Hive module: Migrate logWarn with variables to structured logging framework [spark]
Posted by "gengliangwang (via GitHub)" <gi...@apache.org>.
gengliangwang closed pull request #45927: [SPARK-47587][SQL] Hive module: Migrate logWarn with variables to structured logging framework
URL: https://github.com/apache/spark/pull/45927
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org