You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "zml1206 (via GitHub)" <gi...@apache.org> on 2024/11/20 12:00:16 UTC
[PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
zml1206 opened a new pull request, #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011
## What changes were proposed in this pull request?
(Fixes: \#8010)
## How was this patch tested?
UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2490104585
> How about set metricsUpdater to `MetricsUpdater.None` when getRemainingCondition is null?
Sounds fair. Let's see if it works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2489675724
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2490121202
> BTW I think one of the promising ways is to exclude the filter node from Spark query plan as well. Though I remember there were some issues with an relevant attempt.
We can't exclude the filter node, because it's output may differ from child's output.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2490349741
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][CORE] Don't generate native metrics if transformer don't generate relNode [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1855870807
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/BasicPhysicalOperatorTransformer.scala:
##########
@@ -61,7 +61,9 @@ abstract class FilterExecTransformerBase(val cond: Expression, val input: SparkP
case _ => false
}
- override def metricsUpdater(): MetricsUpdater = if (getRemainingCondition == null) {
+ override def isLoop: Boolean = getRemainingCondition == null
Review Comment:
s/isLoop/isNoop/
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ExpandExecTransformer.scala:
##########
@@ -48,7 +48,9 @@ case class ExpandExecTransformer(
AttributeSet.fromAttributeSets(projections.flatten.map(_.references))
}
- override def metricsUpdater(): MetricsUpdater = if (projections == null || projections.isEmpty) {
+ override def isLoop: Boolean = projections == null || projections.isEmpty
Review Comment:
s/isLoop/isNoop/
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/SortExecTransformer.scala:
##########
@@ -44,7 +44,9 @@ case class SortExecTransformer(
@transient override lazy val metrics =
BackendsApiManager.getMetricsApiInstance.genSortTransformerMetrics(sparkContext)
- override def metricsUpdater(): MetricsUpdater = if (sortOrder == null || sortOrder.isEmpty) {
+ override def isLoop: Boolean = sortOrder == null || sortOrder.isEmpty
Review Comment:
s/isLoop/isNoop/
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/WindowExecTransformer.scala:
##########
@@ -51,9 +51,9 @@ case class WindowExecTransformer(
@transient override lazy val metrics =
BackendsApiManager.getMetricsApiInstance.genWindowTransformerMetrics(sparkContext)
- override def metricsUpdater(): MetricsUpdater = if (
- windowExpression == null || windowExpression.isEmpty
- ) {
+ override def isLoop: Boolean = windowExpression == null || windowExpression.isEmpty
Review Comment:
s/isLoop/isNoop/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1853279317
##########
backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala:
##########
@@ -219,6 +216,7 @@ object MetricsUtil extends Logging {
})
mutNode.updater match {
+ case MetricsUpdater.None =>
Review Comment:
I thought about it carefully later, and I should remove it, which can help us distinguish whether None is set incorrectly. If an UnsupportedOperationException is thrown later, it means that it should not be None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2489962244
Thanks!
Though it feels like a workaround since there is no real filter node in Velox query plan? Do we have other better options?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1853314527
##########
backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala:
##########
@@ -219,6 +216,7 @@ object MetricsUtil extends Logging {
})
mutNode.updater match {
+ case MetricsUpdater.None =>
Review Comment:
`MetricsUpdater.Todo` means that native metrics exist, but does not update them. `MetricsUpdater.None` means that native metrics do not exist, and updateNativeMetrics is not supported, so this is needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][CORE] Don't generate native metrics if transformer don't generate relNode [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2496902157
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2488400902
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1853234409
##########
backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala:
##########
@@ -219,6 +216,7 @@ object MetricsUtil extends Logging {
})
mutNode.updater match {
+ case MetricsUpdater.None =>
Review Comment:
Are we sure about this?
The code is likely something done by `MetricsUpdater.Todo`. I remember `MetricsUpdater.Todo` and `MetricsUpdater.None` don't share the same semantic.
Did you check the UI? Are all the metrics still normal with this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][CORE] Don't generate native metrics if transformer don't generate relNode [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2496916094
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2490718792
@zhztheplayer Can you help take a look again? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2490130446
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][CORE] Don't generate native metrics if transformer don't generate relNode [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1855745497
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ExpandExecTransformer.scala:
##########
@@ -112,13 +115,12 @@ case class ExpandExecTransformer(
override protected def doTransform(context: SubstraitContext): TransformContext = {
val childCtx = child.asInstanceOf[TransformSupport].transform(context)
- val operatorId = context.nextOperatorId(this.nodeName)
- if (projections == null || projections.isEmpty) {
+ if (metricsUpdater == MetricsUpdater.None) {
Review Comment:
Hi @zml1206
Can we create a new utility method, e.g, `isNoop` that can be called both by `doTransform` and `metricUpdater` in each operator? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1851622120
##########
backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala:
##########
@@ -54,9 +54,6 @@ object MetricsUtil extends Logging {
MetricsUpdaterTree(
smj.metricsUpdater(),
Seq(treeifyMetricsUpdaters(smj.bufferedPlan), treeifyMetricsUpdaters(smj.streamedPlan)))
- case t: TransformSupport if t.metricsUpdater() == MetricsUpdater.None =>
- assert(t.children.size == 1, "MetricsUpdater.None can only be used on unary operator")
- treeifyMetricsUpdaters(t.children.head)
Review Comment:
It will cause operatorId inconsistencies. It should be handled in `updateTransformerMetricsInternal`. cc @zhztheplayer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2488400482
https://github.com/apache/incubator-gluten/issues/8010
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2489990082
> Though it feels like a workaround since there is no real filter node in Velox query plan? Do we have other better options?
How about set metricsUpdater to `MetricsUpdater.None` when getRemainingCondition?
In this way, there is no metrics information.
<img width="294" alt="image" src="https://github.com/user-attachments/assets/55a6b61b-a38d-4e17-9453-b7473f50f3a7">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1853243739
##########
backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala:
##########
@@ -219,6 +216,7 @@ object MetricsUtil extends Logging {
})
mutNode.updater match {
+ case MetricsUpdater.None =>
Review Comment:
It doesn't seem to make any difference whether to add it or not.
Theoretically, the size of None's operatorMetrics is 0, and a judgment safety point is added.
Also, I didn't find any UI anomalies.
##########
backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala:
##########
@@ -219,6 +216,7 @@ object MetricsUtil extends Logging {
})
mutNode.updater match {
+ case MetricsUpdater.None =>
Review Comment:
I thought about it carefully later, and I should remove it, which can help us distinguish whether None is set incorrectly. If an UnsupportedOperationException is thrown later, it means that it should not be None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2493225636
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Inherit child metrics when FilterExecTransformer's remainingCondition is null [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#issuecomment-2488639694
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1853243739
##########
backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala:
##########
@@ -219,6 +216,7 @@ object MetricsUtil extends Logging {
})
mutNode.updater match {
+ case MetricsUpdater.None =>
Review Comment:
It doesn't seem to make any difference whether to add it or not.
Theoretically, the size of None's operatorMetrics is 0, and a judgment safety point is added.
Also, I didn't find any UI anomalies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1853408564
##########
backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala:
##########
@@ -219,6 +216,7 @@ object MetricsUtil extends Logging {
})
mutNode.updater match {
+ case MetricsUpdater.None =>
Review Comment:
I see. My impressions about this part of code is:
* When `registerEmptyRelToOperator` is called, a fake operator creation is actually emulated so `MetricsUpdator.Todo` should be considered
* When `child.asInstanceOf[TransformSupport].transform(context)` is used, `MetricsUpdator.None` should be considered
Would you like to check whether the two rule apply?
And if we no longer need `MetricsUpdater.None`, we can just remove this type of metrics updater.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][CORE] Don't generate native metrics if transformer don't generate relNode [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 merged PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null [incubator-gluten]
Posted by "zml1206 (via GitHub)" <gi...@apache.org>.
zml1206 commented on code in PR #8011:
URL: https://github.com/apache/incubator-gluten/pull/8011#discussion_r1851622120
##########
backends-velox/src/main/scala/org/apache/gluten/metrics/MetricsUtil.scala:
##########
@@ -54,9 +54,6 @@ object MetricsUtil extends Logging {
MetricsUpdaterTree(
smj.metricsUpdater(),
Seq(treeifyMetricsUpdaters(smj.bufferedPlan), treeifyMetricsUpdaters(smj.streamedPlan)))
- case t: TransformSupport if t.metricsUpdater() == MetricsUpdater.None =>
- assert(t.children.size == 1, "MetricsUpdater.None can only be used on unary operator")
- treeifyMetricsUpdaters(t.children.head)
Review Comment:
It will cause operatorId inconsistencies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org