You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "wForget (via GitHub)" <gi...@apache.org> on 2026/03/17 09:03:57 UTC
[PR] Fix DataWritingCommand callback naver be called on withListener method [kyuubi]
wForget opened a new pull request, #7362:
URL: https://github.com/apache/kyuubi/pull/7362
### Why are the changes needed?
The DataWritingCommand callback naver be called on withListener method
### How was this patch tested?
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Fix DataWritingCommand callback never be called on withListener method [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #7362:
URL: https://github.com/apache/kyuubi/pull/7362#issuecomment-4288937400
it's a small change, let's fix it on all spark versions in bulk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Fix DataWritingCommand callback naver be called on withListener method [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on PR #7362:
URL: https://github.com/apache/kyuubi/pull/7362#issuecomment-4073386100
Duplicate #6793
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Fix DataWritingCommand callback never be called on withListener method [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7362:
URL: https://github.com/apache/kyuubi/pull/7362#discussion_r3117780835
##########
extensions/spark/kyuubi-extension-spark-4-1/src/test/scala/org/apache/spark/sql/RebalanceBeforeWritingSuite.scala:
##########
@@ -109,7 +109,7 @@ class RebalanceBeforeWritingSuite extends KyuubiSparkSQLExtensionTest {
test("check rebalance does not exists") {
def check(df: DataFrame): Unit = {
- withListener(df) { write =>
+ withListener(df, false) { write =>
Review Comment:
use named arg
```suggestion
withListener(df, requireCallbackInvoked = false) { write =>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Fix DataWritingCommand callback naver be called on withListener method [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on code in PR #7362:
URL: https://github.com/apache/kyuubi/pull/7362#discussion_r2945327726
##########
extensions/spark/kyuubi-extension-spark-4-1/src/test/scala/org/apache/spark/sql/KyuubiSparkSQLExtensionTest.scala:
##########
@@ -100,13 +102,17 @@ trait KyuubiSparkSQLExtensionTest extends QueryTest
withListener(sql(sqlString))(callback)
}
- def withListener(df: => DataFrame)(callback: DataWritingCommand => Unit): Unit = {
+ def withListener(df: => DataFrame, mustBeCalled: Boolean = true)(
+ callback: DataWritingCommand => Unit): Unit = {
+ val called = new AtomicBoolean(false)
val listener = new QueryExecutionListener {
override def onFailure(f: String, qe: QueryExecution, e: Exception): Unit = {}
override def onSuccess(funcName: String, qe: QueryExecution, duration: Long): Unit = {
- qe.executedPlan match {
- case write: DataWritingCommandExec => callback(write.cmd)
+ collect(qe.executedPlan) {
Review Comment:
`DataWritingCommandExec` is wrapped by `AdaptiveSparkPlanExec`, so we should use `AdaptiveSparkPlanHelper#collect`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Fix DataWritingCommand callback never be called on withListener method [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #7362:
URL: https://github.com/apache/kyuubi/pull/7362#discussion_r3117776915
##########
extensions/spark/kyuubi-extension-spark-4-1/src/test/scala/org/apache/spark/sql/KyuubiSparkSQLExtensionTest.scala:
##########
@@ -100,13 +102,17 @@ trait KyuubiSparkSQLExtensionTest extends QueryTest
withListener(sql(sqlString))(callback)
}
- def withListener(df: => DataFrame)(callback: DataWritingCommand => Unit): Unit = {
+ def withListener(df: => DataFrame, mustBeCalled: Boolean = true)(
Review Comment:
`mustBeCalled` => `requireCallbackInvoked`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [PR] Fix DataWritingCommand callback never be called on withListener method [kyuubi]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on PR #7362:
URL: https://github.com/apache/kyuubi/pull/7362#issuecomment-4293075007
> it's a small change, let's fix it on all spark versions in bulk
Thanks, I will fix it in other spark versions as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org