You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@wayang.apache.org by "mohit-devlogs (via GitHub)" <gi...@apache.org> on 2026/03/07 02:22:00 UTC
[PR] Improve diagnostic messages in JdbcExecutor when resolving execution tasks [wayang]
mohit-devlogs opened a new pull request, #714:
URL: https://github.com/apache/wayang/pull/714
The current implementation of the method `findJdbcExecutionOperatorTaskInStage` relies on the use of assertions and `WayangCollections.getSingle` for the acquisition of the consumer of the output channel.
When the assumptions are not met, the error messages given, such as "[] is not a singleton", are not very clear and do not assist very much for debugging purposes.
In the current code, the assumptions have been replaced with validations and `WayangException`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve diagnostic messages in JdbcExecutor when resolving execution tasks [wayang]
Posted by "mohit-devlogs (via GitHub)" <gi...@apache.org>.
mohit-devlogs commented on code in PR #714:
URL: https://github.com/apache/wayang/pull/714#discussion_r3087063639
##########
wayang-platforms/wayang-jdbc-template/src/main/java/org/apache/wayang/jdbc/execution/JdbcExecutor.java:
##########
@@ -158,69 +183,84 @@ protected static Tuple2<String, SqlQueryChannel.Instance> createSqlQuery(final E
final OptimizationContext context, final JdbcExecutor jdbcExecutor) {
final Collection<?> startTasks = stage.getStartTasks();
final Collection<?> termTasks = stage.getTerminalTasks();
-
// Verify that we can handle this instance.
assert startTasks.size() == 1 : "Invalid jdbc stage: multiple sources are not currently supported";
final ExecutionTask startTask = (ExecutionTask) startTasks.toArray()[0];
assert termTasks.size() == 1 : "Invalid JDBC stage: multiple terminal tasks are not currently supported.";
final ExecutionTask termTask = (ExecutionTask) termTasks.toArray()[0];
assert startTask.getOperator() instanceof TableSource
: "Invalid JDBC stage: Start task has to be a TableSource";
-
// Extract the different types of ExecutionOperators from the stage.
final JdbcTableSource tableOp = (JdbcTableSource) startTask.getOperator();
- SqlQueryChannel.Instance tipChannelInstance = JdbcExecutor.instantiateOutboundChannel(startTask, context, jdbcExecutor);
+ SqlQueryChannel.Instance tipChannelInstance =
+ JdbcExecutor.instantiateOutboundChannel(startTask, context, jdbcExecutor);
+
final Collection<JdbcFilterOperator> filterTasks = new ArrayList<>(4);
JdbcProjectionOperator projectionTask = null;
final Collection<JdbcJoinOperator<?>> joinTasks = new ArrayList<>();
+
final Set<ExecutionTask> allTasks = stage.getAllTasks();
assert allTasks.size() <= 3;
+
ExecutionTask nextTask = JdbcExecutor.findJdbcExecutionOperatorTaskInStage(startTask, stage);
+
while (nextTask != null) {
// Evaluate the nextTask.
if (nextTask.getOperator() instanceof final JdbcFilterOperator filterOperator) {
filterTasks.add(filterOperator);
} else if (nextTask.getOperator() instanceof JdbcProjectionOperator projectionOperator) {
- assert projectionTask == null; // Allow one projection operator per stage for now.
+ assert projectionTask == null;
Review Comment:
Thanks for pointing this out. I have restored the missing comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve diagnostic messages in JdbcExecutor when resolving execution tasks [wayang]
Posted by "zkaoudi (via GitHub)" <gi...@apache.org>.
zkaoudi commented on code in PR #714:
URL: https://github.com/apache/wayang/pull/714#discussion_r3086446877
##########
wayang-platforms/wayang-jdbc-template/src/main/java/org/apache/wayang/jdbc/execution/JdbcExecutor.java:
##########
@@ -158,69 +183,84 @@ protected static Tuple2<String, SqlQueryChannel.Instance> createSqlQuery(final E
final OptimizationContext context, final JdbcExecutor jdbcExecutor) {
final Collection<?> startTasks = stage.getStartTasks();
final Collection<?> termTasks = stage.getTerminalTasks();
-
// Verify that we can handle this instance.
assert startTasks.size() == 1 : "Invalid jdbc stage: multiple sources are not currently supported";
final ExecutionTask startTask = (ExecutionTask) startTasks.toArray()[0];
assert termTasks.size() == 1 : "Invalid JDBC stage: multiple terminal tasks are not currently supported.";
final ExecutionTask termTask = (ExecutionTask) termTasks.toArray()[0];
assert startTask.getOperator() instanceof TableSource
: "Invalid JDBC stage: Start task has to be a TableSource";
-
// Extract the different types of ExecutionOperators from the stage.
final JdbcTableSource tableOp = (JdbcTableSource) startTask.getOperator();
- SqlQueryChannel.Instance tipChannelInstance = JdbcExecutor.instantiateOutboundChannel(startTask, context, jdbcExecutor);
+ SqlQueryChannel.Instance tipChannelInstance =
+ JdbcExecutor.instantiateOutboundChannel(startTask, context, jdbcExecutor);
+
final Collection<JdbcFilterOperator> filterTasks = new ArrayList<>(4);
JdbcProjectionOperator projectionTask = null;
final Collection<JdbcJoinOperator<?>> joinTasks = new ArrayList<>();
+
final Set<ExecutionTask> allTasks = stage.getAllTasks();
assert allTasks.size() <= 3;
+
ExecutionTask nextTask = JdbcExecutor.findJdbcExecutionOperatorTaskInStage(startTask, stage);
+
while (nextTask != null) {
// Evaluate the nextTask.
if (nextTask.getOperator() instanceof final JdbcFilterOperator filterOperator) {
filterTasks.add(filterOperator);
} else if (nextTask.getOperator() instanceof JdbcProjectionOperator projectionOperator) {
- assert projectionTask == null; // Allow one projection operator per stage for now.
+ assert projectionTask == null;
Review Comment:
put back the comment please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve diagnostic messages in JdbcExecutor when resolving execution tasks [wayang]
Posted by "zkaoudi (via GitHub)" <gi...@apache.org>.
zkaoudi commented on PR #714:
URL: https://github.com/apache/wayang/pull/714#issuecomment-4017052201
Hi @mohit-devlogs we will take a look at your PR but until then I am wondering why you have removed the comments the code had.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve diagnostic messages in JdbcExecutor when resolving execution tasks [wayang]
Posted by "mohit-devlogs (via GitHub)" <gi...@apache.org>.
mohit-devlogs commented on PR #714:
URL: https://github.com/apache/wayang/pull/714#issuecomment-4018071672
> Hi @mohit-devlogs we will take a look at your PR but until then I am wondering why you have removed the comments the code had.
Thank you for pointing this out. As for the comments, these were not supposed to be removed. When I modified the code to enhance the diagnostic information, it appears that I inadvertently modified the code block, resulting in the removal of the comments.
I will restore the original comments and update the PR so that it only contains the intended diagnostic improvements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org