[I] Confirming a logic change on an operator that cannot be reconfigured turns the paused run into Failed [texera]

["eugenegujing (via GitHub)" <gi...@apache.org>]

eugenegujing opened a new issue, #8967:
URL: https://github.com/apache/texera/issues/8967

   ### What happened?
   
   Three operators advertise runtime reconfiguration without implementing it: `PythonLambdaFunctionOpDesc.scala:74`, `HuggingFaceSentimentAnalysisOpDesc.scala:164` and `PythonUDFSourceOpDescV2.scala:126` set `supportReconfiguration = true` in their `OperatorInfo` but do not override `runtimeReconfiguration`. (`DummyOpDesc.scala:70` does the same, but it is a placeholder that never runs.)
   
   The "Unlock for Logic Change" button is enabled for any paused operator with that flag (`operator-property-edit-frame.component.html:203-222`). Confirming the edit sends a `ModifyLogicRequest`, and `ExecutionReconfigurationService.modifyOperatorLogic` calls the operator's `runtimeReconfiguration` (`ExecutionReconfigurationService.scala:75-91`):
   
   ```scala
         val reconfiguredPhysicalOp =
           currentOp.runtimeReconfiguration(
             workflow.context.workflowId,
             workflow.context.executionId,
             currentOp,
             newOp
           )
         reconfiguredPhysicalOp match {
           case Failure(exception) =>
             ModifyLogicResponse(opId.id, isValid = false, exception.getMessage)
           case Success(op) => {
   ```
   
   The `Failure` arm is meant to tell the user the edit was rejected. It never runs: the base `LogicalOp.runtimeReconfiguration` throws instead of returning `Failure` (`LogicalOp.scala:488-497`), and every override returns `Success` unconditionally.
   
   ```scala
     ): Try[(PhysicalOp, Option[StateTransferFunc])] = {
       throw new UnsupportedOperationException(
         "operator " + getClass.getSimpleName + " does not support reconfiguration"
       )
     }
   ```
   
   So for the three operators above the exception escapes `modifyOperatorLogic`, reaches the websocket catch-all (`WorkflowWebsocketResource.scala:138-162`), and is recorded as a `COMPILATION_ERROR` fatal error attributed to `"unknown operator"`. The metadata diff handler emits a `WorkflowErrorEvent`, which the frontend maps to `ExecutionState.Failed` (`execute-workflow.service.ts:195-201`).
   
   What the user sees: the run they had paused is shown as **Failed**. The run button changes from "Resume" to "Run", the kill button is disabled, and the operators keep their paused colouring. Because the error is attributed to "unknown operator", the result panel shows nothing while an operator is selected; only with nothing selected does the "Static Error" tab list a `COMPILATION_ERROR` reading `operator PythonLambdaFunctionOpDesc does not support reconfiguration` with a Java stack trace. The engine, however, is still paused with its workers alive: nothing told it to stop, and its `workflow_executions` row stays at PAUSED (status 2). The only way forward in the UI is "Run", which shuts the paused execution down, clears its results, and starts over.
   
   A mis-flagged operator is the easiest way in, but not the only one: an exception thrown inside an override (for example from `getPhysicalOp`) takes the same route. `FilterOpDescSpec:103-115` and `MapOpDescSpec:107-119` pin that such an exception propagates rather than becoming a `Failure`.
   
   **Expected:** a logic change the operator cannot take is reported as a rejected edit (`ModifyLogicResponse` with `isValid = false`) and the run stays paused.
   
   ### How to reproduce?
   
   1. Build Text Input -> Python UDF -> Python Lambda Function (Add New Column `tag`, string, expression `"a"`). Give the UDF a `process_tuple` that yields the input tuple once per second for a few minutes, so that a pause is acknowledged promptly:
   
      ```python
      for i in range(900):
          time.sleep(1)
          yield tuple_
      ```
   
   2. Run the workflow, then pause it. The button reads "Resume".
   <img width="1263" height="799" alt="Image" src="https://github.com/user-attachments/assets/27493ca3-25be-457c-98ed-672c3683b861" />
   
   3. Select the Python Lambda Function. "Unlock for Logic Change" is enabled; click it, change the expression to `"b"`, click "Confirm Change".
   4. The button flips to "Run" and the kill button is disabled: the frontend is in Failed. Click on empty canvas to deselect: the result panel's "Static Error" tab shows `COMPILATION_ERROR: operator PythonLambdaFunctionOpDesc does not support reconfiguration`.
   
   <img width="1263" height="799" alt="Image" src="https://github.com/user-attachments/assets/62ca2a79-e78f-4b1c-a4eb-5c8293731d74" />
   
   <img width="1263" height="799" alt="Image" src="https://github.com/user-attachments/assets/a319a4ad-1df7-46c2-8e0f-4c2d143d1d58" />
   
   5. The execution's `workflow_executions` row is still status 2 (PAUSED), and `computing-unit-master.log` logs the exception followed by `workflow status = PAUSED`.
   
   The same happens with Hugging Face Sentiment Analysis and Python UDF Source.
   
   ### Version/Branch
   
   1.4.0-incubating-SNAPSHOT (main)
   
   ### Commit Hash (Optional)
   
   _No response_
   
   ### What browsers are you seeing the problem on?
   
   _No response_
   
   ### Relevant log output
   
   ```shell
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org