You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Lee-W (via GitHub)" <gi...@apache.org> on 2025/10/30 11:23:10 UTC

[PR] fix: HITL params not validating [airflow]

Lee-W opened a new pull request, #57547:
URL: https://github.com/apache/airflow/pull/57547

   <!--
    Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at
   
      http://www.apache.org/licenses/LICENSE-2.0
   
    Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.
    -->
   
   <!--
   Thank you for contributing! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   In case of an existing issue, reference it using one of the following:
   
   closes: #ISSUE
   related: #ISSUE
   
   How to write a good git commit message:
   http://chris.beams.io/posts/git-commit/
   -->
   
   
   
   <!-- Please keep an empty line above the dashes. -->
   ---
   **^ Add meaningful description above**
   Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2501416702


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;
 
     Object.entries(sourceParams ?? {}).forEach(([key, value]) => {
-      const valueType = typeof value === "number" ? "number" : "string";
+      const paramData = hitlDetail.params?.[key] ?? {};
 
       paramsDict[key] = {
-        description: "",
-        schema: {
+        description: paramData?.description ?? "",
+        schema: paramData?.schema ?? {

Review Comment:
   For backward compatibility, we might not be able to do that. Not sure whether there's smart way to do it in the frontend 🤔 
   
   For older provider, we still want to make `{"key": "value"}` work fine



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2501409548


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -125,10 +125,10 @@ export const getHITLParamsDict = (
           minLength: undefined,
           section: undefined,
           title: key,
-          type: valueType,
+          type: typeof value === "number" ? "number" : "string",
           values_display: undefined,
         },
-        value: preloadedHITLParams[key] ?? value,
+        value: paramData?.value ?? paramData ?? value,

Review Comment:
   updated. Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2501624273


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;
 
     Object.entries(sourceParams ?? {}).forEach(([key, value]) => {
-      const valueType = typeof value === "number" ? "number" : "string";
+      const paramData = hitlDetail.params?.[key] ?? {};
 
       paramsDict[key] = {
-        description: "",
-        schema: {
+        description: paramData?.description ?? "",
+        schema: paramData?.schema ?? {

Review Comment:
   The reasoning makes sense to me, but I’d appreciate hearing other opinions. Maybe we could also leave a comment to mention this?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2500218389


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;
 
     Object.entries(sourceParams ?? {}).forEach(([key, value]) => {
-      const valueType = typeof value === "number" ? "number" : "string";
+      const paramData = hitlDetail.params?.[key] ?? {};
 
       paramsDict[key] = {
-        description: "",
-        schema: {
+        description: paramData?.description ?? "",
+        schema: paramData?.schema ?? {

Review Comment:
   The type of `paramData?.schema` is still `[key: string]: unknown`. I think it needs some modification to reflect its exact type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2501414118


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;

Review Comment:
   I'm not sure I understand 🤔 AFAIK, the reason why we need to handle `{"key": "value"}` instead of `{"key": {"value": "value", "schema": {}, "description": null}}` is for backward compatibility. Whehter received or not should stay the same 👀



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #57547:
URL: https://github.com/apache/airflow/pull/57547#issuecomment-3468274496

   Sure, I’ll take a look at it and let you know if I make any progress.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #57547:
URL: https://github.com/apache/airflow/pull/57547#issuecomment-3479666656

   > Sure, I’ll take a look at it and will let you know if I make any progress.
   
   Thanks @guan404ming ! I took extra effort taking a look but still failed to figure our the frontend part


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2508969289


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;
 
     Object.entries(sourceParams ?? {}).forEach(([key, value]) => {
-      const valueType = typeof value === "number" ? "number" : "string";
+      const paramData = hitlDetail.params?.[key] ?? {};
 
       paramsDict[key] = {
-        description: "",
-        schema: {
+        description: paramData?.description ?? "",
+        schema: paramData?.schema ?? {

Review Comment:
   Sure!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2493285446


##########
providers/standard/src/airflow/providers/standard/operators/hitl.py:
##########
@@ -29,7 +29,11 @@
 from urllib.parse import ParseResult, urlencode, urlparse, urlunparse
 
 from airflow.configuration import conf
-from airflow.providers.standard.exceptions import HITLRejectException, HITLTimeoutError, HITLTriggerEventError
+from airflow.providers.standard.exceptions import (
+    HITLRejectException,
+    HITLTimeoutError,
+    HITLTriggerEventError,
+)

Review Comment:
   Can we undo formatting changes please to keep the diff limited to changes?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2500126726


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -125,10 +125,10 @@ export const getHITLParamsDict = (
           minLength: undefined,
           section: undefined,
           title: key,
-          type: valueType,
+          type: typeof value === "number" ? "number" : "string",
           values_display: undefined,
         },
-        value: preloadedHITLParams[key] ?? value,
+        value: paramData?.value ?? paramData ?? value,

Review Comment:
   ```suggestion
           value: paramData?.value ?? value,
   ```
   
   I think we could remove the paramData here



##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;

Review Comment:
   Maybe we could split `received` and `not recieved`  into two parts and handle them separately, since their types are different now, which makes it painful to code them together.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #57547:
URL: https://github.com/apache/airflow/pull/57547#issuecomment-3467581874

   Hey @guan404ming @pierrejeambrun , I think we also need to block it from the frontend side. I tried to dig into it but failed to figure it out today. I guess it has something related to https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/ui/src/components/FlexibleForm/FlexibleForm.tsx. It would be nice if you could offer your help or give me a few directions on what I might be missing. Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2503862906


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;
 
     Object.entries(sourceParams ?? {}).forEach(([key, value]) => {
-      const valueType = typeof value === "number" ? "number" : "string";
+      const paramData = hitlDetail.params?.[key] ?? {};
 
       paramsDict[key] = {
-        description: "",
-        schema: {
+        description: paramData?.description ?? "",
+        schema: paramData?.schema ?? {

Review Comment:
   Then I'll hold and not merge for the time being 👀 Would love to hear @pierrejeambrun 's thought on this one. Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2493711838


##########
providers/standard/src/airflow/providers/standard/operators/hitl.py:
##########
@@ -29,7 +29,11 @@
 from urllib.parse import ParseResult, urlencode, urlparse, urlunparse
 
 from airflow.configuration import conf
-from airflow.providers.standard.exceptions import HITLRejectException, HITLTimeoutError, HITLTriggerEventError
+from airflow.providers.standard.exceptions import (
+    HITLRejectException,
+    HITLTimeoutError,
+    HITLTriggerEventError,
+)

Review Comment:
   The whole PR will be reworked as confirmed with @guan404ming for the frontend's need



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2508964067


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;
 
     Object.entries(sourceParams ?? {}).forEach(([key, value]) => {
-      const valueType = typeof value === "number" ? "number" : "string";
+      const paramData = hitlDetail.params?.[key] ?? {};
 
       paramsDict[key] = {
-        description: "",
-        schema: {
+        description: paramData?.description ?? "",
+        schema: paramData?.schema ?? {

Review Comment:
   as we're to have another core release soon, I'll just merge it and let's polish later



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2501621875


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;

Review Comment:
   Thanks for clarifying! After some offline discussion, it makes sense now. Keeping it this way for backward compatibility sounds reasonable.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57547:
URL: https://github.com/apache/airflow/pull/57547#issuecomment-3511484562

   ### Backport failed to create: v3-1-test. View the failure log <a href='https://github.com/apache/airflow/actions/runs/19232381455'> Run details </a>
   
   <table>
       <tr>
           <th>Status</th>
           <th>Branch</th>
           <th>Result</th>
       </tr>
       <tr>
           <td>❌</td>
           <td>v3-1-test</td>
           <td><a href="https://github.com/apache/airflow/commit/0db5838f7ec5b9d65fb63a8aa1d7337cf076776f"><img src='https://img.shields.io/badge/Commit-0db5838-red' alt='Commit Link'></a></td>
       </tr>
   </table>
   
   You can attempt to backport this manually by running:
   
   ```bash
   cherry_picker 0db5838 v3-1-test
   ```
   
   This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
   the files that need manual conflict resolution.
   
   After you have resolved the conflicts, you can continue the backport process by running:
   
   ```bash
   cherry_picker --continue
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W merged PR #57547:
URL: https://github.com/apache/airflow/pull/57547


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix: HITL params not validating [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #57547:
URL: https://github.com/apache/airflow/pull/57547#discussion_r2500218389


##########
airflow-core/src/airflow/ui/src/utils/hitl.ts:
##########
@@ -108,11 +108,11 @@ export const getHITLParamsDict = (
     const sourceParams = hitlDetail.response_received ? hitlDetail.params_input : hitlDetail.params;
 
     Object.entries(sourceParams ?? {}).forEach(([key, value]) => {
-      const valueType = typeof value === "number" ? "number" : "string";
+      const paramData = hitlDetail.params?.[key] ?? {};
 
       paramsDict[key] = {
-        description: "",
-        schema: {
+        description: paramData?.description ?? "",
+        schema: paramData?.schema ?? {

Review Comment:
   The type of `paramData?.schema` is still `[key: string]: unknown`. I think it needs some modification to reflect it exact type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org