You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jscheffl (via GitHub)" <gi...@apache.org> on 2024/07/17 21:04:53 UTC

[PR] Make BashOperator compatible with Internal API AIP-44 [airflow]

jscheffl opened a new pull request, #40847:
URL: https://github.com/apache/airflow/pull/40847

   This is another PR as improvement for internal API in relation to implementation of AIP-69 in PoC PR #40224 - I relaized that BashOperator also has a DB access to update template fields.
   
   This PR adds one AIP endpoint to un-bundle the Operator from the DB.
   
   Note: I am aware that a similar endpoint was added with airflow/models/taskinstance.py:_update_rtif() but thepurpose it slightly different. Hard to consolidate w/o the need to have more calls over the API. Therefore I propose to add a new one here.
   
   With this actually the [Integration test](https://github.com/jscheffl/airflow/blob/feature/aip-69-poc/airflow/example_dags/integration_test.py) in my AIP-69 is completely working with all basic features! WOW! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make BashOperator compatible with Internal API AIP-44 [airflow]

Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on code in PR #40847:
URL: https://github.com/apache/airflow/pull/40847#discussion_r1681831912


##########
airflow/models/renderedtifields.py:
##########
@@ -153,6 +154,23 @@ def _redact(self):
         for field, rendered in self.rendered_fields.items():
             self.rendered_fields[field] = redact(rendered, field)
 
+    @classmethod
+    @internal_api_call
+    @provide_session
+    def update_runtime_evaluated_template_fields(
+        cls, ti: TaskInstance, session: Session = NEW_SESSION
+    ) -> None:
+        """Update rendered task instance fields for cases where runtime evaluated, not templated."""

Review Comment:
   can we make it "private" by either adding `:meta private:` or leading underscore?
   
   (i think best to avoid making _anything_ public unless really necessary)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make BashOperator compatible with Internal API AIP-44 [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #40847:
URL: https://github.com/apache/airflow/pull/40847


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make BashOperator compatible with Internal API AIP-44 [airflow]

Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #40847:
URL: https://github.com/apache/airflow/pull/40847#discussion_r1683008890


##########
airflow/models/renderedtifields.py:
##########
@@ -153,6 +154,23 @@ def _redact(self):
         for field, rendered in self.rendered_fields.items():
             self.rendered_fields[field] = redact(rendered, field)
 
+    @classmethod

Review Comment:
   +1



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make BashOperator compatible with Internal API AIP-44 [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #40847:
URL: https://github.com/apache/airflow/pull/40847#discussion_r1682454635


##########
airflow/models/renderedtifields.py:
##########
@@ -153,6 +154,23 @@ def _redact(self):
         for field, rendered in self.rendered_fields.items():
             self.rendered_fields[field] = redact(rendered, field)
 
+    @classmethod

Review Comment:
   Yes. There are a number of `@staticmethod` `@internal_api_call` already.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make BashOperator compatible with Internal API AIP-44 [airflow]

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #40847:
URL: https://github.com/apache/airflow/pull/40847#discussion_r1682408210


##########
airflow/models/renderedtifields.py:
##########
@@ -153,6 +154,23 @@ def _redact(self):
         for field, rendered in self.rendered_fields.items():
             self.rendered_fields[field] = redact(rendered, field)
 
+    @classmethod

Review Comment:
   A thought not totally tied to this PR. Should the internal API calls use `staticmethod` instead, since they never use `cls`? Or _can_ they use `cls`? They should if so.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make BashOperator compatible with Internal API AIP-44 [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #40847:
URL: https://github.com/apache/airflow/pull/40847#discussion_r1682454635


##########
airflow/models/renderedtifields.py:
##########
@@ -153,6 +154,23 @@ def _redact(self):
         for field, rendered in self.rendered_fields.items():
             self.rendered_fields[field] = redact(rendered, field)
 
+    @classmethod

Review Comment:
   Yes. Ther ere a number of `@staticmethod` `@internal_api_call` already.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make BashOperator compatible with Internal API AIP-44 [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #40847:
URL: https://github.com/apache/airflow/pull/40847#discussion_r1682107441


##########
airflow/models/renderedtifields.py:
##########
@@ -153,6 +154,23 @@ def _redact(self):
         for field, rendered in self.rendered_fields.items():
             self.rendered_fields[field] = redact(rendered, field)
 
+    @classmethod
+    @internal_api_call
+    @provide_session
+    def update_runtime_evaluated_template_fields(
+        cls, ti: TaskInstance, session: Session = NEW_SESSION
+    ) -> None:
+        """Update rendered task instance fields for cases where runtime evaluated, not templated."""

Review Comment:
   That totally makes sense. Thanks. Adjusted.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org