[GitHub] [airflow] josh-fell opened a new pull request #16867: Updating Airbyte example DAG to use XComArgs

[GitBox <gi...@apache.org>]

josh-fell opened a new pull request #16867:
URL: https://github.com/apache/airflow/pull/16867


   Related to #10285
   
   - Updated example DAG files such that `xcom_pull()` calls use an operator's `.output` property as well access of `TaskInstance` objects from context to use `get_current_context()` function
   - Added comments to which task dependencies, if any, are handled and/or created via `XComArgs` for transparency
   - Removed or refactored the `default_args` pattern where necessary as requested by @ashb (i.e. removed a separated `default_args` declaration for deference for declaration as part of the `DAG` object)
   - Other miscellaneous updates based on `.output` refactoring
   
   >**Note:** There are several instances where the `xcom_pull()` call was not updated.  These instances involve accessing a specific value within the `XCom` or calling user-defined macros with an `XCom` value.  Reference #16618 for an open issue to enhance the `XComArg` functionality to provide similar behavior as the classic `xcom_pull()` method.
   
   > **Note:** Not all DAGs were tested functionally (i.e. with hard integrations to source systems and executed), however each DAG was tested to compile and generate a DAG graph as expected locally.
   
   An detailed summary of all changes made as part of this PR can be found below:
   | DAG File | Converted `xcom_pull()`? | Other Updates? | Comments |
   | ---------| ------------------------ | -------------- | -------- |
   | airflow/providers/airbyte/example_dags/example_airbyte_trigger_job.py | Yes | Yes | Removed explicit task dependencies that are created via `XComArgs`.<br/><br/>Removed unneeded `default_args` pattern. |
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
   In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ashb merged pull request #16867: Updating Airbyte example DAG to use XComArgs

[GitBox <gi...@apache.org>]

ashb merged pull request #16867:
URL: https://github.com/apache/airflow/pull/16867


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on pull request #16867: Updating Airbyte example DAG to use XComArgs

[GitBox <gi...@apache.org>]

github-actions[bot] commented on pull request #16867:
URL: https://github.com/apache/airflow/pull/16867#issuecomment-876281505


   The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org