You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "hussein-awala (via GitHub)" <gi...@apache.org> on 2023/06/20 22:15:54 UTC

[GitHub] [airflow] hussein-awala opened a new pull request, #32038: Add an option to load the dags from db for command tasks run

hussein-awala opened a new pull request, #32038:
URL: https://github.com/apache/airflow/pull/32038

   <!--
   Thank you for contributing! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   In case of an existing issue, reference it using one of the following:
   
   closes: #ISSUE
   related: #ISSUE
   
   How to write a good git commit message:
   http://chris.beams.io/posts/git-commit/
   -->
   closes: #32020
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk merged pull request #32038: Add an option to load the dags from db for command tasks run

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #32038:
URL: https://github.com/apache/airflow/pull/32038


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] ephraimbuddy commented on pull request #32038: Add an option to load the dags from db for command tasks run

Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on PR #32038:
URL: https://github.com/apache/airflow/pull/32038#issuecomment-1600631461

   > don't you think making reading from the DB the default and having a flag to force a full re-parse would make more sense ? Why make the slow command the default when the fast one is going to be what the user wants 99% of the time ?
   > 
   > Also, it'd probably more likely occur to me to look in the doc for an option to force a refresh than for an option that does not refresh.
   
   It will be a breaking change for those that uses `run_as_user` in their dags if you make this change the default
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] ephraimbuddy commented on pull request #32038: Add an option to load the dags from db for command tasks run

Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on PR #32038:
URL: https://github.com/apache/airflow/pull/32038#issuecomment-1600634073

   @hussein-awala , even though you linked the issue you are working on in this PR, it's necessary to provide a good commit message for the PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] shubham22 commented on pull request #32038: Add an option to load the dags from db for command tasks run

Posted by "shubham22 (via GitHub)" <gi...@apache.org>.
shubham22 commented on PR #32038:
URL: https://github.com/apache/airflow/pull/32038#issuecomment-1599678724

   Agree with @vandonr-amz on making the default option as optimized version and then letting customers decide if they want to refresh the DAGs


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] hussein-awala commented on pull request #32038: Add an option to load the dags from db for command tasks run

Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on PR #32038:
URL: https://github.com/apache/airflow/pull/32038#issuecomment-1601285858

   > @hussein-awala , even though you linked the issue you are working on in this PR, it's necessary to provide a good commit message for the PR
   
   :+1:
   I just added a description


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] hussein-awala commented on pull request #32038: Add an option to load the dags from db for command tasks run

Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on PR #32038:
URL: https://github.com/apache/airflow/pull/32038#issuecomment-1601285031

   > It will be a breaking change for those that uses run_as_user in their dags if you make this change the default. Needs investigation...
   
   @ephraimbuddy I agree, but I think we should we can add a new Airflow configuration to specify the default value for this parameter. wdyt? 
   To do this, we would need to replace the boolean parameter with a string parameter, like:
   ```cfg
   DEFAULT_DAGS_SOURCE_FOR_TASKS_RUN = "dags_folder"
   # or
   DEFAULT_DAGS_SOURCE_FOR_TASKS_RUN = "metadata_db"
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] tirkarthi commented on pull request #32038: Add an option to load the dags from db for command tasks run

Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on PR #32038:
URL: https://github.com/apache/airflow/pull/32038#issuecomment-1623547429

   Reading from SerializedModel was originally added as part of AIP-45 in https://github.com/apache/airflow/pull/21877/files#diff-ad618185a072910e49c11770954af009d1cc070b120a4fde5f2fc095a588271bR360-R363 . It was changed in https://github.com/apache/airflow/pull/26750 . Reading from database was intentional as @ephraimbuddy mentioned. I am not sure if I am misunderstanding something where serialized models where changed after 2.4 to allow this.
   
   cc: @pingzh 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org