You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/05/19 22:40:03 UTC

[GitHub] [airflow] jedcunningham opened a new pull request, #23821: Document fix for broken elasticsearch logs with 2.3.0+ upgrade

jedcunningham opened a new pull request, #23821:
URL: https://github.com/apache/airflow/pull/23821

   In certain upgrade paths, Airflow isn't given an opportunity to track
   the old `log_id_template`, so document the fix for folks who run into
   trouble.
   
   I also removed superfluous ES config that makes the important config harder to see.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham merged pull request #23821: Document fix for broken elasticsearch logs with 2.3.0+ upgrade

Posted by GitBox <gi...@apache.org>.
jedcunningham merged PR #23821:
URL: https://github.com/apache/airflow/pull/23821


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on pull request #23821: Document fix for broken elasticsearch logs with 2.3.0+ upgrade

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on PR #23821:
URL: https://github.com/apache/airflow/pull/23821#issuecomment-1132284476

   I also just had an idea on how to cover one more scenario automatically, which I'll experiment with tomorrow and open a separate PR for if it works out.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #23821: Document fix for broken elasticsearch logs with 2.3.0+ upgrade

Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #23821:
URL: https://github.com/apache/airflow/pull/23821#discussion_r877712265


##########
RELEASE_NOTES.rst:
##########
@@ -131,6 +131,13 @@ If you are happy with the new config values you should *remove* the setting in `
 
 If you have customized the templates you should ensure that they contain ``{{ ti.map_index }}`` if you want to use dynamically mapped tasks.
 
+If after upgrading you find your task logs are no longer accessible, try adding a row in the ``log_template`` table with ``id=0``
+containing your previous ``log_id_template`` and ``log_filename_template``. For example, if you used the defaults in 2.2.5:
+
+.. code-block:: sql
+
+    INSERT INTO log_template VALUES (0, '{{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}.log', '{dag_id}_{task_id}_{run_id}_{try_number}', NOW());

Review Comment:
   Might be a good idea to explicitly name the columns?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on pull request #23821: Document fix for broken elasticsearch logs with 2.3.0+ upgrade

Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23821:
URL: https://github.com/apache/airflow/pull/23821#issuecomment-1132273458

   I think in the next release we might an error linking to those release notes when logs cannot be retrieved - but it's not needed now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] github-actions[bot] commented on pull request #23821: Document fix for broken elasticsearch logs with 2.3.0+ upgrade

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23821:
URL: https://github.com/apache/airflow/pull/23821#issuecomment-1132273409

   The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest main or amend the last commit of the PR, and push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] jedcunningham commented on a diff in pull request #23821: Document fix for broken elasticsearch logs with 2.3.0+ upgrade

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #23821:
URL: https://github.com/apache/airflow/pull/23821#discussion_r878293123


##########
RELEASE_NOTES.rst:
##########
@@ -131,6 +131,13 @@ If you are happy with the new config values you should *remove* the setting in `
 
 If you have customized the templates you should ensure that they contain ``{{ ti.map_index }}`` if you want to use dynamically mapped tasks.
 
+If after upgrading you find your task logs are no longer accessible, try adding a row in the ``log_template`` table with ``id=0``
+containing your previous ``log_id_template`` and ``log_filename_template``. For example, if you used the defaults in 2.2.5:
+
+.. code-block:: sql
+
+    INSERT INTO log_template VALUES (0, '{{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}.log', '{dag_id}_{task_id}_{run_id}_{try_number}', NOW());

Review Comment:
   No objections. Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org