You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "SamWheating (via GitHub)" <gi...@apache.org> on 2024/02/26 23:31:20 UTC
[PR] WIP: adding run_id column to log table [airflow]
SamWheating opened a new pull request, #37731:
URL: https://github.com/apache/airflow/pull/37731
closes: https://github.com/apache/airflow/issues/37597
This adds the `run_id` column to the log table (without a backfill, see [discussion](https://github.com/apache/airflow/issues/37597#issuecomment-1965244838)) and exposes it in all of the different surfaces:
- the `/eventLogs` API
- the logs view (`/logs/list`)
- the inset / react audit logs DagRun details page
- the legacy (non-react) audit logs view in the DAG details page
Ignore the react stuff for now, it still needs some work.
#### Open questions:
- Should we be pre-populating the run_id filter in the inset audit logs view (similar to what we do with task_id when a single task is selected)? This would be more precise than just filtering on task_id + dates, but it would exclude relevant events which aren't tagged with a run_id (for example, pausing a DAG mid-execution)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "SamWheating (via GitHub)" <gi...@apache.org>.
SamWheating commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1969541907
> Since we have two migrations touching Log. Should we do both at once?
This is a pretty good idea. Which PR do you want to merge first? If needed I can port the DB changes from the other PR into this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1967559142
We actually have 3 in flight :sweat_smile:: #37734
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi merged PR #37731:
URL: https://github.com/apache/airflow/pull/37731
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "SamWheating (via GitHub)" <gi...@apache.org>.
SamWheating commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1967712398
> Just a watch-out - we have two migrations in-progress
Thanks for the heads up - I'll keep a close 👀 on the other two and try to keep this PR in a mergeable state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1977132329
> > > Since we have two migrations touching Log. Should we do both at once?
@jedcunningham @potiuk What do you prefer? Two separate event log table migrations, or combine them?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1971343757
> > Since we have two migrations touching Log. Should we do both at once?
>
> This is a pretty good idea. Which PR do you want to merge first? If needed I can port the DB changes from the other PR into this one.
The other Audit Log one still has errors in the CI. So let's merge this first. The only change I had to make over there is to increase the max size of the `event` string.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1969413973
> We actually have 3 in flight 😅: #37734
Since we have two migrations touching Log. Should we do both at once?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1965828471
I prepopulating works and then after this I will open up the before/after fields to be user-editable but with some default options like a dag run's entire durations. The before/after range also doesn't capture changing the state or a note.
After this and https://github.com/apache/airflow/pull/37734. We probably want to make sure we are populating run_id everywhere we can.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "SamWheating (via GitHub)" <gi...@apache.org>.
SamWheating commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1971632077
Looks like the migration PR was merged, I'll get this one updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1967615883
> We actually have 3 in flight 😅: #37734
😱 😱 😱 😱 😱 😱 😱 😱 😱
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Adding run_id column to log table [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #37731:
URL: https://github.com/apache/airflow/pull/37731#issuecomment-1967460355
Just a watch-out - we have two migrations in-progress. I think we have a protection that will make the secon one not mergable if the first is but there will be some re-work needed once the other mighration is merged https://github.com/apache/airflow/pull/37708
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org