You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Dev-iL (via GitHub)" <gi...@apache.org> on 2026/01/11 07:59:24 UTC
[I] Bug: DAG Bundle/User Impersonation Causes Permission Issues in Resource Preparation [airflow]
Dev-iL opened a new issue, #60387:
URL: https://github.com/apache/airflow/issues/60387
### Apache Airflow version
3.1.5
### If "Other Airflow 3 version" selected, which one?
_No response_
### What happened?
When running Git-based DAG bundles in Airflow with user impersonation (`run_as_user`), tasks may fail due to permission errors on resources like lock files and tracking directories. This occurs because bundles are initialized by the main airflow user before impersonation, making it necessary for the impersonated user to access these resources. Attempts to mitigate this in PRs [#60270](https://github.com/apache/airflow/pull/60270), [#60278](https://github.com/apache/airflow/pull/60278), and [#60280](https://github.com/apache/airflow/pull/60280) involve configuration changes and runtime warnings, but the underlying architectural problem remains unaddressed.
### What you think should happen instead?
Resource preparation for bundles should be performed post-impersonation—or in a context where only the appropriate user has write access. Impersonated users should not need elevated permissions or write access to resources initialized by the airflow service user. The fix should be holistic, ideally centralizing resource management with privilege separation and avoiding relaxed file permissions or risky group sharing.
### How to reproduce
1. Set up an Airflow deployment using Git-based DAG bundles and enable user impersonation with the `run_as_user` config.
2. Configure a DAG that fetches and interacts with resources stored in a shared repository (e.g., involving lock files, tracking directories).
3. Trigger the DAG with a task that switches user context.
4. Observe that the task fails with permission errors accessing bundle-related files or directories.
5. Attempt to mitigate by setting group-writable permissions or updating git's `safe.directory`; note that this works but relaxes security.
### Operating System
Linux
### Versions of Apache Airflow Providers
_No response_
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else?
- Short-term solutions as seen in recent PRs involve group-writable directories, explicit warnings, and setting `safe.directory`. However, these may not be viable for all production environments, such as containerized or multi-tenant stacks.
- Recommending a long-term refactor, possibly using a supervisor or Execution API to manage resource preparation, with only read-access needed for impersonated task users.
- Would appreciate feedback, especially for edge-case deployments.
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAG Bundle/User Impersonation Causes Permission Issues in Resource Preparation [airflow]
Posted by "Dev-iL (via GitHub)" <gi...@apache.org>.
Dev-iL commented on issue #60387:
URL: https://github.com/apache/airflow/issues/60387#issuecomment-3742897376
@Arunodoy18 That does sound like it aligns with @potiuk's [plan of action](https://github.com/apache/airflow/pull/60270#issuecomment-3729119122):
> ...instead of running git fetch by the task, we could add extra command to the executin API - so that task can ask supervisor to fetch the commit. This could solve all the problems and only require read permission to the bare repository in order to checkout the commit by task after it's been fetched by the supervisor. That has another advantage as well, we could potentially serialize those requests or batch them in supervisor and tha would avoid another quite likely potential issue where multiple tasks of the same dag run are running in parallel (which will pretty much always happen if we have mapped tasks) and asking to fetch the repository at the same time - this will work but it is not optimal - potentially many git conections opened, rate limiting might start playing a role etc. So leaving all the git fetch operations to the supervisor seems like a good idea.
>
> That however I think will require a bit more coupling between the bundle interface and execution api - because this is not exclusively a problem of GitBundle only - other bundles might have very similar problems and we should solve it in a generic way so that there is a generic task -> supervisor ("get bundle version NNN") has to be added to the execution
API.
>
> But I see that as the best long-term option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAG Bundle/User Impersonation Causes Permission Issues in Resource Preparation [airflow]
Posted by "Dev-iL (via GitHub)" <gi...@apache.org>.
Dev-iL commented on issue #60387:
URL: https://github.com/apache/airflow/issues/60387#issuecomment-3740519647
> From my understanding, the root problem here isn’t specific to Git-based bundles themselves, but to when and under which user context bundle-related resources are prepared.
Indeed, while this was observed in a git bundle, it should affect other bundle types too.
> The recent mitigations around group-writable directories, warnings, and `safe.directory` help reduce friction, but they still rely on relaxing filesystem permissions rather than addressing the lifecycle mismatch between resource preparation and execution context.
That's right - the fixes proposed in the mentioned PRs are intended as temporary measures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAG Bundle/User Impersonation Causes Permission Issues in Resource Preparation [airflow]
Posted by "Arunodoy18 (via GitHub)" <gi...@apache.org>.
Arunodoy18 commented on issue #60387:
URL: https://github.com/apache/airflow/issues/60387#issuecomment-3742788825
Thanks for confirming 👍
To move this forward, would it make sense to clearly separate bundle resource preparation from runtime execution context
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAG Bundle/User Impersonation Causes Permission Issues in Resource Preparation [airflow]
Posted by "Arunodoy18 (via GitHub)" <gi...@apache.org>.
Arunodoy18 commented on issue #60387:
URL: https://github.com/apache/airflow/issues/60387#issuecomment-3742931110
Thanks, this makes a lot of sense and I agree this feels like the right long-term direction 👍.
Moving the fetch responsibility to the supervisor via the execution API seems to cleanly address both:
1)the permission / impersonation mismatch
2)the parallel fetch contention issues you mentioned (multiple tasks fetching the same bundle concurrently.
I also agree that solving this generically at the bundle interface level (rather than Git Bundle-specific logic) is important, even if it introduces some additional coupling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAG Bundle/User Impersonation Causes Permission Issues in Resource Preparation [airflow]
Posted by "Arunodoy18 (via GitHub)" <gi...@apache.org>.
Arunodoy18 commented on issue #60387:
URL: https://github.com/apache/airflow/issues/60387#issuecomment-3740421445
From my understanding, the root problem here isn’t specific to Git-based bundles themselves, but to when and under which user context bundle-related resources are prepared.
The recent mitigations around group-writable directories, warnings, and `safe.directory` help reduce friction, but they still rely on relaxing filesystem permissions rather than addressing the lifecycle mismatch between resource preparation and execution context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org