You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/02/06 12:15:04 UTC
[GitHub] [airflow] potiuk opened a new pull request, #29386: Mark Airflow directory in container as safe for git commands
potiuk opened a new pull request, #29386:
URL: https://github.com/apache/airflow/pull/29386
There is a new setting/version of git in GitHub Actions that started checking the ownership of the Git repository. Since in case of the provider commands we run them inside docker image as root user (this is in order to isolate the provider package building from the main CI environment), the owner of such directory is different (runner user) than the user that runs the git command (root).
This change marks the current git directory for such commands as safe, regardles from the discrepancy.
This config is global and run inside the image, so it is safe to leave it after methods complete as containers are torn-down after completing package preparation.
This PR also improves diagnostics. Previously the `git remote add` output was redirected to dev null as there was no way it could fail, but this turned to be false - the output of the `git remote add` commnd is now also printed for diagnostics.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #29386: Mark Airflow directory in container as safe for git commands
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29386:
URL: https://github.com/apache/airflow/pull/29386#issuecomment-1419029189
(Unless someone else wants to take a look till then).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #29386: Mark Airflow directory in container as safe for git commands
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29386:
URL: https://github.com/apache/airflow/pull/29386#issuecomment-1419275692
OK. Seems that it was the same reason - git complained about different user owning the .git repo and version was not retrievable. Added the same exclusion for building airflow packages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #29386: Mark Airflow directory in container as safe for git commands
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #29386:
URL: https://github.com/apache/airflow/pull/29386
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #29386: Mark Airflow directory in container as safe for git commands
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29386:
URL: https://github.com/apache/airflow/pull/29386#issuecomment-1419324181
OK. Issue solved (there are tests failing in main due to some upgraded dependencies, but this is another issue and can be solved later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #29386: Mark Airflow directory in container as safe for git commands
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29386:
URL: https://github.com/apache/airflow/pull/29386#issuecomment-1419028845
Not really seems that the new git has also another back-compat problem with retrieving git revision revision retrieval which prepares Airflow package and returns empty-hash (impact actual airflow prepareation). Need to take a look later (in 1 hour or so).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #29386: Mark Airflow directory in container as safe for git commands
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29386:
URL: https://github.com/apache/airflow/pull/29386#issuecomment-1419065836
BTW. Got at least clarity how we got there:
* we upgraded to latest debian base images automatically (with scheduled run)
* Looks like this is a problem with new git between 2.30.2 (default for previous debian/python image we had) and 2.38.1 (default in latest images).
This method (from airflow's setup.py) stopped returning git hash for airflow.
```
def git_version(version_: str) -> str:
"""
Return a version to identify the state of the underlying git repo. The version will
indicate whether the head of the current git-backed working directory is tied to a
release tag or not : it will indicate the former with a 'release:{version}' prefix
and the latter with a '.dev0' suffix. Following the prefix will be a sha of the current
branch head. Finally, a "dirty" suffix is appended to indicate that uncommitted
changes are present.
:param str version_: Semver version
:return: Found Airflow version in Git repo
"""
try:
import git
try:
repo = git.Repo(str(AIRFLOW_SOURCES_ROOT / ".git"))
except (git.NoSuchPathError):
logger.warning(".git directory not found: Cannot compute the git version")
return ""
except git.InvalidGitRepositoryError:
logger.warning("Invalid .git directory not found: Cannot compute the git version")
return ""
except ImportError:
logger.warning("gitpython not found: Cannot compute the git version.")
return ""
if repo:
sha = repo.head.commit.hexsha
if repo.is_dirty():
return f".dev0+{sha}.dirty"
# commit is clean
return f".release:{version_}+{sha}"
return "no_git_version"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #29386: Mark Airflow directory in container as safe for git commands
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29386:
URL: https://github.com/apache/airflow/pull/29386#issuecomment-1419009861
Almost... Had to explicitly add "/opt/airflow"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #29386: Mark Airflow directory in container as safe for git commands
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #29386:
URL: https://github.com/apache/airflow/pull/29386#issuecomment-1419024391
OK. Package preparation passed the place where it errored out. Build PROD images should get fixed automatically after merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org