You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/04 08:20:16 UTC
[GitHub] [airflow] anitakar opened a new pull request #10151: Acquire lock on db for the time of migration
anitakar opened a new pull request #10151:
URL: https://github.com/apache/airflow/pull/10151
We have a situation when both web server and k8s airflow-initdb can
start migrations at the same time.
Alembic in no way supports concurrent migrations.
The solution is based on https://github.com/sqlalchemy/alembic/issues/633
Change-Id: I5b894c947ec2e56efab622357e160e7c300b7b99
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kurtqq commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
kurtqq commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-799153695
shouldn't this PR be merged?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] feluelle commented on a change in pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
feluelle commented on a change in pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#discussion_r469990594
##########
File path: airflow/migrations/env.py
##########
@@ -98,7 +98,23 @@ def run_migrations_online():
)
with context.begin_transaction():
+ if connection.dialect.name == 'mysql' and \
+ connection.dialect.server_version_info >= (5, 6):
+ connection.execute(
+ "select GET_LOCK('alembic',1800);"
+ )
+ if connection.dialect.name == 'postgresql':
+ context._ensure_version_table() # pylint: disable=protected-access
Review comment:
Actually I think `run_migrations` will take care of this:
```python
...
if self.purge:
if self.as_sql:
raise util.CommandError("Can't use --purge with --sql mode")
self._ensure_version_table(purge=True)
heads = ()
else:
heads = self.get_current_heads()
if not self.as_sql and not heads:
self._ensure_version_table()
...
```
Which case is missing @anitakar?
But, I see, you need to ensure that before locking the table.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-673575275
Fixed it in #10320. Already merged. Unfortunately we have to ask once again for the rebase @anitakar :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] feluelle commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
feluelle commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-669864526
It looks like you need to rebase @anitakar - all other checks are 🟢 :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] anitakar commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
anitakar commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-671381639
> Also, what about other dialects? Or can this happen only for `mysql` not for `postgres` ?
>
> If, for now, we only want to support that for mysql in this PR, please add that info to the commit message and PR title. That would be great, thanks! :)
Totally forgot that I took Postgres solution originally, Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-669259602
@anitakar I rerun all the tests - one of the jobs (only one) for the MySQL tests failing with what it looked like a transient/environment error which did not seem related, but since this is a MySQL related change that is using the internal lock mechanism, I want to rerun those tests to make sure it was a transient error.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #10151:
URL: https://github.com/apache/airflow/pull/10151
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stale[bot] commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-751246649
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] anitakar commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
anitakar commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-801129803
Fixing those static checks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stale[bot] commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-703189617
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-673575731
That is the klind of teething problems when we introduce additional checks :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] anitakar commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
anitakar commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-799402552
Sure
On Mon, 15 Mar 2021, 10:56 Jarek Potiuk, ***@***.***> wrote:
> @anitakar <https://github.com/anitakar> - would you be so kind to rebase
> this one? It's been closed by the stale bot, but maybe indeed it's worth to
> rebase and merge this one?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/airflow/pull/10151#issuecomment-799283916>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABE6AZDTTMCCNILJERNM2G3TDXKTBANCNFSM4PUEPHEQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-804377033
Ah yeah!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] feluelle commented on a change in pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
feluelle commented on a change in pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#discussion_r464960841
##########
File path: airflow/migrations/env.py
##########
@@ -98,7 +98,19 @@ def run_migrations_online():
)
with context.begin_transaction():
+ if connection.dialect.name == 'mysql' and \
+ connection.dialect.server_version_info[0] == 5 and \
+ connection.dialect.server_version_info[1] == 7:
Review comment:
```suggestion
connection.dialect.server_version_info >= (5, 6):
```
##########
File path: airflow/migrations/env.py
##########
@@ -98,7 +98,19 @@ def run_migrations_online():
)
with context.begin_transaction():
+ if connection.dialect.name == 'mysql' and \
+ connection.dialect.server_version_info[0] == 5 and \
+ connection.dialect.server_version_info[1] == 7:
+ connection.execute(
+ "select GET_LOCK('alembic',1800);"
+ )
context.run_migrations()
+ if connection.dialect.name == 'mysql' and \
+ connection.dialect.server_version_info[0] == 5 and \
+ connection.dialect.server_version_info[1] == 7:
Review comment:
```suggestion
connection.dialect.server_version_info >= (5, 6):
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-669257821
Indeed. this is very mysql-y thingie.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] feluelle commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
feluelle commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-668524676
Also, what about other dialects? Or can this happen only for `mysql` not for `postgres` ?
If, for now, we only want to support that for mysql in this PR, please add that info to the commit message and PR title. That would be great, thanks! :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] feluelle commented on a change in pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
feluelle commented on a change in pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#discussion_r469990594
##########
File path: airflow/migrations/env.py
##########
@@ -98,7 +98,23 @@ def run_migrations_online():
)
with context.begin_transaction():
+ if connection.dialect.name == 'mysql' and \
+ connection.dialect.server_version_info >= (5, 6):
+ connection.execute(
+ "select GET_LOCK('alembic',1800);"
+ )
+ if connection.dialect.name == 'postgresql':
+ context._ensure_version_table() # pylint: disable=protected-access
Review comment:
Actually I think `run_migrations` will take care of this:
```python
...
if self.purge:
if self.as_sql:
raise util.CommandError("Can't use --purge with --sql mode")
self._ensure_version_table(purge=True)
heads = ()
else:
heads = self.get_current_heads()
if not self.as_sql and not heads:
self._ensure_version_table()
...
```
Which case is missing @anitakar?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-801012959
[The Workflow run](https://github.com/apache/airflow/actions/runs/660892515) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] anitakar commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
anitakar commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-671381794
> It looks like you need to rebase @anitakar - all other checks are 🟢 :)
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] anitakar commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
anitakar commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-800978965
@potiuk Sorry, for only rebasing it now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-671393330
We had a bad commit that broke the master #10276 but it's reverted now so if it failed with just this I am going to merge it anyway, so no worries if it does @anitakar
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] feluelle commented on a change in pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
feluelle commented on a change in pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#discussion_r469979521
##########
File path: airflow/migrations/env.py
##########
@@ -98,7 +98,23 @@ def run_migrations_online():
)
with context.begin_transaction():
+ if connection.dialect.name == 'mysql' and \
+ connection.dialect.server_version_info >= (5, 6):
+ connection.execute(
+ "select GET_LOCK('alembic',1800);"
+ )
+ if connection.dialect.name == 'postgresql':
+ context._ensure_version_table() # pylint: disable=protected-access
Review comment:
```suggestion
context.get_context()._ensure_version_table() # pylint: disable=protected-access
```
Because the `_ensure_version_table` function is only available in the `MigrationContext` you have to get this context from the (environment) context you are in.
From the `MigrationContext` docs:
```
When inside the ``env.py`` script, the :class:`.MigrationContext`
is available via the
:meth:`.EnvironmentContext.get_context` method,
which is available at ``alembic.context``::
# from within env.py script
from alembic import context
migration_context = context.get_context()
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-673572272
Ah. We just added spell-checker and one of our other PRs failed on "sharded" :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #10151:
URL: https://github.com/apache/airflow/pull/10151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-673578762
BTW. Once Github support resolves one of the issues that I created with them, I will add a new workflow that will enable to run rebase command from the UI of Github From "Actions" tab - and maybe even we will add a bot that will let you know that you should rebase and point you to that action.
Currently, we already have "Sync master from Apache Airflow" one but there is a weird bug that they are investigating that this "Workflow dispatch" workflow is missing in some forks, so I do not want to announce it widely.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kurtqq commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
kurtqq commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-804361870
@potiuk kind reminder :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-668861251
@feluelle From what I've seen, PostgresQL doesn't have a built-in lock function. Only rows, tables, etc. can be locked.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
turbaszek commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-706540657
@anitakar can you rebase the PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-799283916
@anitakar - would you be so kind to rebase this one? It's been closed by the stale bot, but maybe indeed it's worth to rebase and merge this one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #10151: Acquire lock on db for the time of migration
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #10151:
URL: https://github.com/apache/airflow/pull/10151#issuecomment-804377261
Thanks @anitakar !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org