You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "iting0321 (via GitHub)" <gi...@apache.org> on 2026/04/08 13:43:33 UTC

[PR] Add stairway test for DB migrations [airflow]

iting0321 opened a new pull request, #64905:
URL: https://github.com/apache/airflow/pull/64905

   
   Adds a parametrized test that walks every incremental Alembic migration one step at a time  **upgrade → downgrade → re-upgrade**  to verify all migrations are fully reversible.
   
   ## Why:
   Silent breakage can occur when a downgrade forgets to drop a constraint/column added by its upgrade, or references something that doesn't exist. The existing `test_database_schema_and_sqlalchemy_model_are_in_sync` only checks the final state; it won't catch a broken intermediate step.
   
   ## How to test:
   ```bash
   uv run --project airflow-core pytest airflow-core/tests/unit/migrations/test_migration_utils.py -xvs
   ```
   Each test ID is the revision hash (e.g. 937cbd173ca1), so a failure points directly to the broken migration.
   
   
   **Was generative AI tooling used to co-author this PR?**
   - [x] Yes (please specify the tool below) Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "iting0321 (via GitHub)" <gi...@apache.org>.
iting0321 commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3061967395


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the squashed baseline before the stairway test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+    """
+    config = _get_alembic_config()
+
+    # Downgrade to the squashed base so the stairway starts from a known state.
+    command.downgrade(config, revision=_SQUASHED_BASE_REVISION)
+
+    yield
+
+    # Always restore to the latest heads, even if the test failed mid-way.
+    upgradedb()
+
+
+def test_migration_stairway(stairway_db) -> None:
+    """
+    Walk every incremental migration step: upgrade → downgrade → re-upgrade.
+
+    Runs as a single test (not parametrized) so execution order and DB state
+    are guaranteed without relying on xdist grouping or cross-test ordering.
+    Each step uses Alembic's relative ``-1`` target for downgrade so that merge
+    revisions are handled correctly — it always rolls back exactly one step from
+    the current head regardless of how many parents a merge revision has.
+    """
+    config = _get_alembic_config()
+    revisions = _get_revisions_in_order()
+
+    for revision_id in revisions:
+        # Step 1: upgrade to this revision
+        command.upgrade(config, revision=revision_id)
+

Review Comment:
   add pytest.fail(f"... at revision {revision_id!r}: {e}") in the except block



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "iting0321 (via GitHub)" <gi...@apache.org>.
iting0321 commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3061962869


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the squashed baseline before the stairway test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+    """
+    config = _get_alembic_config()
+
+    # Downgrade to the squashed base so the stairway starts from a known state.
+    command.downgrade(config, revision=_SQUASHED_BASE_REVISION)
+

Review Comment:
   Replaced with `upgradedb(to_revision=...)` / `downgrade(to_revision=...)` throughout



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3055070768


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,102 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (revision 4bc4d934e2bc, the 2.6.2
+snapshot) and iterates through each revision in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _get_alembic_config
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = "4bc4d934e2bc"
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    # Find the squashed base and return everything *after* it (the incremental steps)
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1 :]
+
+
+# Force these tests to run sequentially on the same worker to prevent DB state 
+# conflicts during parallel testing (e.g., when using pytest-xdist).
+@pytest.mark.xdist_group(name="migration_stairway")
+@pytest.mark.parametrize(
+    "revision_id",
+    [pytest.param(rev, id=rev) for rev in _get_revisions_in_order()],
+)
+def test_migration_stairway(revision_id: str) -> None:
+    """
+    For each migration step verify that upgrade followed by downgrade works.
+
+    The test applies the migration under test (upgrade to *revision_id*), then
+    immediately rolls it back (downgrade to the previous revision), and finally
+    re-applies it so that subsequent parametrized steps start from the correct
+    state.
+    """
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    revision_obj = script.get_revision(revision_id)
+    if revision_obj is None:
+        pytest.skip(f"Revision {revision_id!r} not found in migration scripts")
+
+    prev_revision = revision_obj.down_revision
+    if isinstance(prev_revision, tuple):
+        # Merge point – pick the first parent.
+        prev_revision = prev_revision[0]
+
+    # Step 1: upgrade to this revision
+    command.upgrade(config, revision=revision_id)
+
+    # Determine the target revision for downgrade
+    target = prev_revision if prev_revision else "base"
+    
+    try:
+        # Step 2: downgrade back to the previous revision
+        command.downgrade(config, revision=target)
+    finally:
+        # Step 3: re-apply so subsequent steps see a consistent DB state.
+        command.upgrade(config, revision=revision_id)

Review Comment:
   The test claims to start from the squashed baseline and then do single-step upgrade→downgrade per revision, but there is no setup that ensures the DB is at `_SQUASHED_BASE_REVISION` (or at the immediate parent) before the first parametrized case runs. In a typical DB-test run the DB is initialized at the current head, so the first `command.downgrade()` will traverse many revisions and won’t validate a single-step downgrade for just `revision_id`. Add an explicit module/session-scoped setup that brings the DB to the squashed baseline before running the stairway, and ensure teardown restores the DB to `heads` even on failure so other DB tests aren’t left on an intermediate schema.



##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,102 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (revision 4bc4d934e2bc, the 2.6.2
+snapshot) and iterates through each revision in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _get_alembic_config
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = "4bc4d934e2bc"
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    # Find the squashed base and return everything *after* it (the incremental steps)
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1 :]
+
+
+# Force these tests to run sequentially on the same worker to prevent DB state 
+# conflicts during parallel testing (e.g., when using pytest-xdist).
+@pytest.mark.xdist_group(name="migration_stairway")
+@pytest.mark.parametrize(
+    "revision_id",
+    [pytest.param(rev, id=rev) for rev in _get_revisions_in_order()],
+)

Review Comment:
   This parametrized test implicitly depends on state being carried from one `revision_id` case to the next (so each case starts from the previous revision). With pytest-xdist, individual parametrized cases are scheduled independently and (with Breeze) xdist is invoked without `--dist=loadgroup`, so `xdist_group` will not actually keep all params on the same worker. Consider rewriting this as a single test that loops through revisions (so order/state is guaranteed), or otherwise make each case self-contained and not rely on cross-test DB state.



##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,102 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (revision 4bc4d934e2bc, the 2.6.2
+snapshot) and iterates through each revision in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _get_alembic_config
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = "4bc4d934e2bc"
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    # Find the squashed base and return everything *after* it (the incremental steps)
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1 :]
+
+
+# Force these tests to run sequentially on the same worker to prevent DB state 
+# conflicts during parallel testing (e.g., when using pytest-xdist).
+@pytest.mark.xdist_group(name="migration_stairway")
+@pytest.mark.parametrize(

Review Comment:
   There’s trailing whitespace on the comment line here (and also around the blank line before the `try:`). Please remove trailing spaces to keep diffs clean and satisfy common linters.



##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,102 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (revision 4bc4d934e2bc, the 2.6.2
+snapshot) and iterates through each revision in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _get_alembic_config
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = "4bc4d934e2bc"
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    # Find the squashed base and return everything *after* it (the incremental steps)
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1 :]
+
+
+# Force these tests to run sequentially on the same worker to prevent DB state 
+# conflicts during parallel testing (e.g., when using pytest-xdist).
+@pytest.mark.xdist_group(name="migration_stairway")
+@pytest.mark.parametrize(
+    "revision_id",
+    [pytest.param(rev, id=rev) for rev in _get_revisions_in_order()],
+)
+def test_migration_stairway(revision_id: str) -> None:
+    """
+    For each migration step verify that upgrade followed by downgrade works.
+
+    The test applies the migration under test (upgrade to *revision_id*), then
+    immediately rolls it back (downgrade to the previous revision), and finally
+    re-applies it so that subsequent parametrized steps start from the correct
+    state.
+    """
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    revision_obj = script.get_revision(revision_id)
+    if revision_obj is None:
+        pytest.skip(f"Revision {revision_id!r} not found in migration scripts")
+
+    prev_revision = revision_obj.down_revision
+    if isinstance(prev_revision, tuple):
+        # Merge point – pick the first parent.
+        prev_revision = prev_revision[0]
+

Review Comment:
   Merge revisions aren’t handled correctly: when `down_revision` is a tuple, picking the first parent can send the downgrade to an arbitrary branch and won’t exercise the real “one step back” for the merge revision. Prefer using Alembic’s relative targets (e.g. downgrade one step from the current revision) or otherwise compute the correct downgrade target for merge points so the test validates reversibility of the merge migration itself.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "iting0321 (via GitHub)" <gi...@apache.org>.
iting0321 commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3061963457


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the squashed baseline before the stairway test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+    """
+    config = _get_alembic_config()
+
+    # Downgrade to the squashed base so the stairway starts from a known state.
+    command.downgrade(config, revision=_SQUASHED_BASE_REVISION)
+
+    yield
+
+    # Always restore to the latest heads, even if the test failed mid-way.
+    upgradedb()
+
+
+def test_migration_stairway(stairway_db) -> None:
+    """
+    Walk every incremental migration step: upgrade → downgrade → re-upgrade.
+
+    Runs as a single test (not parametrized) so execution order and DB state
+    are guaranteed without relying on xdist grouping or cross-test ordering.
+    Each step uses Alembic's relative ``-1`` target for downgrade so that merge
+    revisions are handled correctly — it always rolls back exactly one step from
+    the current head regardless of how many parents a merge revision has.
+    """
+    config = _get_alembic_config()
+    revisions = _get_revisions_in_order()
+
+    for revision_id in revisions:
+        # Step 1: upgrade to this revision
+        command.upgrade(config, revision=revision_id)
+
+        # Step 2: downgrade exactly one step back.
+        # Using the relative specifier "-1" from the current head is correct for
+        # both normal and merge revisions — it never picks an arbitrary parent.
+        command.downgrade(config, revision="-1")

Review Comment:
   Raises ValueError with a clear message pointing to `_REVISION_HEADS_MAP`



##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the squashed baseline before the stairway test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+    """
+    config = _get_alembic_config()
+
+    # Downgrade to the squashed base so the stairway starts from a known state.
+    command.downgrade(config, revision=_SQUASHED_BASE_REVISION)
+
+    yield
+
+    # Always restore to the latest heads, even if the test failed mid-way.
+    upgradedb()
+
+
+def test_migration_stairway(stairway_db) -> None:
+    """
+    Walk every incremental migration step: upgrade → downgrade → re-upgrade.
+
+    Runs as a single test (not parametrized) so execution order and DB state
+    are guaranteed without relying on xdist grouping or cross-test ordering.
+    Each step uses Alembic's relative ``-1`` target for downgrade so that merge
+    revisions are handled correctly — it always rolls back exactly one step from
+    the current head regardless of how many parents a merge revision has.
+    """
+    config = _get_alembic_config()
+    revisions = _get_revisions_in_order()
+
+    for revision_id in revisions:
+        # Step 1: upgrade to this revision
+        command.upgrade(config, revision=revision_id)
+
+        # Step 2: downgrade exactly one step back.
+        # Using the relative specifier "-1" from the current head is correct for
+        # both normal and merge revisions — it never picks an arbitrary parent.
+        command.downgrade(config, revision="-1")

Review Comment:
   Raises ValueError with a clear message pointing to `_REVISION_HEADS_MAP`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 merged PR #64905:
URL: https://github.com/apache/airflow/pull/64905


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "iting0321 (via GitHub)" <gi...@apache.org>.
iting0321 commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3061966240


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]

Review Comment:
   Raises ValueError with a clear message pointing to `_REVISION_HEADS_MAP`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "iting0321 (via GitHub)" <gi...@apache.org>.
iting0321 commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3061981438


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the squashed baseline before the stairway test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+    """
+    config = _get_alembic_config()
+
+    # Downgrade to the squashed base so the stairway starts from a known state.
+    command.downgrade(config, revision=_SQUASHED_BASE_REVISION)
+
+    yield
+
+    # Always restore to the latest heads, even if the test failed mid-way.
+    upgradedb()
+
+
+def test_migration_stairway(stairway_db) -> None:
+    """
+    Walk every incremental migration step: upgrade → downgrade → re-upgrade.
+
+    Runs as a single test (not parametrized) so execution order and DB state
+    are guaranteed without relying on xdist grouping or cross-test ordering.
+    Each step uses Alembic's relative ``-1`` target for downgrade so that merge

Review Comment:
   Delete parametrized in the PR description, because of Order safety and DB state safety concerns.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "iting0321 (via GitHub)" <gi...@apache.org>.
iting0321 commented on PR #64905:
URL: https://github.com/apache/airflow/pull/64905#issuecomment-4220296799

   > Interesting idea. Can you please address the comments from copilot ?
   
   No problem, thank you for your feedback!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3056860475


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the squashed baseline before the stairway test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+    """
+    config = _get_alembic_config()
+
+    # Downgrade to the squashed base so the stairway starts from a known state.
+    command.downgrade(config, revision=_SQUASHED_BASE_REVISION)
+

Review Comment:
   The test runs migrations via raw `alembic.command.upgrade/downgrade`, bypassing Airflow’s `upgradedb()`/`downgrade()` wrappers that add important safeguards (global migration lock, MySQL metadata-lock handling, and single-connection pool). It also skips the FAB downgrade handling that `airflow.utils.db.downgrade()` performs for downgrades to < 2.10.3, which this test does when going back to the 2.6.2 squashed baseline. Prefer using `airflow.utils.db.upgradedb(to_revision=...)` and `airflow.utils.db.downgrade(to_revision=...)` (or reusing the same locking/pool context managers) for each step so the test is reliable across supported DB backends and installed providers.



##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]

Review Comment:
   `_get_revisions_in_order()` falls back to returning all revision IDs when the squashed base revision is not present, but the fixture always tries to downgrade to `_SQUASHED_BASE_REVISION` first. If that revision is genuinely missing, the test will fail earlier during `command.downgrade(...)` anyway. Consider making this an explicit failure (assert) or `pytest.skip` with a clear message, rather than returning a list that can’t be used with the chosen starting point.



##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the squashed baseline before the stairway test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+    """
+    config = _get_alembic_config()
+
+    # Downgrade to the squashed base so the stairway starts from a known state.
+    command.downgrade(config, revision=_SQUASHED_BASE_REVISION)
+
+    yield
+
+    # Always restore to the latest heads, even if the test failed mid-way.
+    upgradedb()
+
+
+def test_migration_stairway(stairway_db) -> None:
+    """
+    Walk every incremental migration step: upgrade → downgrade → re-upgrade.
+
+    Runs as a single test (not parametrized) so execution order and DB state
+    are guaranteed without relying on xdist grouping or cross-test ordering.
+    Each step uses Alembic's relative ``-1`` target for downgrade so that merge

Review Comment:
   The PR description says this is a parametrized test with one test ID per revision hash, but this implementation is intentionally a single non-parametrized test. Either update the PR description to match the implementation, or change the test to be parametrized (while still ensuring DB state/order safety) so failures are attributed to a specific revision in the test report.



##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the squashed baseline before the stairway test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+    """
+    config = _get_alembic_config()
+
+    # Downgrade to the squashed base so the stairway starts from a known state.
+    command.downgrade(config, revision=_SQUASHED_BASE_REVISION)
+
+    yield
+
+    # Always restore to the latest heads, even if the test failed mid-way.
+    upgradedb()
+
+
+def test_migration_stairway(stairway_db) -> None:
+    """
+    Walk every incremental migration step: upgrade → downgrade → re-upgrade.
+
+    Runs as a single test (not parametrized) so execution order and DB state
+    are guaranteed without relying on xdist grouping or cross-test ordering.
+    Each step uses Alembic's relative ``-1`` target for downgrade so that merge
+    revisions are handled correctly — it always rolls back exactly one step from
+    the current head regardless of how many parents a merge revision has.
+    """
+    config = _get_alembic_config()
+    revisions = _get_revisions_in_order()
+
+    for revision_id in revisions:
+        # Step 1: upgrade to this revision
+        command.upgrade(config, revision=revision_id)
+
+        # Step 2: downgrade exactly one step back.
+        # Using the relative specifier "-1" from the current head is correct for
+        # both normal and merge revisions — it never picks an arbitrary parent.
+        command.downgrade(config, revision="-1")

Review Comment:
   Same issue in the main loop: calling Alembic commands directly skips Airflow’s migration wrappers (locking, MySQL handling, etc.). Using `upgradedb(to_revision=revision_id)` / `downgrade(to_revision='-1')` (or equivalent shared context managers around Alembic calls) will avoid backend-specific hangs and provider-related downgrade failures.



##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,103 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (the 2.6.2 snapshot, looked up via
+``_REVISION_HEADS_MAP["2.6.2"]``) and iterates through each subsequent revision
+in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _REVISION_HEADS_MAP, _get_alembic_config, upgradedb
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = _REVISION_HEADS_MAP["2.6.2"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the squashed base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    try:
+        base_index = revision_ids.index(_SQUASHED_BASE_REVISION)
+    except ValueError:
+        return revision_ids  # squashed revision not present, return all
+
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the squashed baseline before the stairway test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+    """
+    config = _get_alembic_config()
+
+    # Downgrade to the squashed base so the stairway starts from a known state.
+    command.downgrade(config, revision=_SQUASHED_BASE_REVISION)
+
+    yield
+
+    # Always restore to the latest heads, even if the test failed mid-way.
+    upgradedb()
+
+
+def test_migration_stairway(stairway_db) -> None:
+    """
+    Walk every incremental migration step: upgrade → downgrade → re-upgrade.
+
+    Runs as a single test (not parametrized) so execution order and DB state
+    are guaranteed without relying on xdist grouping or cross-test ordering.
+    Each step uses Alembic's relative ``-1`` target for downgrade so that merge
+    revisions are handled correctly — it always rolls back exactly one step from
+    the current head regardless of how many parents a merge revision has.
+    """
+    config = _get_alembic_config()
+    revisions = _get_revisions_in_order()
+
+    for revision_id in revisions:
+        # Step 1: upgrade to this revision
+        command.upgrade(config, revision=revision_id)
+

Review Comment:
   If a migration fails, the exception will surface without explicitly stating which `revision_id` was being exercised, which can make failures harder to triage in CI logs. Consider wrapping the per-revision steps to re-raise/`pytest.fail` with the current `revision_id` included in the message so the broken migration is immediately obvious.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "iting0321 (via GitHub)" <gi...@apache.org>.
iting0321 commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3061976451


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,102 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (revision 4bc4d934e2bc, the 2.6.2
+snapshot) and iterates through each revision in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _get_alembic_config
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = "4bc4d934e2bc"

Review Comment:
   fixed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #64905:
URL: https://github.com/apache/airflow/pull/64905#issuecomment-4212499345

   Interesting idea. Can you please address the comments from copilot ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3051753103


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,102 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (revision 4bc4d934e2bc, the 2.6.2
+snapshot) and iterates through each revision in topological order.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic import command
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import _get_alembic_config
+
+pytestmark = pytest.mark.db_test
+
+# The squashed "start-of-history" revision – stairway starts here.
+_SQUASHED_BASE_REVISION = "4bc4d934e2bc"

Review Comment:
   Should this revision import from core instead of hard coded here?



##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,102 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every individual migration step forward (upgrade) then backward (downgrade)
+to verify that all migrations are fully reversible.  This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test starts from the squashed baseline (revision 4bc4d934e2bc, the 2.6.2
+snapshot) and iterates through each revision in topological order.

Review Comment:
   Just FYI that we define the revision here: 
   
   https://github.com/apache/airflow/blob/498acc8cf269e3ec2717eec25483aac7bf344533/airflow-core/src/airflow/utils/db.py#L105-L120



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #64905:
URL: https://github.com/apache/airflow/pull/64905#issuecomment-4343931866

   Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #64905:
URL: https://github.com/apache/airflow/pull/64905#issuecomment-4323600190

   @iting0321 This PR has a few issues that need to be addressed before it can be reviewed — please see our [Pull Request quality criteria](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-quality-criteria).
   
   **Issues found:**
   - :x: **Pre-commit / static checks**: The `Static checks` job is failing. Static-check failures are deterministic and need a code fix — re-running CI won't help. Run `prek run --from-ref main --stage pre-commit` locally to reproduce, then push the fix.
   
   **What to do next:**
   - Fix the static-check failures listed above.
   - Run `prek run --from-ref main --stage pre-commit` locally to verify before pushing.
   - Push the fix when ready.
   
   There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the [Airflow Slack](https://s.apache.org/airflow-slack).
   
   ---
   
   _Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this [two-stage triage process](https://github.com/apache/airflow/blob/main/contributing-docs/25_maintainer_pr_triage.md#why-the-first-pass-is-automated) so that our maintainers' limited time is spent where it matters most: the conversation with you._
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "iting0321 (via GitHub)" <gi...@apache.org>.
iting0321 commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3067777874


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every migration step since the 3.0.0 release forward (upgrade) then
+backward (downgrade) to verify that all migrations are fully reversible.
+This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test runs as a single (non-parametrized) function so that execution order
+and DB state are guaranteed without relying on xdist grouping or cross-test
+state.  Failures include the specific revision ID so the broken migration is
+immediately identifiable in CI logs.

Review Comment:
   update PR description



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3067772818


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every migration step since the 3.0.0 release forward (upgrade) then
+backward (downgrade) to verify that all migrations are fully reversible.
+This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test runs as a single (non-parametrized) function so that execution order
+and DB state are guaranteed without relying on xdist grouping or cross-test
+state.  Failures include the specific revision ID so the broken migration is
+immediately identifiable in CI logs.

Review Comment:
   PR description still states that failures are reported as one test ID per revision hash, but this test is intentionally implemented as a single non-parametrized test (as described in this module docstring). Please update the PR description to match the implementation (or, if you still want per-revision test IDs, refactor to parametrization while preserving DB/order safety).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add stairway test for DB migrations [airflow]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #64905:
URL: https://github.com/apache/airflow/pull/64905#discussion_r3066510778


##########
airflow-core/tests/unit/migrations/test_migration_utils.py:
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Stairway test for Airflow DB migrations.
+
+Walks every migration step since the 3.0.0 release forward (upgrade) then
+backward (downgrade) to verify that all migrations are fully reversible.
+This catches problems like:
+- a migration that creates a constraint the downgrade forgets to drop
+- a downgrade that references a column that was never added by the upgrade
+
+The test runs as a single (non-parametrized) function so that execution order
+and DB state are guaranteed without relying on xdist grouping or cross-test
+state.  Failures include the specific revision ID so the broken migration is
+immediately identifiable in CI logs.
+"""
+
+from __future__ import annotations
+
+import pytest
+from alembic.script import ScriptDirectory
+
+from airflow.utils.db import (
+    _REVISION_HEADS_MAP,
+    _get_alembic_config,
+    downgrade,
+    upgradedb,
+)
+
+pytestmark = pytest.mark.db_test
+
+# Stairway starts from the 3.0.0 head revision.  Starting here (rather than
+# the 2.6.2 squashed baseline) avoids triggering FAB-provider downgrade
+# handling that airflow.utils.db.downgrade() applies when the target is below
+# 2.10.3, and keeps the number of steps manageable on slower backends.
+_STAIRWAY_START_REVISION = _REVISION_HEADS_MAP["3.0.0"]
+
+
+def _get_revisions_in_order() -> list[str]:
+    """Return revision IDs in upgrade order, starting after the stairway base."""
+    config = _get_alembic_config()
+    script = ScriptDirectory.from_config(config)
+
+    # walk_revisions() yields from head → base; reverse for upgrade order.
+    all_revisions = list(script.walk_revisions())
+    all_revisions.reverse()
+
+    revision_ids = [rev.revision for rev in all_revisions]
+
+    if _STAIRWAY_START_REVISION not in revision_ids:
+        raise ValueError(
+            f"Stairway base revision {_STAIRWAY_START_REVISION!r} not found in migration scripts. "
+            "Update _STAIRWAY_START_REVISION to a valid revision from _REVISION_HEADS_MAP."
+        )
+
+    base_index = revision_ids.index(_STAIRWAY_START_REVISION)
+    return revision_ids[base_index + 1:]
+
+
+@pytest.fixture(scope="module")
+def stairway_db():
+    """
+    Bring the DB to the stairway base revision before the test runs, and
+    restore it to the current heads afterwards so other tests are unaffected.
+
+    Uses Airflow's downgrade()/upgradedb() wrappers so that backend-specific
+    handling (MySQL metadata-lock, global migration lock, single-connection
+    pool) is applied consistently.
+    """
+    downgrade(to_revision=_STAIRWAY_START_REVISION)
+
+    yield
+
+    # Restore to latest heads even if the test failed mid-way.
+    upgradedb()
+
+
+def test_migration_stairway(stairway_db) -> None:
+    """
+    Walk every incremental migration step since 3.0.0: upgrade → downgrade → re-upgrade.
+
+    Uses Airflow's upgradedb()/downgrade() wrappers for every step so that
+    the global migration lock, MySQL metadata-lock handling, and
+    single-connection pool are active — matching real-world upgrade/downgrade
+    behaviour and preventing backend-specific hangs.
+
+    The relative ``-1`` specifier is used for downgrade so that merge revisions
+    are handled correctly: it always rolls back exactly one step from the
+    current head regardless of how many parents a merge revision has.
+    """
+    revisions = _get_revisions_in_order()
+
+    for revision_id in revisions:
+        try:
+            # Step 1: upgrade to this revision.
+            upgradedb(to_revision=revision_id)
+
+            # Step 2: downgrade exactly one step back.
+            downgrade(to_revision="-1")
+
+            # Step 3: re-apply so the next iteration starts from the right state.
+            upgradedb(to_revision=revision_id)
+        except Exception as e:
+            pytest.fail(f"Stairway test failed at revision {revision_id!r}: {e}")

Review Comment:
   Catching the exception and calling `pytest.fail(...)` with only `str(e)` loses the original traceback/context from the failing migration, making failures harder to debug. Prefer re-raising with an exception chain (e.g., raising an `AssertionError`/custom exception `from e`) or otherwise preserving the original traceback while still including the `revision_id` in the message.
   ```suggestion
               raise AssertionError(f"Stairway test failed at revision {revision_id!r}") from e
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org