You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Eason09053360 (via GitHub)" <gi...@apache.org> on 2026/08/02 14:41:00 UTC
[PR] Stop variables export/import from silently corrupting values [airflow]
Eason09053360 opened a new pull request, #70944:
URL: https://github.com/apache/airflow/pull/70944
`airflow variables export` followed by `airflow variables import` can silently rewrite a variable's value and invent a description for it.
Export writes a bare value when a variable has no description, while import treats any dict carrying a `"value"` key as a `{"value": ..., "description": ...}` envelope — nothing distinguishes the envelope from a value that happens to look like one. Export also decodes the stored JSON, losing the difference between raw text and a JSON-encoded string.
Round-tripping through a file on `main` :
| DB value before | exported as | after re-import |
|---|---|---|
| `{"value": "a", "description": "b"}` | `{"value": "a", "description": "b"}` | value → `a`, description → `b` |
| `{"value": 1, "other": 2}` | `{"value": 1, "other": 2}` | value → `1`, `other` dropped |
| `"hello"` | `hello` | `hello` — no longer valid JSON, so `deserialize_json=True` starts raising |
Import's reconstruction is deterministic — strings are stored verbatim, everything else is JSON-encoded, and one envelope layer is unwrapped — so export alone can be made lossless, leaving hand-written import files behaving exactly as before. Export now emits the stored form when the decoded value is a string, and wraps values that are themselves envelope-shaped so import's unwrap consumes our envelope rather than the user's data.
Exported files are unchanged for values that were never ambiguous. One visible change worth flagging: a variable stored as a JSON-encoded string (`Variable.set(k, "text", serialize_json=True)`) now exports as `"\"text\""` rather than `"text"`. The old output could not be re-imported without corruption, so there is no lossless way to keep it.
Left for follow-up PRs: import still truncates a hand-written `{"value": 1, "other": 2}` to `1`, and the UI import path does not unwrap the envelope at all.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 5)
Generated-by: Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop variables export/import from silently corrupting values [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #70944:
URL: https://github.com/apache/airflow/pull/70944#issuecomment-5332558818
### Backport successfully created: v3-3-test
Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>✅</td>
<td>v3-3-test</td>
<td><a href="https://github.com/apache/airflow/pull/71791"><img src="https://img.shields.io/badge/PR-71791-blue" alt="PR Link"></a></td>
</tr>
</table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop variables export/import from silently corrupting values [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #70944:
URL: https://github.com/apache/airflow/pull/70944
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop variables export/import from silently corrupting values [airflow]
Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on code in PR #70944:
URL: https://github.com/apache/airflow/pull/70944#discussion_r3710052037
##########
airflow-core/src/airflow/cli/commands/variable_command.py:
##########
@@ -186,11 +186,16 @@ def variables_export(args):
data = json.JSONDecoder()
for var in qry:
+ # Emit a form variables_import turns back into var.val: it stores strings verbatim,
+ # JSON-encodes everything else, and unwraps any dict carrying a "value" key.
try:
val = data.decode(var.val)
except Exception:
val = var.val
- if var.description:
+ else:
+ if isinstance(val, str):
+ val = var.val
Review Comment:
```suggestion
if isinstance(val, str):
val = var.val
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop variables export/import from silently corrupting values [airflow]
Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on code in PR #70944:
URL: https://github.com/apache/airflow/pull/70944#discussion_r3710077860
##########
airflow-core/src/airflow/cli/commands/variable_command.py:
##########
@@ -186,11 +186,16 @@ def variables_export(args):
data = json.JSONDecoder()
for var in qry:
+ # Emit a form variables_import turns back into var.val: it stores strings verbatim,
+ # JSON-encodes everything else, and unwraps any dict carrying a "value" key.
Review Comment:
```suggestion
# Mirror variables_import's reconstruction so export/import round-trips.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop variables export/import from silently corrupting values [airflow]
Posted by "Eason09053360 (via GitHub)" <gi...@apache.org>.
Eason09053360 commented on code in PR #70944:
URL: https://github.com/apache/airflow/pull/70944#discussion_r3766724268
##########
airflow-core/src/airflow/cli/commands/variable_command.py:
##########
@@ -186,11 +186,16 @@ def variables_export(args):
data = json.JSONDecoder()
for var in qry:
+ # Emit a form variables_import turns back into var.val: it stores strings verbatim,
+ # JSON-encodes everything else, and unwraps any dict carrying a "value" key.
try:
val = data.decode(var.val)
except Exception:
val = var.val
- if var.description:
+ else:
+ if isinstance(val, str):
+ val = var.val
Review Comment:
Thanks for reviewing!
aplied
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop variables export/import from silently corrupting values [airflow]
Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on code in PR #70944:
URL: https://github.com/apache/airflow/pull/70944#discussion_r3699838783
##########
airflow-core/newsfragments/70944.bugfix.rst:
##########
Review Comment:
I think we don't need this, please remove it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop variables export/import from silently corrupting values [airflow]
Posted by "Eason09053360 (via GitHub)" <gi...@apache.org>.
Eason09053360 commented on code in PR #70944:
URL: https://github.com/apache/airflow/pull/70944#discussion_r3766726198
##########
airflow-core/src/airflow/cli/commands/variable_command.py:
##########
@@ -186,11 +186,16 @@ def variables_export(args):
data = json.JSONDecoder()
for var in qry:
+ # Emit a form variables_import turns back into var.val: it stores strings verbatim,
+ # JSON-encodes everything else, and unwraps any dict carrying a "value" key.
Review Comment:
aplied
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org