You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "hugoarnal (via GitHub)" <gi...@apache.org> on 2025/10/02 14:21:13 UTC
[I] Pydantic warning on Variable deserialization (PydanticSerializationUnexpectedValue) [airflow]
hugoarnal opened a new issue, #56332:
URL: https://github.com/apache/airflow/issues/56332
### Apache Airflow version
3.1.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
I'm currently having warnings whenever there is a deserialization of a Variable which stores a JSON array with dicts inside.
I tried with also a simple array and I'm getting the same errors.
```
[2025-10-02 15:56:59] WARNING - Pydantic serializer warnings:
PydanticSerializationUnexpectedValue(Expected `str` - serialized value may not be as expected [input_value=<cropped>, input_type=list]) category=UserWarning source=py.warnings loc=/<cropped>/.venv/lib/python3.12/site-packages/pydantic/main.py:463
```
### What you think should happen instead?
I started getting this error on all of my dags & tasks after migrating a codebase from Airflow 2.x to Airflow 3.x.
I don't think this should happen as this didn't before on Airflow 2.x.
### How to reproduce
The following example is based on a Variable storing a JSON array inside of it.
Here's a code example:
```py
from airflow.providers.standard.operators.empty import EmptyOperator
from airflow.sdk import TaskGroup, Variable, dag, task
import logging
from datetime import datetime
logger = logging.getLogger("airflow.task")
@task(retries=3)
def log_stuff(stuff: any) -> None:
logger.info(stuff)
@dag(
dag_id="variable_test",
start_date=datetime(2000, 1, 1),
tags=["test"],
)
def variable_test():
split_stuff = EmptyOperator(task_id="split_stuff")
stuff_list = Variable.get(
"list_of_stuff",
default=[],
deserialize_json=True,
)
for stuff in stuff_list:
with TaskGroup(
group_id=stuff
) as task_group:
log_stuff(stuff)
split_stuff >> task_group
variable_test()
```
Variable: `list_of_stuff`
`["hello_world", "testing", "last_one"]`
When using `json.loads` instead of `deserialize_json`, it doesn't show the warning. But I'm not sure it's a great idea to replace `deserialize_json` with `json.loads`. Here's the diff, if needed:
```diff
3a4
> import json
23,24c24,25
< default=[],
< deserialize_json=True,
---
> default="[]",
> deserialize_json=False,
25a27,28
>
> stuff_list = json.loads(stuff_list)
```
### Operating System
Ubuntu 24.04.3 LTS
### Versions of Apache Airflow Providers
`pip freeze | grep airflow`:
```ini
apache-airflow==3.1.0
apache-airflow-core==3.1.0
apache-airflow-providers-common-compat==1.7.4
apache-airflow-providers-common-io==1.6.3
apache-airflow-providers-common-sql==1.28.1
apache-airflow-providers-smtp==2.3.1
apache-airflow-providers-standard==1.9.0
apache-airflow-task-sdk==1.1.0
```
`pip freeze | grep pydantic`:
```ini
pydantic==2.11.9
pydantic_core==2.33.2
```
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Pydantic warning on Variable deserialization (PydanticSerializationUnexpectedValue) [airflow]
Posted by "hugoarnal (via GitHub)" <gi...@apache.org>.
hugoarnal closed issue #56332: Pydantic warning on Variable deserialization (PydanticSerializationUnexpectedValue)
URL: https://github.com/apache/airflow/issues/56332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Pydantic warning on Variable deserialization (PydanticSerializationUnexpectedValue) [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #56332:
URL: https://github.com/apache/airflow/issues/56332#issuecomment-3361448873
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Pydantic warning on Variable deserialization (PydanticSerializationUnexpectedValue) [airflow]
Posted by "hugoarnal (via GitHub)" <gi...@apache.org>.
hugoarnal commented on issue #56332:
URL: https://github.com/apache/airflow/issues/56332#issuecomment-3455525661
Seems to have been fixed with the update to Airflow 3.1.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org