You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ThePsyjo (via GitHub)" <gi...@apache.org> on 2024/06/13 12:11:56 UTC
[I] Data type convertion for extra in connection [airflow]
ThePsyjo opened a new issue, #40220:
URL: https://github.com/apache/airflow/issues/40220
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.7.3
### What happened?
Web-interface > Admin > Connections
When saving a float value in `extra`, it is getting stored correctly as float. Upon loading the connection again, the float is converted to int if it can be represented as int. After saving the connection again, i.e. after updating the password, the value is stored as int.
### What you think should happen instead?
Data types in extra should be preserved.
### How to reproduce
- in the web interface
- go to Admin > Connections
- add connection
- insert `{"version": 18.0}` in extra
- save
- edit this connection
- find it being `{"version": 18}`
### Operating System
na
### Versions of Apache Airflow Providers
https://cloud.google.com/composer/docs/concepts/versioning/composer-versions?hl=en
composer-2.8.1-airflow-2.7.3
### Deployment
Google Cloud Composer
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Data type conversion for extra in connection [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #40220: Data type conversion for extra in connection
URL: https://github.com/apache/airflow/issues/40220
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Data type conversion for extra in connection [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #40220:
URL: https://github.com/apache/airflow/issues/40220#issuecomment-2166649279
BTW. If you want to preserve ".0" and it makes a difference for you - you should store it as string and expect string in the value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Data type conversion for extra in connection [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #40220:
URL: https://github.com/apache/airflow/issues/40220#issuecomment-2166647697
Json does not make a difference between float and ints. https://www.json.org/json-en.html - this is just number. There is no way you can distinguish the two 18.0 when parsed is just 18 and that's how it is printed. Those two are equivalent and you can't do anything about it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Data type convertion for extra in connection [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #40220:
URL: https://github.com/apache/airflow/issues/40220#issuecomment-2165478889
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Data type conversion for extra in connection [airflow]
Posted by "ThePsyjo (via GitHub)" <gi...@apache.org>.
ThePsyjo commented on issue #40220:
URL: https://github.com/apache/airflow/issues/40220#issuecomment-2167454440
```
>>> import json
>>> json.dumps(18.0)
'18.0'
>>> json.loads(json.dumps(18.0))
18.0
```
The json module **does** preserve fractions though. As described earlier, the fraction **is** stored correctly. It is only converted to int when loaded by the web interface.
> BTW. If you want to preserve ".0" and it makes a difference for you - you should store it as string and expect string in the value.
I implemented this already to get my stuff working reliably again :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Data type conversion for extra in connection [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #40220:
URL: https://github.com/apache/airflow/issues/40220#issuecomment-2167526211
What you are referring to is an implementation detail of Python JSON parser - when you are using Python dictionaries you have indeed different types of fields - float and int are different. And the Parser might choose to add '.0' in this case.
But in this case, JSON specification is very clear. Any number that you pass in json is a "number" and JSON specificatio does not mandate or even hint, whether floating points should be preserved. 18.0 and 18 are exactly the same representation of the same number and json parser and producer has absolutely no expectation and requirement the .0 will be preserved when serializing/deserializing JSON. So JS does not do ugly stuff. It acts according to the specification. And you should never rely on the fact that .0 is preserved when you go through serialize/deserialize because it's not guaranteed or not even hinted that it should be preserved.
BTW. I am not sure if you are aware but `J` in JSON stands for `Javascript` because that's where JSON was originally used, so I'd argue that Python implementation diverged from what original JSON implementation did.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org