You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/25 04:22:48 UTC
[GitHub] [airflow] dstandish opened a new pull request #22520: Deprecate use of prefx `extra____` in connections
dstandish opened a new pull request #22520:
URL: https://github.com/apache/airflow/pull/22520
We namespaced custom connection fields (which are stored in the `extra` json field) with prefix `extra__<conn type>__` in order to keep them unique in a global dictionary of fields.
This made it ugly and clunky to use these fields when you deal with the `extra` field directly, e.g. when using an external secrets backend or storing connections in env vars.
With a slight change in logic, we can keep this namespacing (which ensures that different connections can have fields with the same name and different labels) and strip the prefix prior to saving (and add it back on read) so that the `extra` JSON doesn't have the prefix.
That's the easy part. What was a little more challenging (or at least tedious) was to preserve backward compatibility, so users will be warned when their extras still have the deprecated prefix, but it will continue to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #22520: Deprecate use of prefx `extra____` in connections
Posted by GitBox <gi...@apache.org>.
dstandish commented on pull request #22520:
URL: https://github.com/apache/airflow/pull/22520#issuecomment-1078641035
@ashb @kaxil @josh-fell @potiuk still a draft, need to some more review and testing, but..... this seems to be a viable path forward re making it so we no longer need that ugly prefix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #22520: Deprecate use of prefx `extra____` in connections
Posted by GitBox <gi...@apache.org>.
dstandish commented on pull request #22520:
URL: https://github.com/apache/airflow/pull/22520#issuecomment-1079336972
ok so ... i updated to enhance the backcompat a bit.
the issue i was thinking of is, if user updates airflow but doesn't update provider. then, provider hook may expect conn attrs to have prefix still, so if they create a conn (or edit) from web UI, then we still need to read / store the fields with the prefix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #22520: Deprecate use of prefx `extra____` in connections
Posted by GitBox <gi...@apache.org>.
dstandish commented on pull request #22520:
URL: https://github.com/apache/airflow/pull/22520#issuecomment-1079769125
doing this all in one PR is too much
splitting it up
first PR, which just handles the change to form / providers manager is here: https://github.com/apache/airflow/pull/22546
then i will update providers, one at a time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish closed pull request #22520: Deprecate use of prefx `extra____` in connections
Posted by GitBox <gi...@apache.org>.
dstandish closed pull request #22520:
URL: https://github.com/apache/airflow/pull/22520
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish edited a comment on pull request #22520: Deprecate use of prefx `extra____` in connections
Posted by GitBox <gi...@apache.org>.
dstandish edited a comment on pull request #22520:
URL: https://github.com/apache/airflow/pull/22520#issuecomment-1078641035
@ashb @kaxil @josh-fell @potiuk still a draft, need to some more review and testing, but..... this seems to be a viable path forward re making it so we no longer need that ugly prefix
kindof exciting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org