You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/01 20:56:19 UTC
[GitHub] [airflow] davlum opened a new pull request #15146: Validate that the extra parameter is parseable as JSON
davlum opened a new pull request #15146:
URL: https://github.com/apache/airflow/pull/15146
Before this commit there was a documented but unenforced
limitation that the extra parameter be encoded JSON.
In #15013 this issue garnered attention and motivated
this PR.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #15146: Validate that the extra parameter is parseable as JSON
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #15146:
URL: https://github.com/apache/airflow/pull/15146#discussion_r606654260
##########
File path: airflow/models/connection.py
##########
@@ -137,7 +137,25 @@ def __init__( # pylint: disable=too-many-arguments
self.password = password
self.schema = schema
self.port = port
- self.extra = extra
+ self.extra = self.validate_extra(extra)
+
+ @staticmethod
+ def validate_extra(extra: str):
Review comment:
`None` is also the correct value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #15146: Validate that the extra parameter is parseable as JSON
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #15146:
URL: https://github.com/apache/airflow/pull/15146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15146: Validate that the extra parameter is parseable as JSON
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15146:
URL: https://github.com/apache/airflow/pull/15146#issuecomment-846648895
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #15146: Validate that the extra parameter is parseable as JSON
Posted by GitBox <gi...@apache.org>.
dstandish commented on pull request #15146:
URL: https://github.com/apache/airflow/pull/15146#issuecomment-813687022
alternatively we could continute to allow users to put arbitrary string in `extra`, but have change `extra_dejson` so that it returns `self.extra` in that case (instead of empty dict)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on a change in pull request #15146: Validate that the extra parameter is parseable as JSON
Posted by GitBox <gi...@apache.org>.
dstandish commented on a change in pull request #15146:
URL: https://github.com/apache/airflow/pull/15146#discussion_r606302213
##########
File path: airflow/models/connection.py
##########
@@ -137,7 +137,25 @@ def __init__( # pylint: disable=too-many-arguments
self.password = password
self.schema = schema
self.port = port
- self.extra = extra
+ self.extra = self.validate_extra(extra)
+
+ @staticmethod
+ def validate_extra(extra: str):
+ """
+ `extra` parameter is a JSON encoded object. This methods validates that the data
+ adheres to this specification.
+
+ :param extra: The extra section of the .
+ :type extra: str
+
+ :return str
+ """
+ try:
+ json.loads(extra)
Review comment:
@mik-laj has suggested we should forbid the case where `json.loads(extra)` produces a list or primitive
anyone else agree with that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] davlum commented on pull request #15146: Validate that the extra parameter is parseable as JSON
Posted by GitBox <gi...@apache.org>.
davlum commented on pull request #15146:
URL: https://github.com/apache/airflow/pull/15146#issuecomment-815075428
@dimberman waddup!
Added tests but tbh I'm also down for what @dstandish suggested. It would be more backwards compat.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman commented on pull request #15146: Validate that the extra parameter is parseable as JSON
Posted by GitBox <gi...@apache.org>.
dimberman commented on pull request #15146:
URL: https://github.com/apache/airflow/pull/15146#issuecomment-813539233
Hey @davlum, been a minute!
Could you add a unit test? After that I'm good to merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish edited a comment on pull request #15146: Validate that the extra parameter is parseable as JSON
Posted by GitBox <gi...@apache.org>.
dstandish edited a comment on pull request #15146:
URL: https://github.com/apache/airflow/pull/15146#issuecomment-813687022
alternatively we could continute to allow users to put arbitrary string in `extra`, but change `extra_dejson` so that it returns `self.extra` in that case (instead of empty dict)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org