You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/03 17:56:32 UTC
[GitHub] [airflow] natanweinberger opened a new pull request #15177: Import connections from a file
natanweinberger opened a new pull request #15177:
URL: https://github.com/apache/airflow/pull/15177
This PR adds the ability to import connections from a file.
This implementation enables basic import functionality, mirroring what already exists for variables. There's room for additional features in future (as proposed in #9907 discussion), but I believe it's reasonable to add those extra features as incremental improvements in a future PR. For example:
- handling conflict disposition consistently for variables and connections
- accepting input from stdin
Currently, in the event of a conflict, the connection to be imported is skipped and a message is printed to stdout indicating to the user that there's a conflict. Any connections without conflicts are still imported.
closes: #9855
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman merged pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
dimberman merged pull request #15177:
URL: https://github.com/apache/airflow/pull/15177
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] natanweinberger commented on pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
natanweinberger commented on pull request #15177:
URL: https://github.com/apache/airflow/pull/15177#issuecomment-822059520
Hey @mudravrik, I see the issue. Thanks for the heads up, I will fix this and open a new PR shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mudravrik commented on pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
mudravrik commented on pull request #15177:
URL: https://github.com/apache/airflow/pull/15177#issuecomment-822056108
Sorry, @natanweinberger , it must be wrong place to ask such question, but I tried this feature and get following error:
```
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line 91, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/connection_command.py", line 244, in connections_import
_import_helper(args.file)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/connection_command.py", line 272, in _import_helper
key: value for key, value in conn_values.items() if key in allowed_fields
AttributeError: 'Connection' object has no attribute 'items'
```
Am I doing something completly wrong?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15177:
URL: https://github.com/apache/airflow/pull/15177#issuecomment-905751128
'And answering your other question @mm-lehmann - in case you have not noticed the "import" command is counterpart of the "export" command https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html#export which result you can inspect and check the format.
If you really think the format needs to be described - you are absolutely invited to contribute documentation for both export and import after reverse engineering it! I am not kidding. I think this might be valuable contribution and great way to repay for all the countless - often unpaid hours other contributors spent on building airflow (which you can now use for free).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15177:
URL: https://github.com/apache/airflow/pull/15177#issuecomment-905665622
If you really cared to ask this kind of question @MM-Lehmann, then I believe because apparently you wanted to here the answer rather than show your superiority over those who volunteer their time so that you can get great FREE software.
So here it is.
My one answer is - because people are humans and when they try to improve something, they sometimes make mistakes (as opposed to people who do not do do much so they do not do mistakes either).
Or maybe another answer - because you weren't around with helpful comment like that when it was discussed @MM-Lehmann . I am sure if you were around such mistakes would not have happened.
Please next time be sure to follow up the changed PRs and catch such omissions early.
Airflow PRs can be commented and suggested by everyone. Please do so next time before merging!
You are most welcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #15177:
URL: https://github.com/apache/airflow/pull/15177#issuecomment-813537035
Awesome work, congrats on your first merged pull request!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman commented on pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
dimberman commented on pull request #15177:
URL: https://github.com/apache/airflow/pull/15177#issuecomment-813537153
Congrats @natanweinberger !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] MM-Lehmann commented on pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
MM-Lehmann commented on pull request #15177:
URL: https://github.com/apache/airflow/pull/15177#issuecomment-905511912
How did this not get a documentation entry? Any definition of how the json must look like? Even the sample.json mentioned in the tests is not in this PR, is it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #15177:
URL: https://github.com/apache/airflow/pull/15177#issuecomment-812901548
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (flake8, pylint and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/master/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for testing locally, itโs a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better ๐.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #15177: Import connections from a file
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #15177:
URL: https://github.com/apache/airflow/pull/15177#issuecomment-905665622
If you really cared to ask this kind of question @MM-Lehmann, then I believe because apparently you wanted to hear the answer rather than show your superiority over those who volunteer their time so that you can get great FREE software.
So here it is.
My one answer is - because people are humans and when they try to improve something, they sometimes make mistakes (as opposed to people who do not do do much so they do not do mistakes either).
Or maybe another answer - because you weren't around with helpful comment like that when it was discussed @MM-Lehmann . I am sure if you were around such mistakes would not have happened.
Please next time be sure to follow up the changed PRs and catch such omissions early.
Airflow PRs can be commented and suggested by everyone. Please do so next time before merging!
You are most welcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org