You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/05/25 15:08:33 UTC

[GitHub] [airflow] nhojjohn opened a new pull request, #23916: Add support to specify language name in PapermillOperator

nhojjohn opened a new pull request, #23916:
URL: https://github.com/apache/airflow/pull/23916

   <!--
   Thank you for contributing! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   In case of existing issue, reference it using one of the following:
   
   closes: #ISSUE
   related: #ISSUE
   
   How to write a good git commit message:
   http://chris.beams.io/posts/git-commit/
   -->
   This PR adds a language_name field to PapermillOperator so we can specify the programming language the notebook uses.
   If language_name is specified, papermill ignores language_info.name in the notebook document metadata (if any).
   
   This is based entirely on a previous merged PR which added kernel_name https://github.com/apache/airflow/pull/20035 option.  This feature would be useful to allow the Operator to be used with any ipynb files which having missing language metadata which the python papermill package requires.
   ---
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] boring-cyborg[bot] commented on pull request #23916: Add support to specify language name in PapermillOperator

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on PR #23916:
URL: https://github.com/apache/airflow/pull/23916#issuecomment-1137409267

   Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (flake8, mypy and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
   - Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for testing locally, itโ€™s a heavy docker but it ships with a working Airflow and a lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
   - Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
   - Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
   Apache Airflow is a community-driven project and together we are making it better ๐Ÿš€.
   In case of doubts contact the developers at:
   Mailing List: dev@airflow.apache.org
   Slack: https://s.apache.org/airflow-slack
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] boring-cyborg[bot] commented on pull request #23916: Add support to specify language name in PapermillOperator

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on PR #23916:
URL: https://github.com/apache/airflow/pull/23916#issuecomment-1139990228

   Awesome work, congrats on your first merged pull request!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] uranusjr commented on a diff in pull request #23916: Add support to specify language name in PapermillOperator

Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #23916:
URL: https://github.com/apache/airflow/pull/23916#discussion_r881784882


##########
tests/providers/papermill/operators/test_papermill.py:
##########
@@ -75,3 +79,4 @@ def test_render_template(self):
         assert '/tmp/out-test_render_template.ipynb' == getattr(operator, 'output_nb')
         assert {"msgs": "dag id is test_render_template!"} == getattr(operator, 'parameters')
         assert "python3" == getattr(operator, 'kernel_name')
+        assert "python" == getattr(operator, 'language_name')

Review Comment:
   Could you change these asserts to use simply attribute access? e.g. `assert "python" == operator.language_name`. These `getattr` calls are non-sensical.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] github-actions[bot] commented on pull request #23916: Add support to specify language name in PapermillOperator

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23916:
URL: https://github.com/apache/airflow/pull/23916#issuecomment-1137441054

   The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] nhojjohn commented on a diff in pull request #23916: Add support to specify language name in PapermillOperator

Posted by GitBox <gi...@apache.org>.
nhojjohn commented on code in PR #23916:
URL: https://github.com/apache/airflow/pull/23916#discussion_r881799239


##########
tests/providers/papermill/operators/test_papermill.py:
##########
@@ -75,3 +79,4 @@ def test_render_template(self):
         assert '/tmp/out-test_render_template.ipynb' == getattr(operator, 'output_nb')
         assert {"msgs": "dag id is test_render_template!"} == getattr(operator, 'parameters')
         assert "python3" == getattr(operator, 'kernel_name')
+        assert "python" == getattr(operator, 'language_name')

Review Comment:
   I've done this for all 5



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] josh-fell merged pull request #23916: Add support to specify language name in PapermillOperator

Posted by GitBox <gi...@apache.org>.
josh-fell merged PR #23916:
URL: https://github.com/apache/airflow/pull/23916


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org