You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/06/14 17:25:33 UTC
[GitHub] [airflow] bryce-lewis opened a new pull request #16436: Add ElasticSearch Connection Doc
bryce-lewis opened a new pull request #16436:
URL: https://github.com/apache/airflow/pull/16436
Added a connection doc for the ElasticSearch connection. Added links inside the ElasticSearch module doc strings for user to have access to the ElasticSearch connection doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #16436: Add ElasticSearch Connection Doc
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #16436:
URL: https://github.com/apache/airflow/pull/16436#issuecomment-861039107
Pre-commit checks are failing, can you install and re-run pre-commit checks on your machine please:
https://github.com/apache/airflow/pull/16436/checks?check_run_id=2822574119#step:10:273
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #16436: Add ElasticSearch Connection Doc
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #16436:
URL: https://github.com/apache/airflow/pull/16436
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #16436: Add ElasticSearch Connection Doc
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #16436:
URL: https://github.com/apache/airflow/pull/16436#discussion_r651323405
##########
File path: docs/apache-airflow-providers-elasticsearch/connections/elasticsearch.rst
##########
@@ -0,0 +1,74 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+
+.. _howto/connection:elasticsearch:
+
+ElasticSearch Connection
+========================
+
+The ElasticSearch connection that enables ElasticSearch integrations.
+
+Authenticating to ElasticSearch
+-------------------------------
+
+Authenticate with the `ElasticSearch DBAPI
+<https://pypi.org/project/elasticsearch-dbapi/>`_
+
+Default Connection IDs
+----------------------
+
+Some hooks and operators related to ElasticSearch use elasticsearch_default by default.
+
+Configuring the Connection
+--------------------------
+
+User
+ Specify the login used for the initial connection
+
+Password
+ Specify the Elasticsearch API key used for the `initial connection
+ <https://www.elastic.co/guide/en/cloud/current/ec-api-authentication.html#ec-api-authentication>`_
+
+Host
+ Specify the Elasticsearch host used for the initial connection
+
+Port
+ Specify the Elasticsearch port for the initial connection
+
+Scheme
+ Specify the schema for the Elasticsearch API. `http` by default
+
+Extra (Optional)
+ Specify the extra parameters (as json dictionary) that can be used in Azure connection.
+ The following parameters are all optional:
+
+ * ``http_compress``: specify whether or not to use ``http_compress``. False by default.
+ * ``timeout``: specify the time frame of the ``timeout``. False by default.
+
+When specifying the connection in environment variable you should specify
+it using URI syntax.
+
+Note that all components of the URI should be URL-encoded.
+
+For example:
+
+.. code-block:: bash
+
+ export AIRFLOW_CONN_ELASTICSEARCH_DEFAULT='elasticsearch://elasticsearchlogin:elasticsearchpassword@elastic.co:80/http'
+
Review comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #16436: Add ElasticSearch Connection Doc
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #16436:
URL: https://github.com/apache/airflow/pull/16436#discussion_r651323643
##########
File path: docs/apache-airflow-providers-elasticsearch/connections/elasticsearch.rst
##########
@@ -0,0 +1,74 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+
+.. _howto/connection:elasticsearch:
+
+ElasticSearch Connection
+========================
+
+The ElasticSearch connection that enables ElasticSearch integrations.
+
+Authenticating to ElasticSearch
+-------------------------------
+
+Authenticate with the `ElasticSearch DBAPI
+<https://pypi.org/project/elasticsearch-dbapi/>`_
+
+Default Connection IDs
+----------------------
+
+Some hooks and operators related to ElasticSearch use elasticsearch_default by default.
+
+Configuring the Connection
+--------------------------
+
+User
+ Specify the login used for the initial connection
+
+Password
Review comment:
```suggestion
User
Specify the login used for the initial connection
Password
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #16436: Add ElasticSearch Connection Doc
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #16436:
URL: https://github.com/apache/airflow/pull/16436#discussion_r651323745
##########
File path: airflow/providers/elasticsearch/hooks/elasticsearch.py
##########
@@ -30,7 +30,8 @@ class ElasticsearchHook(DbApiHook):
This hook uses the Elasticsearch conn_id.
- :param elasticsearch_conn_id: The Airflow connection used for Elasticsearch credentials.
+ :param elasticsearch_conn_id: The :ref:`ElasticSearch connection id <howto/connection:elasticsearch>`
+ used for Elasticsearch credentials.
Review comment:
```suggestion
:param elasticsearch_conn_id: The :ref:`ElasticSearch connection id <howto/connection:elasticsearch>`
used for Elasticsearch credentials.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #16436: Add ElasticSearch Connection Doc
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #16436:
URL: https://github.com/apache/airflow/pull/16436#issuecomment-861047934
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #16436: Add ElasticSearch Connection Doc
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #16436:
URL: https://github.com/apache/airflow/pull/16436#discussion_r651323523
##########
File path: docs/apache-airflow-providers-elasticsearch/connections/elasticsearch.rst
##########
@@ -0,0 +1,74 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+
+.. _howto/connection:elasticsearch:
+
+ElasticSearch Connection
+========================
+
+The ElasticSearch connection that enables ElasticSearch integrations.
+
+Authenticating to ElasticSearch
+-------------------------------
+
+Authenticate with the `ElasticSearch DBAPI
+<https://pypi.org/project/elasticsearch-dbapi/>`_
+
+Default Connection IDs
+----------------------
+
+Some hooks and operators related to ElasticSearch use elasticsearch_default by default.
+
+Configuring the Connection
+--------------------------
+
+User
+ Specify the login used for the initial connection
+
+Password
+ Specify the Elasticsearch API key used for the `initial connection
+ <https://www.elastic.co/guide/en/cloud/current/ec-api-authentication.html#ec-api-authentication>`_
+
+Host
+ Specify the Elasticsearch host used for the initial connection
+
+Port
+ Specify the Elasticsearch port for the initial connection
+
+Scheme
+ Specify the schema for the Elasticsearch API. `http` by default
+
+Extra (Optional)
+ Specify the extra parameters (as json dictionary) that can be used in Azure connection.
+ The following parameters are all optional:
+
Review comment:
```suggestion
Specify the extra parameters (as json dictionary) that can be used in Azure connection.
The following parameters are all optional:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org