You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2021/08/23 21:57:33 UTC
[airflow] 01/02: Update description about the new
``connection-types`` provider meta-data
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit be75dcd39cd10264048c86e74110365bd5daf8b7
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Sat Aug 21 12:12:39 2021 +0200
Update description about the new ``connection-types`` provider meta-data
The ``hook-class-names`` provider's meta-data property has been deprecated and
is now replaced by ``connection-types`` property. This documents the
change.
---
airflow/providers/airbyte/provider.yaml | 2 +-
airflow/providers/alibaba/provider.yaml | 2 +-
airflow/providers/amazon/provider.yaml | 2 +-
airflow/providers/apache/cassandra/provider.yaml | 2 +-
airflow/providers/apache/drill/provider.yaml | 2 +-
airflow/providers/apache/druid/provider.yaml | 2 +-
airflow/providers/apache/hdfs/provider.yaml | 2 +-
airflow/providers/apache/hive/provider.yaml | 2 +-
airflow/providers/apache/livy/provider.yaml | 2 +-
airflow/providers/apache/pig/provider.yaml | 2 +-
airflow/providers/apache/spark/provider.yaml | 2 +-
airflow/providers/apache/sqoop/provider.yaml | 2 +-
airflow/providers/asana/provider.yaml | 2 +-
airflow/providers/cloudant/provider.yaml | 2 +-
airflow/providers/cncf/kubernetes/provider.yaml | 2 +-
airflow/providers/databricks/provider.yaml | 2 +-
airflow/providers/dingding/provider.yaml | 2 +-
airflow/providers/discord/provider.yaml | 2 +-
airflow/providers/docker/provider.yaml | 2 +-
airflow/providers/elasticsearch/provider.yaml | 2 +-
airflow/providers/exasol/provider.yaml | 2 +-
airflow/providers/facebook/provider.yaml | 2 +-
airflow/providers/ftp/provider.yaml | 2 +-
airflow/providers/google/provider.yaml | 2 +-
airflow/providers/grpc/provider.yaml | 2 +-
airflow/providers/hashicorp/provider.yaml | 2 +-
airflow/providers/http/provider.yaml | 2 +-
airflow/providers/imap/provider.yaml | 2 +-
airflow/providers/jdbc/provider.yaml | 2 +-
airflow/providers/jenkins/provider.yaml | 2 +-
airflow/providers/jira/provider.yaml | 2 +-
airflow/providers/microsoft/azure/provider.yaml | 2 +-
airflow/providers/microsoft/mssql/provider.yaml | 2 +-
airflow/providers/mongo/provider.yaml | 2 +-
airflow/providers/mysql/provider.yaml | 2 +-
airflow/providers/neo4j/provider.yaml | 2 +-
airflow/providers/odbc/provider.yaml | 2 +-
airflow/providers/opsgenie/provider.yaml | 2 +-
airflow/providers/oracle/provider.yaml | 2 +-
airflow/providers/postgres/provider.yaml | 2 +-
airflow/providers/presto/provider.yaml | 2 +-
airflow/providers/qubole/provider.yaml | 2 +-
airflow/providers/redis/provider.yaml | 2 +-
airflow/providers/salesforce/provider.yaml | 2 +-
airflow/providers/samba/provider.yaml | 2 +-
airflow/providers/segment/provider.yaml | 2 +-
airflow/providers/sftp/provider.yaml | 2 +-
airflow/providers/slack/provider.yaml | 2 +-
airflow/providers/snowflake/provider.yaml | 2 +-
airflow/providers/sqlite/provider.yaml | 2 +-
airflow/providers/ssh/provider.yaml | 2 +-
airflow/providers/tableau/provider.yaml | 2 +-
airflow/providers/trino/provider.yaml | 2 +-
airflow/providers/vertica/provider.yaml | 2 +-
airflow/providers/yandex/provider.yaml | 2 +-
.../howto/create-update-providers.rst | 17 ++++++++++++++---
docs/apache-airflow-providers/index.rst | 17 +++++++++++++----
docs/apache-airflow/howto/connection.rst | 12 ++++++++++--
58 files changed, 92 insertions(+), 64 deletions(-)
diff --git a/airflow/providers/airbyte/provider.yaml b/airflow/providers/airbyte/provider.yaml
index e082e66..4d2778d 100644
--- a/airflow/providers/airbyte/provider.yaml
+++ b/airflow/providers/airbyte/provider.yaml
@@ -52,7 +52,7 @@ sensors:
python-modules:
- airflow.providers.airbyte.sensors.airbyte
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.airbyte.hooks.airbyte.AirbyteHook
connection-types:
diff --git a/airflow/providers/alibaba/provider.yaml b/airflow/providers/alibaba/provider.yaml
index 14fe683..ed82dff 100644
--- a/airflow/providers/alibaba/provider.yaml
+++ b/airflow/providers/alibaba/provider.yaml
@@ -50,7 +50,7 @@ hooks:
python-modules:
- airflow.providers.alibaba.cloud.hooks.oss
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.alibaba.cloud.hooks.oss.OSSHook
connection-types:
diff --git a/airflow/providers/amazon/provider.yaml b/airflow/providers/amazon/provider.yaml
index 56b6621..b82caf8 100644
--- a/airflow/providers/amazon/provider.yaml
+++ b/airflow/providers/amazon/provider.yaml
@@ -410,7 +410,7 @@ transfers:
how-to-guide: /docs/apache-airflow-providers-amazon/operators/salesforce_to_s3.rst
python-module: airflow.providers.amazon.aws.transfers.salesforce_to_s3
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.amazon.aws.hooks.s3.S3Hook
- airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook
- airflow.providers.amazon.aws.hooks.emr.EmrHook
diff --git a/airflow/providers/apache/cassandra/provider.yaml b/airflow/providers/apache/cassandra/provider.yaml
index 9e6bcef..61fcfe7 100644
--- a/airflow/providers/apache/cassandra/provider.yaml
+++ b/airflow/providers/apache/cassandra/provider.yaml
@@ -48,7 +48,7 @@ hooks:
python-modules:
- airflow.providers.apache.cassandra.hooks.cassandra
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.apache.cassandra.hooks.cassandra.CassandraHook
connection-types:
diff --git a/airflow/providers/apache/drill/provider.yaml b/airflow/providers/apache/drill/provider.yaml
index 213a509..2bfac05 100644
--- a/airflow/providers/apache/drill/provider.yaml
+++ b/airflow/providers/apache/drill/provider.yaml
@@ -45,7 +45,7 @@ hooks:
python-modules:
- airflow.providers.apache.drill.hooks.drill
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.apache.drill.hooks.drill.DrillHook
connection-types:
diff --git a/airflow/providers/apache/druid/provider.yaml b/airflow/providers/apache/druid/provider.yaml
index 9ec4af7..7492e9a 100644
--- a/airflow/providers/apache/druid/provider.yaml
+++ b/airflow/providers/apache/druid/provider.yaml
@@ -48,7 +48,7 @@ hooks:
python-modules:
- airflow.providers.apache.druid.hooks.druid
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.apache.druid.hooks.druid.DruidDbApiHook
connection-types:
diff --git a/airflow/providers/apache/hdfs/provider.yaml b/airflow/providers/apache/hdfs/provider.yaml
index 87a0820..8228916 100644
--- a/airflow/providers/apache/hdfs/provider.yaml
+++ b/airflow/providers/apache/hdfs/provider.yaml
@@ -58,7 +58,7 @@ hooks:
python-modules:
- airflow.providers.apache.hdfs.hooks.webhdfs
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.apache.hdfs.hooks.hdfs.HDFSHook
connection-types:
diff --git a/airflow/providers/apache/hive/provider.yaml b/airflow/providers/apache/hive/provider.yaml
index f8610f5..8b7673d 100644
--- a/airflow/providers/apache/hive/provider.yaml
+++ b/airflow/providers/apache/hive/provider.yaml
@@ -79,7 +79,7 @@ transfers:
target-integration-name: Apache Hive
python-module: airflow.providers.apache.hive.transfers.mssql_to_hive
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.apache.hive.hooks.hive.HiveCliHook
- airflow.providers.apache.hive.hooks.hive.HiveServer2Hook
- airflow.providers.apache.hive.hooks.hive.HiveMetastoreHook
diff --git a/airflow/providers/apache/livy/provider.yaml b/airflow/providers/apache/livy/provider.yaml
index 28e4d73..2ed65c1 100644
--- a/airflow/providers/apache/livy/provider.yaml
+++ b/airflow/providers/apache/livy/provider.yaml
@@ -51,7 +51,7 @@ hooks:
python-modules:
- airflow.providers.apache.livy.hooks.livy
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.apache.livy.hooks.livy.LivyHook
connection-types:
diff --git a/airflow/providers/apache/pig/provider.yaml b/airflow/providers/apache/pig/provider.yaml
index e513176..364588a 100644
--- a/airflow/providers/apache/pig/provider.yaml
+++ b/airflow/providers/apache/pig/provider.yaml
@@ -45,7 +45,7 @@ hooks:
python-modules:
- airflow.providers.apache.pig.hooks.pig
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.apache.pig.hooks.pig.PigCliHook
connection-types:
diff --git a/airflow/providers/apache/spark/provider.yaml b/airflow/providers/apache/spark/provider.yaml
index 7964c6b..9f212d4 100644
--- a/airflow/providers/apache/spark/provider.yaml
+++ b/airflow/providers/apache/spark/provider.yaml
@@ -54,7 +54,7 @@ hooks:
- airflow.providers.apache.spark.hooks.spark_sql
- airflow.providers.apache.spark.hooks.spark_submit
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.apache.spark.hooks.spark_jdbc.SparkJDBCHook
- airflow.providers.apache.spark.hooks.spark_sql.SparkSqlHook
- airflow.providers.apache.spark.hooks.spark_submit.SparkSubmitHook
diff --git a/airflow/providers/apache/sqoop/provider.yaml b/airflow/providers/apache/sqoop/provider.yaml
index bf34cdb..7eb40af 100644
--- a/airflow/providers/apache/sqoop/provider.yaml
+++ b/airflow/providers/apache/sqoop/provider.yaml
@@ -46,7 +46,7 @@ hooks:
python-modules:
- airflow.providers.apache.sqoop.hooks.sqoop
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.apache.sqoop.hooks.sqoop.SqoopHook
connection-types:
diff --git a/airflow/providers/asana/provider.yaml b/airflow/providers/asana/provider.yaml
index bbd9d33..d3a31e7 100644
--- a/airflow/providers/asana/provider.yaml
+++ b/airflow/providers/asana/provider.yaml
@@ -44,7 +44,7 @@ hooks:
python-modules:
- airflow.providers.asana.hooks.asana
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.asana.hooks.asana.AsanaHook
connection-types:
diff --git a/airflow/providers/cloudant/provider.yaml b/airflow/providers/cloudant/provider.yaml
index 82e40b5..7834024 100644
--- a/airflow/providers/cloudant/provider.yaml
+++ b/airflow/providers/cloudant/provider.yaml
@@ -40,7 +40,7 @@ hooks:
python-modules:
- airflow.providers.cloudant.hooks.cloudant
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.cloudant.hooks.cloudant.CloudantHook
connection-types:
diff --git a/airflow/providers/cncf/kubernetes/provider.yaml b/airflow/providers/cncf/kubernetes/provider.yaml
index 242d256..957f7db 100644
--- a/airflow/providers/cncf/kubernetes/provider.yaml
+++ b/airflow/providers/cncf/kubernetes/provider.yaml
@@ -61,7 +61,7 @@ hooks:
python-modules:
- airflow.providers.cncf.kubernetes.hooks.kubernetes
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.cncf.kubernetes.hooks.kubernetes.KubernetesHook
connection-types:
diff --git a/airflow/providers/databricks/provider.yaml b/airflow/providers/databricks/provider.yaml
index 64f2a1f..3ce4f2f 100644
--- a/airflow/providers/databricks/provider.yaml
+++ b/airflow/providers/databricks/provider.yaml
@@ -46,7 +46,7 @@ hooks:
python-modules:
- airflow.providers.databricks.hooks.databricks
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.databricks.hooks.databricks.DatabricksHook
connection-types:
diff --git a/airflow/providers/dingding/provider.yaml b/airflow/providers/dingding/provider.yaml
index 20743a0..b130d8b 100644
--- a/airflow/providers/dingding/provider.yaml
+++ b/airflow/providers/dingding/provider.yaml
@@ -48,7 +48,7 @@ hooks:
python-modules:
- airflow.providers.dingding.hooks.dingding
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.dingding.hooks.dingding.DingdingHook
connection-types:
diff --git a/airflow/providers/discord/provider.yaml b/airflow/providers/discord/provider.yaml
index e9f6b0a..c1c7c7f 100644
--- a/airflow/providers/discord/provider.yaml
+++ b/airflow/providers/discord/provider.yaml
@@ -45,7 +45,7 @@ hooks:
python-modules:
- airflow.providers.discord.hooks.discord_webhook
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.discord.hooks.discord_webhook.DiscordWebhookHook
connection-types:
diff --git a/airflow/providers/docker/provider.yaml b/airflow/providers/docker/provider.yaml
index 91fcd96..b8956d8 100644
--- a/airflow/providers/docker/provider.yaml
+++ b/airflow/providers/docker/provider.yaml
@@ -56,7 +56,7 @@ hooks:
python-modules:
- airflow.providers.docker.hooks.docker
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.docker.hooks.docker.DockerHook
connection-types:
diff --git a/airflow/providers/elasticsearch/provider.yaml b/airflow/providers/elasticsearch/provider.yaml
index 8a8134d..b24bdbe 100644
--- a/airflow/providers/elasticsearch/provider.yaml
+++ b/airflow/providers/elasticsearch/provider.yaml
@@ -44,7 +44,7 @@ hooks:
python-modules:
- airflow.providers.elasticsearch.hooks.elasticsearch
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.elasticsearch.hooks.elasticsearch.ElasticsearchHook
connection-types:
diff --git a/airflow/providers/exasol/provider.yaml b/airflow/providers/exasol/provider.yaml
index e92747f..552a459 100644
--- a/airflow/providers/exasol/provider.yaml
+++ b/airflow/providers/exasol/provider.yaml
@@ -46,7 +46,7 @@ hooks:
python-modules:
- airflow.providers.exasol.hooks.exasol
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.exasol.hooks.exasol.ExasolHook
connection-types:
diff --git a/airflow/providers/facebook/provider.yaml b/airflow/providers/facebook/provider.yaml
index 2f6943f..30f18f4 100644
--- a/airflow/providers/facebook/provider.yaml
+++ b/airflow/providers/facebook/provider.yaml
@@ -41,7 +41,7 @@ hooks:
python-modules:
- airflow.providers.facebook.ads.hooks.ads
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.facebook.ads.hooks.ads.FacebookAdsReportingHook
connection-types:
diff --git a/airflow/providers/ftp/provider.yaml b/airflow/providers/ftp/provider.yaml
index 85bb289..bffb7de 100644
--- a/airflow/providers/ftp/provider.yaml
+++ b/airflow/providers/ftp/provider.yaml
@@ -43,7 +43,7 @@ hooks:
python-modules:
- airflow.providers.ftp.hooks.ftp
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.ftp.hooks.ftp.FTPHook
connection-types:
diff --git a/airflow/providers/google/provider.yaml b/airflow/providers/google/provider.yaml
index 2b978d9..288f63b 100644
--- a/airflow/providers/google/provider.yaml
+++ b/airflow/providers/google/provider.yaml
@@ -738,7 +738,7 @@ transfers:
target-integration-name: Google Cloud Storage (GCS)
python-module: airflow.providers.google.ads.transfers.ads_to_gcs
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.google.common.hooks.base_google.GoogleBaseHook
- airflow.providers.google.cloud.hooks.dataprep.GoogleDataprepHook
- airflow.providers.google.cloud.hooks.cloud_sql.CloudSQLHook
diff --git a/airflow/providers/grpc/provider.yaml b/airflow/providers/grpc/provider.yaml
index e9b1294..366a6f7 100644
--- a/airflow/providers/grpc/provider.yaml
+++ b/airflow/providers/grpc/provider.yaml
@@ -45,7 +45,7 @@ hooks:
python-modules:
- airflow.providers.grpc.hooks.grpc
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.grpc.hooks.grpc.GrpcHook
connection-types:
diff --git a/airflow/providers/hashicorp/provider.yaml b/airflow/providers/hashicorp/provider.yaml
index 6d022e7..44eb4a7 100644
--- a/airflow/providers/hashicorp/provider.yaml
+++ b/airflow/providers/hashicorp/provider.yaml
@@ -41,7 +41,7 @@ hooks:
python-modules:
- airflow.providers.hashicorp.hooks.vault
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.hashicorp.hooks.vault.VaultHook
connection-types:
diff --git a/airflow/providers/http/provider.yaml b/airflow/providers/http/provider.yaml
index cc3c081..3c3918b 100644
--- a/airflow/providers/http/provider.yaml
+++ b/airflow/providers/http/provider.yaml
@@ -52,7 +52,7 @@ hooks:
python-modules:
- airflow.providers.http.hooks.http
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.http.hooks.http.HttpHook
connection-types:
diff --git a/airflow/providers/imap/provider.yaml b/airflow/providers/imap/provider.yaml
index 3cd6217..e567602 100644
--- a/airflow/providers/imap/provider.yaml
+++ b/airflow/providers/imap/provider.yaml
@@ -42,7 +42,7 @@ hooks:
python-modules:
- airflow.providers.imap.hooks.imap
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.imap.hooks.imap.ImapHook
connection-types:
diff --git a/airflow/providers/jdbc/provider.yaml b/airflow/providers/jdbc/provider.yaml
index d4e4e62..2e87ceb 100644
--- a/airflow/providers/jdbc/provider.yaml
+++ b/airflow/providers/jdbc/provider.yaml
@@ -46,7 +46,7 @@ hooks:
python-modules:
- airflow.providers.jdbc.hooks.jdbc
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.jdbc.hooks.jdbc.JdbcHook
connection-types:
diff --git a/airflow/providers/jenkins/provider.yaml b/airflow/providers/jenkins/provider.yaml
index d4d480c..05ce6bd 100644
--- a/airflow/providers/jenkins/provider.yaml
+++ b/airflow/providers/jenkins/provider.yaml
@@ -46,7 +46,7 @@ hooks:
python-modules:
- airflow.providers.jenkins.hooks.jenkins
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.jenkins.hooks.jenkins.JenkinsHook
connection-types:
diff --git a/airflow/providers/jira/provider.yaml b/airflow/providers/jira/provider.yaml
index a7073cc..4dd4276 100644
--- a/airflow/providers/jira/provider.yaml
+++ b/airflow/providers/jira/provider.yaml
@@ -51,7 +51,7 @@ hooks:
python-modules:
- airflow.providers.jira.hooks.jira
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.jira.hooks.jira.JiraHook
connection-types:
diff --git a/airflow/providers/microsoft/azure/provider.yaml b/airflow/providers/microsoft/azure/provider.yaml
index 6547641..230eb4a 100644
--- a/airflow/providers/microsoft/azure/provider.yaml
+++ b/airflow/providers/microsoft/azure/provider.yaml
@@ -153,7 +153,7 @@ transfers:
how-to-guide: /docs/apache-airflow-providers-microsoft-azure/operators/azure_blob_to_gcs.rst
python-module: airflow.providers.microsoft.azure.transfers.azure_blob_to_gcs
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.microsoft.azure.hooks.base_azure.AzureBaseHook
- airflow.providers.microsoft.azure.hooks.adx.AzureDataExplorerHook
- airflow.providers.microsoft.azure.hooks.azure_batch.AzureBatchHook
diff --git a/airflow/providers/microsoft/mssql/provider.yaml b/airflow/providers/microsoft/mssql/provider.yaml
index fc8cfc9..0b1414e 100644
--- a/airflow/providers/microsoft/mssql/provider.yaml
+++ b/airflow/providers/microsoft/mssql/provider.yaml
@@ -46,7 +46,7 @@ hooks:
python-modules:
- airflow.providers.microsoft.mssql.hooks.mssql
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.microsoft.mssql.hooks.mssql.MsSqlHook
connection-types:
diff --git a/airflow/providers/mongo/provider.yaml b/airflow/providers/mongo/provider.yaml
index 3d83d37..682317c 100644
--- a/airflow/providers/mongo/provider.yaml
+++ b/airflow/providers/mongo/provider.yaml
@@ -44,7 +44,7 @@ hooks:
python-modules:
- airflow.providers.mongo.hooks.mongo
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.mongo.hooks.mongo.MongoHook
connection-types:
diff --git a/airflow/providers/mysql/provider.yaml b/airflow/providers/mysql/provider.yaml
index cd5be3a..cb719a9 100644
--- a/airflow/providers/mysql/provider.yaml
+++ b/airflow/providers/mysql/provider.yaml
@@ -65,7 +65,7 @@ transfers:
target-integration-name: MySQL
python-module: airflow.providers.mysql.transfers.trino_to_mysql
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.mysql.hooks.mysql.MySqlHook
connection-types:
diff --git a/airflow/providers/neo4j/provider.yaml b/airflow/providers/neo4j/provider.yaml
index a3416f5..d62489e 100644
--- a/airflow/providers/neo4j/provider.yaml
+++ b/airflow/providers/neo4j/provider.yaml
@@ -46,7 +46,7 @@ hooks:
python-modules:
- airflow.providers.neo4j.hooks.neo4j
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.neo4j.hooks.neo4j.Neo4jHook
connection-types:
diff --git a/airflow/providers/odbc/provider.yaml b/airflow/providers/odbc/provider.yaml
index 84fcf93..e3cfaa6 100644
--- a/airflow/providers/odbc/provider.yaml
+++ b/airflow/providers/odbc/provider.yaml
@@ -40,7 +40,7 @@ hooks:
python-modules:
- airflow.providers.odbc.hooks.odbc
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.odbc.hooks.odbc.OdbcHook
connection-types:
diff --git a/airflow/providers/opsgenie/provider.yaml b/airflow/providers/opsgenie/provider.yaml
index 313f5ac..8693ae9 100644
--- a/airflow/providers/opsgenie/provider.yaml
+++ b/airflow/providers/opsgenie/provider.yaml
@@ -46,7 +46,7 @@ hooks:
python-modules:
- airflow.providers.opsgenie.hooks.opsgenie_alert
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.opsgenie.hooks.opsgenie_alert.OpsgenieAlertHook
connection-types:
diff --git a/airflow/providers/oracle/provider.yaml b/airflow/providers/oracle/provider.yaml
index 8452f7a..02bb455 100644
--- a/airflow/providers/oracle/provider.yaml
+++ b/airflow/providers/oracle/provider.yaml
@@ -51,7 +51,7 @@ transfers:
target-integration-name: Oracle
python-module: airflow.providers.oracle.transfers.oracle_to_oracle
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.oracle.hooks.oracle.OracleHook
connection-types:
diff --git a/airflow/providers/postgres/provider.yaml b/airflow/providers/postgres/provider.yaml
index b320a77..2a33635 100644
--- a/airflow/providers/postgres/provider.yaml
+++ b/airflow/providers/postgres/provider.yaml
@@ -49,7 +49,7 @@ hooks:
python-modules:
- airflow.providers.postgres.hooks.postgres
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.postgres.hooks.postgres.PostgresHook
connection-types:
diff --git a/airflow/providers/presto/provider.yaml b/airflow/providers/presto/provider.yaml
index 891a68b..26b7981 100644
--- a/airflow/providers/presto/provider.yaml
+++ b/airflow/providers/presto/provider.yaml
@@ -41,7 +41,7 @@ hooks:
python-modules:
- airflow.providers.presto.hooks.presto
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.presto.hooks.presto.PrestoHook
connection-types:
diff --git a/airflow/providers/qubole/provider.yaml b/airflow/providers/qubole/provider.yaml
index e5b86fc..2246b39 100644
--- a/airflow/providers/qubole/provider.yaml
+++ b/airflow/providers/qubole/provider.yaml
@@ -53,7 +53,7 @@ hooks:
- airflow.providers.qubole.hooks.qubole
- airflow.providers.qubole.hooks.qubole_check
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.qubole.hooks.qubole.QuboleHook
connection-types:
diff --git a/airflow/providers/redis/provider.yaml b/airflow/providers/redis/provider.yaml
index 96b103e..965db20 100644
--- a/airflow/providers/redis/provider.yaml
+++ b/airflow/providers/redis/provider.yaml
@@ -51,7 +51,7 @@ hooks:
python-modules:
- airflow.providers.redis.hooks.redis
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.redis.hooks.redis.RedisHook
connection-types:
diff --git a/airflow/providers/salesforce/provider.yaml b/airflow/providers/salesforce/provider.yaml
index e99beb9..650f9f2 100644
--- a/airflow/providers/salesforce/provider.yaml
+++ b/airflow/providers/salesforce/provider.yaml
@@ -55,7 +55,7 @@ hooks:
python-modules:
- airflow.providers.salesforce.hooks.salesforce
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.salesforce.hooks.salesforce.SalesforceHook
connection-types:
diff --git a/airflow/providers/samba/provider.yaml b/airflow/providers/samba/provider.yaml
index c02987b..af50853 100644
--- a/airflow/providers/samba/provider.yaml
+++ b/airflow/providers/samba/provider.yaml
@@ -40,7 +40,7 @@ hooks:
python-modules:
- airflow.providers.samba.hooks.samba
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.samba.hooks.samba.SambaHook
connection-types:
diff --git a/airflow/providers/segment/provider.yaml b/airflow/providers/segment/provider.yaml
index 6432f6f..fa9bb4a 100644
--- a/airflow/providers/segment/provider.yaml
+++ b/airflow/providers/segment/provider.yaml
@@ -45,7 +45,7 @@ hooks:
python-modules:
- airflow.providers.segment.hooks.segment
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.segment.hooks.segment.SegmentHook
connection-types:
diff --git a/airflow/providers/sftp/provider.yaml b/airflow/providers/sftp/provider.yaml
index 7e44b6b..6540b82 100644
--- a/airflow/providers/sftp/provider.yaml
+++ b/airflow/providers/sftp/provider.yaml
@@ -54,7 +54,7 @@ hooks:
python-modules:
- airflow.providers.sftp.hooks.sftp
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.sftp.hooks.sftp.SFTPHook
connection-types:
diff --git a/airflow/providers/slack/provider.yaml b/airflow/providers/slack/provider.yaml
index 7c3f5d6..9742ce7 100644
--- a/airflow/providers/slack/provider.yaml
+++ b/airflow/providers/slack/provider.yaml
@@ -48,7 +48,7 @@ hooks:
- airflow.providers.slack.hooks.slack
- airflow.providers.slack.hooks.slack_webhook
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.slack.hooks.slack_webhook.SlackWebhookHook
connection-types:
diff --git a/airflow/providers/snowflake/provider.yaml b/airflow/providers/snowflake/provider.yaml
index e100ae5..7ac22e1 100644
--- a/airflow/providers/snowflake/provider.yaml
+++ b/airflow/providers/snowflake/provider.yaml
@@ -61,7 +61,7 @@ transfers:
python-module: airflow.providers.snowflake.transfers.snowflake_to_slack
how-to-guide: /docs/apache-airflow-providers-snowflake/operators/snowflake_to_slack.rst
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.snowflake.hooks.snowflake.SnowflakeHook
connection-types:
diff --git a/airflow/providers/sqlite/provider.yaml b/airflow/providers/sqlite/provider.yaml
index 5374973..e169a72 100644
--- a/airflow/providers/sqlite/provider.yaml
+++ b/airflow/providers/sqlite/provider.yaml
@@ -46,7 +46,7 @@ hooks:
python-modules:
- airflow.providers.sqlite.hooks.sqlite
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.sqlite.hooks.sqlite.SqliteHook
connection-types:
diff --git a/airflow/providers/ssh/provider.yaml b/airflow/providers/ssh/provider.yaml
index 3ba53eb..ab76d2b 100644
--- a/airflow/providers/ssh/provider.yaml
+++ b/airflow/providers/ssh/provider.yaml
@@ -48,7 +48,7 @@ hooks:
python-modules:
- airflow.providers.ssh.hooks.ssh
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.ssh.hooks.ssh.SSHHook
connection-types:
diff --git a/airflow/providers/tableau/provider.yaml b/airflow/providers/tableau/provider.yaml
index 6902ec8..949286e 100644
--- a/airflow/providers/tableau/provider.yaml
+++ b/airflow/providers/tableau/provider.yaml
@@ -53,7 +53,7 @@ hooks:
python-modules:
- airflow.providers.tableau.hooks.tableau
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.tableau.hooks.tableau.TableauHook
connection-types:
diff --git a/airflow/providers/trino/provider.yaml b/airflow/providers/trino/provider.yaml
index a5fcbfc..5ec1723 100644
--- a/airflow/providers/trino/provider.yaml
+++ b/airflow/providers/trino/provider.yaml
@@ -39,7 +39,7 @@ hooks:
python-modules:
- airflow.providers.trino.hooks.trino
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.trino.hooks.trino.TrinoHook
connection-types:
diff --git a/airflow/providers/vertica/provider.yaml b/airflow/providers/vertica/provider.yaml
index 21951c7..288f877 100644
--- a/airflow/providers/vertica/provider.yaml
+++ b/airflow/providers/vertica/provider.yaml
@@ -45,7 +45,7 @@ hooks:
python-modules:
- airflow.providers.vertica.hooks.vertica
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.vertica.hooks.vertica.VerticaHook
connection-types:
diff --git a/airflow/providers/yandex/provider.yaml b/airflow/providers/yandex/provider.yaml
index 69c2ea9..c2fd261 100644
--- a/airflow/providers/yandex/provider.yaml
+++ b/airflow/providers/yandex/provider.yaml
@@ -55,7 +55,7 @@ hooks:
python-modules:
- airflow.providers.yandex.hooks.yandexcloud_dataproc
-hook-class-names:
+hook-class-names: # deprecated - to be removed after providers add dependency on Airflow 2.2.0+
- airflow.providers.yandex.hooks.yandex.YandexCloudBaseHook
connection-types:
diff --git a/docs/apache-airflow-providers/howto/create-update-providers.rst b/docs/apache-airflow-providers/howto/create-update-providers.rst
index 68fc8b6..7f9cb61 100644
--- a/docs/apache-airflow-providers/howto/create-update-providers.rst
+++ b/docs/apache-airflow-providers/howto/create-update-providers.rst
@@ -279,11 +279,22 @@ In the ``airflow/providers/<NEW_PROVIDER>/provider.yaml`` add information of you
python-modules:
- airflow.providers.<NEW_PROVIDER>.sensors.<NEW_PROVIDER>
- hook-class-names:
+ connection-types:
+ - hook-class-name: airflow.providers.<NEW_PROVIDER>.hooks.<NEW_PROVIDER>.NewProviderHook
+ - connection-type: provider-connection-type
+
+ hook-class-names: # deprecated in Airflow 2.2.0
- airflow.providers.<NEW_PROVIDER>.hooks.<NEW_PROVIDER>.NewProviderHook
-You only need to add ``hook-class-names`` in case you have some hooks that have customized UI behavior.
-For more information see `Custom connection types <http://airflow.apache.org/docs/apache-airflow/stable/howto/connection.html#custom-connection-types>`_
+.. note:: Defining your own connection types
+
+ You only need to add ``connection-types`` in case you have some hooks that have customized UI behavior. However
+ it is only supported for Airflow 2.2.0. If your providers are also targeting Airflow below 2.2.0 you should
+ provide the deprecated ``hook-class-names`` array. The ``connection-types`` array allows for optimization
+ of importing of individual connections and while Airflow 2.2.0 is able to handle both definition, the
+ ``connection-types`` is recommended.
+
+ For more information see `Custom connection types <http://airflow.apache.org/docs/apache-airflow/stable/howto/connection.html#custom-connection-types>`_
After changing and creating these files you can build the documentation locally. The two commands below will
diff --git a/docs/apache-airflow-providers/index.rst b/docs/apache-airflow-providers/index.rst
index 71c5132..99ac4b7 100644
--- a/docs/apache-airflow-providers/index.rst
+++ b/docs/apache-airflow-providers/index.rst
@@ -143,9 +143,17 @@ Exposing customized functionality to the Airflow's core:
capability. See :doc:`apache-airflow:howto/define_extra_link` for description of how to add extra link
capability to the operators of yours.
-* ``hook-class-names`` - this field should contain the list of all hook class names that provide
- custom connection types with custom extra fields and field behaviour. See
- :doc:`apache-airflow:howto/connection` for more details.
+* ``connection-types`` - this field should contain the list of all connection types together with hook
+ class names implementing those custom connection types (providing custom extra fields and
+ custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
+ ``hook-class-names``. See :doc:`apache-airflow:howto/connection` for more details
+
+* ``hook-class-names`` (deprecated) - this field should contain the list of all hook class names that provide
+ custom connection types with custom extra fields and field behaviour. The ``hook-class-names`` array
+ is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
+ providers are targeting Airflow 2.2.0+ you do not have to include the ``hook-class-names`` array, if
+ you want to also target earlier versions of Airflow 2, you should include both ``hook-class-names`` and
+ ``connection-types`` arrays. See :doc:`apache-airflow:howto/connection` for more details.
When your providers are installed you can query the installed providers and their capabilities with the
@@ -209,7 +217,8 @@ Creating your own providers
**When I write my own provider, do I need to do anything special to make it available to others?**
You do not need to do anything special besides creating the ``apache_airflow_provider`` entry point
-returning properly formatted meta-data (dictionary with ``extra-links`` and ``hook-class-names`` fields).
+returning properly formatted meta-data - dictionary with ``extra-links`` and ``connection-types`` fields
+(and deprecated ``hook-class-names`` field if you are also targeting versions of Airflow before 2.2.0).
Anyone who runs airflow in an environment that has your Python package installed will be able to use the
package as a provider package.
diff --git a/docs/apache-airflow/howto/connection.rst b/docs/apache-airflow/howto/connection.rst
index 63dd054..d95766b 100644
--- a/docs/apache-airflow/howto/connection.rst
+++ b/docs/apache-airflow/howto/connection.rst
@@ -401,8 +401,8 @@ custom Hook should not derive from this class, this class is a dummy example to
regarding about class fields and methods that your Hook might define. Another good example is
:py:class:`~airflow.providers.jdbc.hooks.jdbc.JdbcHook`.
-By implementing those methods in your hooks and exposing them via ``hook-class-names`` array in
-the provider meta-data you can customize Airflow by:
+By implementing those methods in your hooks and exposing them via ``connection-types`` array (and
+deprecated ``hook-class-names``) in the provider meta-data, you can customize Airflow by:
* Adding custom connection types
* Adding automated Hook creation from the connection type
@@ -411,3 +411,11 @@ the provider meta-data you can customize Airflow by:
* Adding placeholders showing examples of how fields should be formatted
You can read more about details how to add custom provider packages in the :doc:`apache-airflow-providers:index`
+
+.. note:: Deprecated ``hook-class-names``
+
+ Prior to Airflow 2.2.0, the connections in providers have been exposed via ``hook-class-names`` array
+ in provider's meta-data, this however has proven to be not well optimized for using individual hooks
+ in workers and the ``hook-class-names`` array is now replaced by ``connection-types`` array. Until
+ provider supports Airflow below 2.2.0, both ``connection-types`` and ``hook-class-names`` should be
+ present. Automated checks during CI build will verify consistency of those two arrays.