You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2023/01/08 10:26:51 UTC

[GitHub] [airflow] Taragolis opened a new issue, #28790: Add missing connection types and documentation for community providers

Taragolis opened a new issue, #28790:
URL: https://github.com/apache/airflow/issues/28790

   ### Body
   
   Some of Hooks provide connection ability, however quite a few of them do not provide any documentation and/or connection type (missing in the UI). It would be nice if we add missing parts which might help Airflow users.
   
   [**apache.druid**](https://airflow.apache.org/docs/apache-airflow-providers-apache-druid/stable/index.html)
   - [ ] [DruidHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-druid/stable/_modules/airflow/providers/apache/druid/hooks/druid.html#DruidHook) (Missing conn_type, conn_name_attr = "druid_ingest_conn_id")
   
   [**apache.hdfs**](https://airflow.apache.org/docs/apache-airflow-providers-apache-hdfs/stable/index.html)
   Mixin connections documentation [in the single page](https://airflow.apache.org/docs/apache-airflow-providers-apache-hdfs/stable/connections.html#apache-hdfs-connection)
   - [ ] [HDFSHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-hdfs/stable/_modules/airflow/providers/apache/hdfs/hooks/hdfs.html#HDFSHook) (conn_type = "hdfs", conn_name_attr = "hdfs_conn_id", hook_name = "HDFS")
   - [ ] [WebHDFSHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-hdfs/stable/_modules/airflow/providers/apache/hdfs/hooks/webhdfs.html#WebHDFSHook) (Missing conn_type, conn_name_attr = "webhdfs_conn_id")
   
   [**apache.kylin**](https://airflow.apache.org/docs/apache-airflow-providers-apache-kylin/stable/index.html)
   - [ ] [KylinHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-kylin/stable/_modules/airflow/providers/apache/kylin/hooks/kylin.html#KylinHook) (Missing conn_type, conn_name_attr = "kylin_conn_id")
   
   [**apache.livy**](https://airflow.apache.org/docs/apache-airflow-providers-apache-livy/stable/index.html)
   - [ ] [LivyHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-livy/stable/_modules/airflow/providers/apache/livy/hooks/livy.html#LivyHook) (conn_type = "livy", conn_name_attr = "livy_conn_id", hook_name = "Apache Livy")
   
   [**apache.pig**](https://airflow.apache.org/docs/apache-airflow-providers-apache-pig/stable/index.html)
   - [ ] [PigCliHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-pig/stable/_modules/airflow/providers/apache/pig/hooks/pig.html#PigCliHook) (conn_type = "pig_cli", conn_name_attr = "pig_cli_conn_id", hook_name = "Pig Client Wrapper")
   
   [**apache.pinot**](https://airflow.apache.org/docs/apache-airflow-providers-apache-pinot/stable/index.html)
   - [ ] [PinotAdminHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-pinot/stable/_modules/airflow/providers/apache/pinot/hooks/pinot.html#PinotAdminHook) (Missing conn_type, conn_name_attr = "conn_id")
   - [ ] [PinotDbApiHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-pinot/stable/_modules/airflow/providers/apache/pinot/hooks/pinot.html#PinotDbApiHook) (Missing conn_type, conn_name_attr = "pinot_broker_conn_id")
   
   [**apache.spark**](https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/stable/index.html)
   Mixin connections documentation [in the single page](https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/stable/connections/spark.html).
   - [ ] [SparkSubmitHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/stable/_modules/airflow/providers/apache/spark/hooks/spark_submit.html#SparkSubmitHook) (conn_type = "spark", conn_name_attr = "conn_id", hook_name = "Spark")
   - [ ] [SparkJDBCHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/stable/_modules/airflow/providers/apache/spark/hooks/spark_jdbc.html#SparkJDBCHook) (conn_type = "spark_jdbc", conn_name_attr = "spark_conn_id", hook_name = "Spark JDBC")
   - [ ] [SparkSqlHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-spark/stable/_modules/airflow/providers/apache/spark/hooks/spark_sql.html#SparkSqlHook) (conn_type = "spark_sql", conn_name_attr = "conn_id", hook_name = "Spark SQL")
   
   [**apache.sqoop**](https://airflow.apache.org/docs/apache-airflow-providers-apache-sqoop/stable/index.html)
   - [ ] [SqoopHook](https://airflow.apache.org/docs/apache-airflow-providers-apache-sqoop/stable/_modules/airflow/providers/apache/sqoop/hooks/sqoop.html#SqoopHook) (conn_type = "sqoop", conn_name_attr = "conn_id", hook_name = "Sqoop")
   
   [**datadog**](https://airflow.apache.org/docs/apache-airflow-providers-datadog/stable/index.html)
   - [ ] [DatadogHook](https://airflow.apache.org/docs/apache-airflow-providers-datadog/stable/_modules/airflow/providers/datadog/hooks/datadog.html#DatadogHook) (Missing conn_type, conn_name_attr = "datadog_conn_id")
   
   [**dingding**](https://airflow.apache.org/docs/apache-airflow-providers-dingding/stable/index.html)
   - [ ] [DingdingHook](https://airflow.apache.org/docs/apache-airflow-providers-dingding/stable/_modules/airflow/providers/dingding/hooks/dingding.html#DingdingHook) (conn_type = "dingding", conn_name_attr = "dingding_conn_id", hook_name = "Dingding")
   
   [**discord**](https://airflow.apache.org/docs/apache-airflow-providers-discord/stable/index.html)
   Discord built in top of HTTP Connection with specific settings, which only described in docstring
   - [ ] [DiscordWebhookHook](https://airflow.apache.org/docs/apache-airflow-providers-discord/stable/_modules/airflow/providers/discord/hooks/discord_webhook.html#DiscordWebhookHook) (conn_type = "discord", conn_name_attr = "http_conn_id", hook_name = "Discord")
   
   [**exasol**](https://airflow.apache.org/docs/apache-airflow-providers-exasol/stable/index.html)
   - [ ] [ExasolHook](https://airflow.apache.org/docs/apache-airflow-providers-exasol/stable/_modules/airflow/providers/exasol/hooks/exasol.html#ExasolHook) (conn_type = "exasol", conn_name_attr = "exasol_conn_id", hook_name = "Exasol")
   
   [**facebook**](https://airflow.apache.org/docs/apache-airflow-providers-facebook/stable/index.html)
   - [ ] [FacebookAdsReportingHook](https://airflow.apache.org/docs/apache-airflow-providers-facebook/stable/_modules/airflow/providers/facebook/ads/hooks/ads.html#FacebookAdsReportingHook) (conn_type = "facebook_social", conn_name_attr = "facebook_conn_id", hook_name = "Facebook Ads")
   
   [**google**](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/index.html)
   - [ ] [GoogleAdsHook](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/_modules/airflow/providers/google/ads/hooks/ads.html#GoogleAdsHook) (Missing conn_type, conn_name_attr = "google_ads_conn_id")
   - [ ] [LookerHook](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/_modules/airflow/providers/google/cloud/hooks/looker.html#LookerHook) (Missing conn_type, conn_name_attr = "looker_conn_id", **Note**: Connection Type has [documentation](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/connections/gcp_looker.html))
   - [ ] [LevelDBHook](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/_modules/airflow/providers/google/leveldb/hooks/leveldb.html#LevelDBHook) (conn_type = "leveldb", conn_name_attr = "leveldb_conn_id", hook_name = "LevelDB")
   
   [**hashicorp**](https://airflow.apache.org/docs/apache-airflow-providers-hashicorp/stable/index.html)
   - [ ] [VaultHook](https://airflow.apache.org/docs/apache-airflow-providers-hashicorp/stable/_modules/airflow/providers/hashicorp/hooks/vault.html#VaultHook) (conn_type = "vault", conn_name_attr = "vault_conn_id", hook_name = "Hashicorp Vault")
   
   [**cloudant**](https://airflow.apache.org/docs/apache-airflow-providers-cloudant/stable/index.html)
   - [ ] [CloudantHook](PLACEHOLDER) (conn_type = "cloudant", conn_name_attr = "cloudant_conn_id", hook_name = "Cloudant")
   
   [**atlassian.jira**](https://airflow.apache.org/docs/apache-airflow-providers-atlassian-jira/stable/index.html)
   - [ ] [JiraHook](https://airflow.apache.org/docs/apache-airflow-providers-atlassian-jira/stable/_modules/airflow/providers/atlassian/jira/hooks/jira.html#JiraHook) (conn_type = "jira", conn_name_attr = "jira_conn_id", hook_name = "JIRA")
   
   [**microsoft.azure**](https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/index.html)
   - [ ] [AzureContainerInstanceHook](https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/_modules/airflow/providers/microsoft/azure/hooks/container_instance.html#AzureContainerInstanceHook) (conn_type = "azure_container_instance", conn_name_attr = "azure_conn_id", hook_name = "Azure Container Instance")
   
   [**microsoft.psrp**](https://airflow.apache.org/docs/apache-airflow-providers-microsoft-psrp/stable/index.html)
   - [ ] [PsrpHook](https://airflow.apache.org/docs/apache-airflow-providers-microsoft-psrp/stable/_modules/airflow/providers/microsoft/psrp/hooks/psrp.html#PsrpHook) (Missing conn_type, conn_name_attr = "psrp_conn_id")
   
   [**microsoft.winrm**](https://airflow.apache.org/docs/apache-airflow-providers-microsoft-winrm/stable/index.html)
   I'm not familiar with WinRM Protocol, but seems like it actually doesn't use SSH Connection, however connection attribute is `ssh_conn_id` which might confuse end users.
   - [ ] [WinRMHook](https://airflow.apache.org/docs/apache-airflow-providers-microsoft-winrm/stable/_modules/airflow/providers/microsoft/winrm/hooks/winrm.html#WinRMHook) (Missing conn_type, conn_name_attr = "ssh_conn_id")
   
   [**openfaas**](https://airflow.apache.org/docs/apache-airflow-providers-openfaas/stable/index.html)
   - [ ] [OpenFaasHook](https://airflow.apache.org/docs/apache-airflow-providers-openfaas/stable/_modules/airflow/providers/openfaas/hooks/openfaas.html#OpenFaasHook) (Missing conn_type, conn_name_attr = "conn_id")
   
   [**opsgenie**](https://airflow.apache.org/docs/apache-airflow-providers-opsgenie/stable/index.html)
   - [ ] [OpsgenieAlertHook](https://airflow.apache.org/docs/apache-airflow-providers-opsgenie/stable/_modules/airflow/providers/opsgenie/hooks/opsgenie.html#OpsgenieAlertHook) (conn_type = "opsgenie", conn_name_attr = "opsgenie_conn_id", hook_name = "Opsgenie")
   
   [**pagerduty**](https://airflow.apache.org/docs/apache-airflow-providers-pagerduty/stable/index.html)
   - [ ] [PagerdutyHook](https://airflow.apache.org/docs/apache-airflow-providers-pagerduty/stable/_modules/airflow/providers/pagerduty/hooks/pagerduty.html#PagerdutyHook) (conn_type = "pagerduty", conn_name_attr = "pagerduty_conn_id", hook_name = "Pagerduty")
   - [ ] [PagerdutyEventsHook](https://airflow.apache.org/docs/apache-airflow-providers-pagerduty/stable/_modules/airflow/providers/pagerduty/hooks/pagerduty_events.html#PagerdutyEventsHook) (conn_type = "pagerduty_events", conn_name_attr = "pagerduty_events_conn_id", hook_name = "Pagerduty Events")
   
   [**presto**](https://airflow.apache.org/docs/apache-airflow-providers-presto/stable/index.html)
   - [ ] [PrestoHook](https://airflow.apache.org/docs/apache-airflow-providers-presto/stable/_modules/airflow/providers/presto/hooks/presto.html#PrestoHook) (conn_type = "presto", conn_name_attr = "presto_conn_id", hook_name = "Presto")
   
   [**qubole**](https://airflow.apache.org/docs/apache-airflow-providers-qubole/stable/index.html)
   - [ ] [QuboleHook](https://airflow.apache.org/docs/apache-airflow-providers-qubole/stable/_modules/airflow/providers/qubole/hooks/qubole.html#QuboleHook) (conn_type = "qubole", conn_name_attr = "qubole_conn_id", hook_name = "Qubole")
   
   [**redis**](https://airflow.apache.org/docs/apache-airflow-providers-redis/stable/index.html)
   - [ ] [RedisHook](https://airflow.apache.org/docs/apache-airflow-providers-redis/stable/_modules/airflow/providers/redis/hooks/redis.html#RedisHook) (conn_type = "redis", conn_name_attr = "redis_conn_id", hook_name = "Redis")
   
   [**samba**](https://airflow.apache.org/docs/apache-airflow-providers-samba/stable/index.html)
   - [ ] [SambaHook](https://airflow.apache.org/docs/apache-airflow-providers-samba/stable/_modules/airflow/providers/samba/hooks/samba.html#SambaHook) (conn_type = "samba", conn_name_attr = "samba_conn_id", hook_name = "Samba")
   
   [**segment**](https://airflow.apache.org/docs/apache-airflow-providers-segment/stable/index.html)
   - [ ] [SegmentHook](https://airflow.apache.org/docs/apache-airflow-providers-segment/stable/_modules/airflow/providers/segment/hooks/segment.html#SegmentHook) (conn_type = "segment", conn_name_attr = "segment_conn_id", hook_name = "Segment")
   
   [**telegram**](https://airflow.apache.org/docs/apache-airflow-providers-telegram/stable/index.html)
   - [ ] [TelegramHook](https://airflow.apache.org/docs/apache-airflow-providers-telegram/stable/_modules/airflow/providers/telegram/hooks/telegram.html#TelegramHook) (Missing conn_type, conn_name_attr = "telegram_conn_id")
   
   [**vertica**](https://airflow.apache.org/docs/apache-airflow-providers-vertica/stable/index.html)
   - [ ] [VerticaHook](https://airflow.apache.org/docs/apache-airflow-providers-vertica/stable/_modules/airflow/providers/vertica/hooks/vertica.html#VerticaHook) (conn_type = "vertica", conn_name_attr = "vertica_conn_id", hook_name = "Vertica")
   
   [**zendesk**](https://airflow.apache.org/docs/apache-airflow-providers-zendesk/stable/index.html)
   - [ ] [Zendesk](https://airflow.apache.org/docs/apache-airflow-providers-zendesk/stable/_modules/airflow/providers/zendesk/hooks/zendesk.html#ZendeskHook) (conn_type = "zendesk", conn_name_attr = "zendesk_conn_id", hook_name = "Zendesk")
   
   ### Committer
   
   - [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] hussein-awala commented on issue #28790: Add missing connection types and documentation for community providers

Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.

hussein-awala commented on issue #28790:
URL: https://github.com/apache/airflow/issues/28790#issuecomment-1465280186

   @Taragolis I just created #30057 to fix Vault connection and add missing doc, can you check it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Add missing connection types and documentation for community providers [airflow]

Posted by "shohamy7 (via GitHub)" <gi...@apache.org>.

shohamy7 commented on issue #28790:
URL: https://github.com/apache/airflow/issues/28790#issuecomment-1868604302

   Hi @Taragolis 
   Looks like WebHDFSHook has been resolved in #36145 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org