[GitHub] [airflow] ashb commented on a change in pull request #5007: [AIRFLOW-XXX] Remove duplicate information

[GitBox <gi...@apache.org>]

ashb commented on a change in pull request #5007: [AIRFLOW-XXX] Remove duplicate information
URL: https://github.com/apache/airflow/pull/5007#discussion_r272981312
 
 

 ##########
 File path: docs/integration.rst
 ##########
 @@ -100,163 +91,118 @@ Airflow connection of type `azure_data_lake` exists. Authorization can be done b
 login (=Client ID), password (=Client Secret) and extra fields tenant (Tenant) and account_name (Account Name)
 (see connection `azure_data_lake_default` for an example).
 
-:class:`airflow.contrib.hooks.azure_data_lake_hook.AzureDataLakeHook`
-    Interface with Azure Data Lake.
-
-:class:`airflow.contrib.operators.adls_list_operator.AzureDataLakeStorageListOperator`
-    Lists the files located in a specified Azure Data Lake path.
-
-:class:`airflow.contrib.operators.adls_to_gcs.AdlsToGoogleCloudStorageOperator`
-    Copies files from an Azure Data Lake path to a Google Cloud Storage bucket.
-
-
-Azure Container Instances
-'''''''''''''''''''''''''
-
-Azure Container Instances provides a method to run a docker container without having to worry
-about managing infrastructure. The AzureContainerInstanceHook requires a service principal. The
-credentials for this principal can either be defined in the extra field ``key_path``, as an
-environment variable named ``AZURE_AUTH_LOCATION``,
-or by providing a login/password and tenantId in extras.
-
-The AzureContainerRegistryHook requires a host/login/password to be defined in the connection.
-
-:class:`airflow.contrib.hooks.azure_container_volume_hook.AzureContainerVolumeHook`
-    Interface with Azure Container Volumes
+For further information, look at API Reference:
 
-:class:`airflow.contrib.operators.azure_container_instances_operator.AzureContainerInstancesOperator`
-    Start/Monitor a new ACI.
+* :py:mod:`airflow.contrib.hooks.azure_data_lake_hook`
+* :py:mod:`airflow.contrib.operators.adls_list_operator`
+* :py:mod:`airflow.contrib.operators.adls_to_gcs`
 
-:class:`airflow.contrib.hooks.azure_container_instance_hook.AzureContainerInstanceHook`
-    Wrapper around a single ACI.
+Azure File Share
+''''''''''''''''
 
-:class:`airflow.contrib.hooks.azure_container_registry_hook.AzureContainerRegistryHook`
-    Interface with ACR
+Cloud variant of a SMB file share. Make sure that a Airflow connection of
+type `wasb` exists. Authorization can be done by supplying a login (=Storage account name)
+and password (=Storage account key), or login and SAS token in the extra field
+(see connection `wasb_default` for an example).
 
+For further information, look at API Reference:
 
+* :py:mod:`airflow.contrib.hooks.azure_fileshare_hook`
 
 .. _AWS:
 
-AWS: Amazon Web Services
-------------------------
-
-Airflow has extensive support for Amazon Web Services. But note that the Hooks, Sensors and
-Operators are in the contrib section.
-
-AWS EMR
-'''''''
-
-:class:`airflow.contrib.hooks.emr_hook.EmrHook`
-    Interface with AWS EMR.
+Amazon Web Services
+-------------------
 
-:class:`airflow.contrib.operators.emr_add_steps_operator.EmrAddStepsOperator`
-    Adds steps to an existing EMR JobFlow.
-
-:class:`airflow.contrib.operators.emr_create_job_flow_operator.EmrCreateJobFlowOperator`
-    Creates an EMR JobFlow, reading the config from the EMR connection.
-
-:class:`airflow.contrib.operators.emr_terminate_job_flow_operator.EmrTerminateJobFlowOperator`
-    Terminates an EMR JobFlow.
-
-
-AWS S3
-''''''
-
-:class:`airflow.hooks.S3_hook.S3Hook`
-    Interface with AWS S3.
-
-:class:`airflow.operators.s3_file_transform_operator.S3FileTransformOperator`
-    Copies data from a source S3 location to a temporary location on the local filesystem.
-
-:class:`airflow.contrib.operators.s3_list_operator.S3ListOperator`
-    Lists the files matching a key prefix from a S3 location.
-
-:class:`airflow.contrib.operators.s3_to_gcs_operator.S3ToGoogleCloudStorageOperator`
-    Syncs an S3 location with a Google Cloud Storage bucket.
-
-:class:`airflow.contrib.operators.s3_to_gcs_transfer_operator.S3ToGoogleCloudStorageTransferOperator`
-    Syncs an S3 bucket with a Google Cloud Storage bucket using the GCP Storage Transfer Service.
-
-:class:`airflow.operators.s3_to_hive_operator.S3ToHiveTransfer`
-    Moves data from S3 to Hive. The operator downloads a file from S3, stores the file locally before loading it into a Hive table.
-
-
-AWS Batch Service
-'''''''''''''''''
-
-:class:`airflow.contrib.operators.awsbatch_operator.AWSBatchOperator`
-    Execute a task on AWS Batch Service.
+Airflow has extensive support for Amazon Web Services.
 
+Logging
+^^^^^^^
 
-AWS RedShift
-''''''''''''
+Airflow can be configured to read and write task logs in Amazon Simple Storage Service (Amazon S3).
+See :ref:`write-logs-amazon`.
 
-:class:`airflow.contrib.sensors.aws_redshift_cluster_sensor.AwsRedshiftClusterSensor`
-    Waits for a Redshift cluster to reach a specific status.
+Operators and hooks
+^^^^^^^^^^^^^^^^^^^
 
-:class:`airflow.contrib.hooks.redshift_hook.RedshiftHook`
-    Interact with AWS Redshift, using the boto3 library.
+Amazon DynamoDB
+'''''''''''''''
 
-:class:`airflow.operators.redshift_to_s3_operator.RedshiftToS3Transfer`
-    Executes an unload command to S3 as CSV with or without headers.
+For information, look at API Reference:
 
-:class:`airflow.operators.s3_to_redshift_operator.S3ToRedshiftTransfer`
-    Executes an copy command from S3 as CSV with or without headers.
+* :py:mod:`airflow.contrib.operators.hive_to_dynamodb`
+* :py:mod:`airflow.contrib.hooks.aws_dynamodb_hook`
 
+Amazon EMR
+''''''''''
 
+For information, look at API Reference:
 
-AWS DynamoDB
-''''''''''''
+* :py:mod:`airflow.contrib.hooks.emr_hook`
+* :py:mod:`airflow.contrib.operators.emr_add_steps_operator`
+* :py:mod:`airflow.contrib.operators.emr_create_job_flow_operator`
+* :py:mod:`airflow.contrib.operators.emr_terminate_job_flow_operator`
 
-:class:`airflow.contrib.operators.hive_to_dynamodb.HiveToDynamoDBTransferOperator`
-     Moves data from Hive to DynamoDB.
+Amazon RedShift
+'''''''''''''''
 
-:class:`airflow.contrib.hooks.aws_dynamodb_hook.AwsDynamoDBHook`
-    Interface with AWS DynamoDB.
+For information, look at API Reference:
 
+* :py:mod:`airflow.contrib.sensors.aws_redshift_cluster_sensor`
+* :py:mod:`airflow.contrib.hooks.redshift_hook`
+* :py:mod:`airflow.operators.redshift_to_s3_operator`
+* :py:mod:`airflow.operators.s3_to_redshift_operator`
 
-AWS Lambda
-''''''''''
+Amazon SageMaker
+''''''''''''''''
 
-:class:`airflow.contrib.hooks.aws_lambda_hook.AwsLambdaHook`
-    Interface with AWS Lambda.
+For instructions on using Amazon SageMaker in Airflow, please see `the SageMaker Python SDK README`_.
 
+.. _the SageMaker Python SDK README: https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/workflow/README.rst
 
-AWS Kinesis
-'''''''''''
+Additional information is available at API Reference:
 
-:class:`airflow.contrib.hooks.aws_firehose_hook.AwsFirehoseHook`
-    Interface with AWS Kinesis Firehose.
+* :py:mod:`airflow.contrib.hooks.sagemaker_hook`
+* :py:mod:`airflow.contrib.operators.sagemaker_training_operator`
+* :py:mod:`airflow.contrib.operators.sagemaker_tuning_operator`
+* :py:mod:`airflow.contrib.hooks.azure_container_registry_hook`
+* :py:mod:`airflow.contrib.operators.sagemaker_model_operator`
+* :py:mod:`airflow.contrib.operators.sagemaker_transform_operator`
+* :py:mod:`airflow.contrib.operators.sagemaker_endpoint_config_operator`
+* :py:mod:`airflow.contrib.operators.sagemaker_endpoint_operator`
 
+Amazon Simple Storage Service (Amazon S3)
+'''''''''''''''''''''''''''''''''''''''''
 
-Amazon SageMaker
-''''''''''''''''
+For information, look at API Reference:
 
-For more instructions on using Amazon SageMaker in Airflow, please see `the SageMaker Python SDK README`_.
+* :py:mod:`airflow.hooks.S3_hook`
+* :py:mod:`airflow.operators.s3_file_transform_operator`
+* :py:mod:`airflow.contrib.operators.s3_list_operator`
+* :py:mod:`airflow.contrib.operators.s3_to_gcs_operator`
+* :py:class:`airflow.contrib.operators.gcp_transfer_operator.S3ToGoogleCloudStorageTransferOperator`
+* :py:mod:`airflow.operators.s3_to_hive_operator.S3ToHiveTransfer`
 
-.. _the SageMaker Python SDK README: https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/workflow/README.rst
+Amazon Web Service Batch Service
 
 Review comment:
   AWS vs "Amazon Web Service" (an it should be Service**s** I think if we expand it)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services