You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by el...@apache.org on 2023/04/26 09:46:19 UTC
[airflow-site] branch add-documentation-2023-04-26 created (now 84b45cbe72)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a change to branch add-documentation-2023-04-26
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
at 84b45cbe72 Add documentation for packages - 2023-04-26
This branch includes the following new commits:
new 84b45cbe72 Add documentation for packages - 2023-04-26
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[airflow-site] 01/01: Add documentation for packages - 2023-04-26
Posted by el...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch add-documentation-2023-04-26
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
commit 84b45cbe72f2b6566791db4d9c297795d275412d
Author: Elad Kalif <45...@users.noreply.github.com>
AuthorDate: Wed Apr 26 12:45:21 2023 +0300
Add documentation for packages - 2023-04-26
---
.../8.0.0/.buildinfo | 4 +
.../providers/amazon/aws/exceptions}/index.html | 659 +-
.../providers/amazon/aws/hooks/appflow}/index.html | 694 +-
.../providers/amazon/aws/hooks/athena}/index.html | 810 +-
.../providers/amazon/aws/hooks/base_aws/index.html | 1453 +++
.../amazon/aws/hooks/batch_client}/index.html | 1136 +-
.../amazon/aws/hooks/batch_waiters}/index.html | 809 +-
.../amazon/aws/hooks/cloud_formation}/index.html | 684 +-
.../amazon/aws/hooks/datasync}/index.html | 941 +-
.../providers/amazon/aws/hooks/dms}/index.html | 823 +-
.../amazon/aws/hooks/dynamodb}/index.html | 665 +-
.../providers/amazon/aws/hooks/ec2}/index.html | 837 +-
.../providers/amazon/aws/hooks/ecr}/index.html | 734 +-
.../providers/amazon/aws/hooks/ecs}/index.html | 991 +-
.../providers/amazon/aws/hooks/eks/index.html | 1543 +++
.../elasticache_replication_group}/index.html | 831 +-
.../providers/amazon/aws/hooks/emr}/index.html | 985 +-
.../providers/amazon/aws/hooks/glacier}/index.html | 688 +-
.../providers/amazon/aws/hooks/glue}/index.html | 872 +-
.../amazon/aws/hooks/glue_catalog}/index.html | 783 +-
.../amazon/aws/hooks/glue_crawler}/index.html | 795 +-
.../airflow/providers/amazon/aws/hooks}/index.html | 655 +-
.../providers/amazon/aws/hooks/kinesis}/index.html | 661 +-
.../amazon/aws/hooks/lambda_function}/index.html | 712 +-
.../providers/amazon/aws/hooks/logs}/index.html | 695 +-
.../amazon/aws/hooks/quicksight}/index.html | 750 +-
.../providers/amazon/aws/hooks/rds}/index.html | 913 +-
.../amazon/aws/hooks/redshift_cluster}/index.html | 850 +-
.../amazon/aws/hooks/redshift_data}/index.html | 707 +-
.../amazon/aws/hooks/redshift_sql}/index.html | 727 +-
.../providers/amazon/aws/hooks/s3/index.html | 1802 +++
.../amazon/aws/hooks/sagemaker/index.html | 1920 ++++
.../amazon/aws/hooks/secrets_manager}/index.html | 681 +-
.../providers/amazon/aws/hooks/ses}/index.html | 676 +-
.../providers/amazon/aws/hooks/sns}/index.html | 666 +-
.../providers/amazon/aws/hooks/sqs}/index.html | 687 +-
.../providers/amazon/aws/hooks/ssm}/index.html | 657 +-
.../amazon/aws/hooks/step_function}/index.html | 681 +-
.../providers/amazon/aws/hooks/sts}/index.html | 652 +-
.../_api/airflow/providers/amazon/aws}/index.html | 760 +-
.../amazon/aws/links/base_aws}/index.html | 724 +-
.../providers/amazon/aws/links/batch}/index.html | 720 +-
.../providers/amazon/aws/links/emr}/index.html | 689 +-
.../providers/amazon/aws/links/glue}/index.html | 658 +-
.../airflow/providers/amazon/aws/links}/index.html | 624 +-
.../providers/amazon/aws/links/logs}/index.html | 669 +-
.../aws/log/cloudwatch_task_handler}/index.html | 704 +-
.../airflow/providers/amazon/aws/log}/index.html | 621 +-
.../amazon/aws/log/s3_task_handler}/index.html | 766 +-
.../amazon/aws/operators/appflow}/index.html | 920 +-
.../amazon/aws/operators/athena}/index.html | 711 +-
.../amazon/aws/operators/batch}/index.html | 835 +-
.../aws/operators/cloud_formation}/index.html | 742 +-
.../amazon/aws/operators/datasync}/index.html | 776 +-
.../providers/amazon/aws/operators/dms}/index.html | 923 +-
.../providers/amazon/aws/operators/ec2}/index.html | 832 +-
.../providers/amazon/aws/operators/ecs}/index.html | 979 +-
.../providers/amazon/aws/operators/eks/index.html | 1461 +++
.../providers/amazon/aws/operators/emr/index.html | 1683 +++
.../amazon/aws/operators/glacier}/index.html | 711 +-
.../amazon/aws/operators/glue}/index.html | 706 +-
.../amazon/aws/operators/glue_crawler}/index.html | 686 +-
.../providers/amazon/aws/operators}/index.html | 642 +-
.../aws/operators/lambda_function}/index.html | 754 +-
.../amazon/aws/operators/quicksight}/index.html | 705 +-
.../providers/amazon/aws/operators/rds/index.html | 1383 +++
.../aws/operators/redshift_cluster/index.html | 1316 +++
.../amazon/aws/operators/redshift_data}/index.html | 712 +-
.../providers/amazon/aws/operators/s3/index.html | 1646 +++
.../amazon/aws/operators/sagemaker/index.html | 1801 +++
.../providers/amazon/aws/operators/sns}/index.html | 683 +-
.../providers/amazon/aws/operators/sqs}/index.html | 693 +-
.../amazon/aws/operators/step_function}/index.html | 734 +-
.../providers/amazon/aws/secrets}/index.html | 621 +-
.../amazon/aws/secrets/secrets_manager}/index.html | 743 +-
.../amazon/aws/secrets/systems_manager}/index.html | 731 +-
.../amazon/aws/sensors/athena}/index.html | 708 +-
.../providers/amazon/aws/sensors/batch}/index.html | 817 +-
.../amazon/aws/sensors/cloud_formation}/index.html | 736 +-
.../providers/amazon/aws/sensors/dms}/index.html | 722 +-
.../amazon/aws/sensors/dynamodb}/index.html | 668 +-
.../providers/amazon/aws/sensors/ec2}/index.html | 690 +-
.../providers/amazon/aws/sensors/ecs}/index.html | 800 +-
.../providers/amazon/aws/sensors/eks}/index.html | 938 +-
.../providers/amazon/aws/sensors/emr/index.html | 1531 +++
.../amazon/aws/sensors/glacier}/index.html | 710 +-
.../providers/amazon/aws/sensors/glue}/index.html | 672 +-
.../aws/sensors/glue_catalog_partition}/index.html | 697 +-
.../amazon/aws/sensors/glue_crawler}/index.html | 682 +-
.../providers/amazon/aws/sensors}/index.html | 640 +-
.../amazon/aws/sensors/lambda_function}/index.html | 681 +-
.../amazon/aws/sensors/quicksight}/index.html | 690 +-
.../providers/amazon/aws/sensors/rds}/index.html | 740 +-
.../aws/sensors/redshift_cluster}/index.html | 681 +-
.../providers/amazon/aws/sensors/s3}/index.html | 802 +-
.../amazon/aws/sensors/sagemaker/index.html | 1413 +++
.../providers/amazon/aws/sensors/sqs}/index.html | 748 +-
.../amazon/aws/sensors/step_function}/index.html | 715 +-
.../amazon/aws/transfers/base}/index.html | 659 +-
.../aws/transfers/dynamodb_to_s3}/index.html | 703 +-
.../amazon/aws/transfers/exasol_to_s3}/index.html | 695 +-
.../amazon/aws/transfers/ftp_to_s3}/index.html | 688 +-
.../amazon/aws/transfers/gcs_to_s3}/index.html | 715 +-
.../aws/transfers/glacier_to_gcs}/index.html | 679 +-
.../aws/transfers/google_api_to_s3}/index.html | 715 +-
.../aws/transfers/hive_to_dynamodb}/index.html | 696 +-
.../transfers/imap_attachment_to_s3}/index.html | 676 +-
.../providers/amazon/aws/transfers}/index.html | 638 +-
.../amazon/aws/transfers/local_to_s3}/index.html | 696 +-
.../amazon/aws/transfers/mongo_to_s3}/index.html | 704 +-
.../aws/transfers/redshift_to_s3}/index.html | 722 +-
.../amazon/aws/transfers/s3_to_ftp}/index.html | 673 +-
.../aws/transfers/s3_to_redshift}/index.html | 727 +-
.../amazon/aws/transfers/s3_to_sftp}/index.html | 682 +-
.../amazon/aws/transfers/s3_to_sql}/index.html | 707 +-
.../aws/transfers/salesforce_to_s3}/index.html | 697 +-
.../amazon/aws/transfers/sftp_to_s3}/index.html | 686 +-
.../amazon/aws/transfers/sql_to_s3}/index.html | 761 +-
.../providers/amazon/aws/triggers}/index.html | 620 +-
.../aws/triggers/redshift_cluster}/index.html | 737 +-
.../aws/utils/connection_wrapper}/index.html | 819 +-
.../amazon/aws/utils/eks_get_token}/index.html | 673 +-
.../providers/amazon/aws/utils/emailer}/index.html | 640 +-
.../airflow/providers/amazon/aws/utils}/index.html | 693 +-
.../providers/amazon/aws/utils/rds}/index.html | 652 +-
.../amazon/aws/utils/redshift}/index.html | 669 +-
.../amazon/aws/utils/sagemaker}/index.html | 658 +-
.../providers/amazon/aws/utils/tags}/index.html | 647 +-
.../providers/amazon/aws/utils/waiter}/index.html | 689 +-
.../amazon/aws/waiters/base_waiter}/index.html | 650 +-
.../providers/amazon/aws/waiters}/index.html | 620 +-
.../_api/airflow/providers/amazon}/index.html | 757 +-
.../amazon/aws/example_appflow}/index.html | 641 +-
.../amazon/aws/example_athena}/index.html | 711 +-
.../providers/amazon/aws/example_batch}/index.html | 770 +-
.../amazon/aws/example_cloudformation}/index.html | 647 +-
.../amazon/aws/example_datasync}/index.html | 761 +-
.../providers/amazon/aws/example_dms}/index.html | 800 +-
.../amazon/aws/example_dynamodb}/index.html | 720 +-
.../amazon/aws/example_dynamodb_to_s3}/index.html | 738 +-
.../providers/amazon/aws/example_ec2}/index.html | 714 +-
.../providers/amazon/aws/example_ecs}/index.html | 698 +-
.../amazon/aws/example_ecs_fargate}/index.html | 737 +-
.../amazon/aws/example_eks_templated}/index.html | 641 +-
.../index.html | 659 +-
.../example_eks_with_fargate_profile}/index.html | 665 +-
.../index.html | 711 +-
.../aws/example_eks_with_nodegroups}/index.html | 711 +-
.../providers/amazon/aws/example_emr}/index.html | 759 +-
.../amazon/aws/example_emr_eks}/index.html | 768 +-
.../aws/example_emr_notebook_execution}/index.html | 653 +-
.../amazon/aws/example_emr_serverless}/index.html | 647 +-
.../amazon/aws/example_ftp_to_s3}/index.html | 641 +-
.../amazon/aws/example_gcs_to_s3}/index.html | 641 +-
.../amazon/aws/example_glacier_to_gcs}/index.html | 689 +-
.../providers/amazon/aws/example_glue}/index.html | 736 +-
.../example_google_api_sheets_to_s3}/index.html | 661 +-
.../example_google_api_youtube_to_s3}/index.html | 777 +-
.../aws/example_hive_to_dynamodb}/index.html | 748 +-
.../aws/example_imap_attachment_to_s3}/index.html | 655 +-
.../amazon/aws/example_lambda}/index.html | 715 +-
.../amazon/aws/example_local_to_s3}/index.html | 707 +-
.../amazon/aws/example_mongo_to_s3}/index.html | 653 +-
.../amazon/aws/example_quicksight}/index.html | 760 +-
.../amazon/aws/example_rds_event}/index.html | 693 +-
.../amazon/aws/example_rds_export}/index.html | 700 +-
.../amazon/aws/example_rds_instance}/index.html | 653 +-
.../amazon/aws/example_rds_snapshot}/index.html | 641 +-
.../amazon/aws/example_redshift}/index.html | 749 +-
.../aws/example_redshift_s3_transfers}/index.html | 812 +-
.../providers/amazon/aws/example_s3}/index.html | 676 +-
.../amazon/aws/example_s3_to_ftp}/index.html | 641 +-
.../amazon/aws/example_s3_to_sftp}/index.html | 641 +-
.../amazon/aws/example_s3_to_sql}/index.html | 771 +-
.../amazon/aws/example_sagemaker}/index.html | 801 +-
.../aws/example_sagemaker_endpoint}/index.html | 757 +-
.../aws/example_salesforce_to_s3}/index.html | 643 +-
.../amazon/aws/example_sftp_to_s3}/index.html | 641 +-
.../providers/amazon/aws/example_sns}/index.html | 693 +-
.../amazon/aws/example_sql_to_s3}/index.html | 776 +-
.../providers/amazon/aws/example_sqs}/index.html | 691 +-
.../amazon/aws/example_step_functions}/index.html | 707 +-
.../tests/system/providers/amazon/aws}/index.html | 671 +-
.../providers/amazon/aws/utils/ec2}/index.html | 728 +-
.../system/providers/amazon/aws/utils}/index.html | 903 +-
.../_api/tests/system/providers/amazon}/index.html | 668 +-
.../8.0.0/_images/aws-base-conn-airflow.png | Bin 0 -> 186727 bytes
.../8.0.0/_images/aws-secrets-manager-json.png | Bin 0 -> 86477 bytes
.../8.0.0/_images/aws-secrets-manager-uri.png | Bin 0 -> 60559 bytes
.../_images/aws-web-identity-federation-gcp.png | Bin 0 -> 40170 bytes
.../airflow/providers/amazon/aws/exceptions.html} | 682 +-
.../providers/amazon/aws/hooks/appflow.html} | 750 +-
.../providers/amazon/aws/hooks/athena.html} | 941 +-
.../providers/amazon/aws/hooks/base_aws.html | 1891 ++++
.../providers/amazon/aws/hooks/batch_client.html} | 1211 ++-
.../providers/amazon/aws/hooks/batch_waiters.html} | 857 +-
.../amazon/aws/hooks/cloud_formation.html} | 721 +-
.../providers/amazon/aws/hooks/datasync.html} | 967 +-
.../airflow/providers/amazon/aws/hooks/dms.html} | 853 +-
.../providers/amazon/aws/hooks/dynamodb.html} | 702 +-
.../airflow/providers/amazon/aws/hooks/ec2.html} | 831 +-
.../airflow/providers/amazon/aws/hooks/ecr.html} | 732 +-
.../airflow/providers/amazon/aws/hooks/ecs.html} | 882 +-
.../airflow/providers/amazon/aws/hooks/eks.html | 1474 +++
.../aws/hooks/elasticache_replication_group.html} | 912 +-
.../airflow/providers/amazon/aws/hooks/emr.html} | 1136 +-
.../providers/amazon/aws/hooks/glacier.html} | 717 +-
.../airflow/providers/amazon/aws/hooks/glue.html} | 990 +-
.../providers/amazon/aws/hooks/glue_catalog.html} | 825 +-
.../providers/amazon/aws/hooks/glue_crawler.html} | 851 +-
.../providers/amazon/aws/hooks/kinesis.html} | 685 +-
.../amazon/aws/hooks/lambda_function.html} | 809 +-
.../airflow/providers/amazon/aws/hooks/logs.html} | 752 +-
.../providers/amazon/aws/hooks/quicksight.html} | 803 +-
.../airflow/providers/amazon/aws/hooks/rds.html} | 984 +-
.../amazon/aws/hooks/redshift_cluster.html} | 933 +-
.../providers/amazon/aws/hooks/redshift_data.html} | 824 +-
.../providers/amazon/aws/hooks/redshift_sql.html} | 761 +-
.../airflow/providers/amazon/aws/hooks/s3.html | 1950 ++++
.../providers/amazon/aws/hooks/sagemaker.html | 2132 ++++
.../amazon/aws/hooks/secrets_manager.html} | 701 +-
.../airflow/providers/amazon/aws/hooks/ses.html} | 735 +-
.../airflow/providers/amazon/aws/hooks/sns.html} | 727 +-
.../airflow/providers/amazon/aws/hooks/sqs.html} | 715 +-
.../airflow/providers/amazon/aws/hooks/ssm.html} | 693 +-
.../providers/amazon/aws/hooks/step_function.html} | 713 +-
.../airflow/providers/amazon/aws/hooks/sts.html} | 679 +-
.../providers/amazon/aws/links/base_aws.html} | 727 +-
.../airflow/providers/amazon/aws/links/batch.html} | 677 +-
.../airflow/providers/amazon/aws/links/emr.html} | 667 +-
.../airflow/providers/amazon/aws/links/glue.html} | 661 +-
.../airflow/providers/amazon/aws/links/logs.html} | 674 +-
.../amazon/aws/log/cloudwatch_task_handler.html} | 756 +-
.../providers/amazon/aws/log/s3_task_handler.html} | 858 +-
.../providers/amazon/aws/operators/appflow.html | 1338 +++
.../providers/amazon/aws/operators/athena.html} | 774 +-
.../providers/amazon/aws/operators/batch.html} | 1049 +-
.../amazon/aws/operators/cloud_formation.html} | 730 +-
.../providers/amazon/aws/operators/datasync.html} | 1034 +-
.../providers/amazon/aws/operators/dms.html} | 921 +-
.../providers/amazon/aws/operators/ec2.html} | 888 +-
.../providers/amazon/aws/operators/ecs.html | 1524 +++
.../providers/amazon/aws/operators/eks.html | 1555 +++
.../providers/amazon/aws/operators/emr.html | 1992 ++++
.../providers/amazon/aws/operators/glacier.html} | 738 +-
.../providers/amazon/aws/operators/glue.html} | 810 +-
.../amazon/aws/operators/glue_crawler.html} | 720 +-
.../amazon/aws/operators/lambda_function.html} | 828 +-
.../amazon/aws/operators/quicksight.html} | 731 +-
.../providers/amazon/aws/operators/rds.html | 1569 +++
.../amazon/aws/operators/redshift_cluster.html | 1481 +++
.../amazon/aws/operators/redshift_data.html} | 779 +-
.../airflow/providers/amazon/aws/operators/s3.html | 1590 +++
.../providers/amazon/aws/operators/sagemaker.html | 1965 ++++
.../providers/amazon/aws/operators/sns.html} | 716 +-
.../providers/amazon/aws/operators/sqs.html} | 728 +-
.../amazon/aws/operators/step_function.html} | 752 +-
.../amazon/aws/secrets/secrets_manager.html} | 980 +-
.../amazon/aws/secrets/systems_manager.html} | 832 +-
.../providers/amazon/aws/sensors/athena.html} | 723 +-
.../providers/amazon/aws/sensors/batch.html} | 848 +-
.../amazon/aws/sensors/cloud_formation.html} | 741 +-
.../airflow/providers/amazon/aws/sensors/dms.html} | 751 +-
.../providers/amazon/aws/sensors/dynamodb.html} | 729 +-
.../airflow/providers/amazon/aws/sensors/ec2.html} | 705 +-
.../airflow/providers/amazon/aws/sensors/ecs.html} | 820 +-
.../airflow/providers/amazon/aws/sensors/eks.html} | 886 +-
.../airflow/providers/amazon/aws/sensors/emr.html | 1359 +++
.../providers/amazon/aws/sensors/glacier.html} | 735 +-
.../providers/amazon/aws/sensors/glue.html} | 724 +-
.../aws/sensors/glue_catalog_partition.html} | 727 +-
.../amazon/aws/sensors/glue_crawler.html} | 708 +-
.../amazon/aws/sensors/lambda_function.html} | 718 +-
.../providers/amazon/aws/sensors/quicksight.html} | 722 +-
.../airflow/providers/amazon/aws/sensors/rds.html} | 825 +-
.../amazon/aws/sensors/redshift_cluster.html} | 707 +-
.../airflow/providers/amazon/aws/sensors/s3.html} | 915 +-
.../providers/amazon/aws/sensors/sagemaker.html} | 977 +-
.../airflow/providers/amazon/aws/sensors/sqs.html} | 859 +-
.../amazon/aws/sensors/step_function.html} | 729 +-
.../providers/amazon/aws/transfers/base.html} | 701 +-
.../amazon/aws/transfers/dynamodb_to_s3.html} | 789 +-
.../amazon/aws/transfers/exasol_to_s3.html} | 742 +-
.../providers/amazon/aws/transfers/ftp_to_s3.html} | 784 +-
.../providers/amazon/aws/transfers/gcs_to_s3.html} | 809 +-
.../amazon/aws/transfers/glacier_to_gcs.html} | 742 +-
.../amazon/aws/transfers/google_api_to_s3.html} | 825 +-
.../amazon/aws/transfers/hive_to_dynamodb.html} | 742 +-
.../aws/transfers/imap_attachment_to_s3.html} | 737 +-
.../amazon/aws/transfers/local_to_s3.html} | 740 +-
.../amazon/aws/transfers/mongo_to_s3.html} | 778 +-
.../amazon/aws/transfers/redshift_to_s3.html} | 813 +-
.../providers/amazon/aws/transfers/s3_to_ftp.html} | 712 +-
.../amazon/aws/transfers/s3_to_redshift.html} | 821 +-
.../amazon/aws/transfers/s3_to_sftp.html} | 718 +-
.../providers/amazon/aws/transfers/s3_to_sql.html} | 760 +-
.../amazon/aws/transfers/salesforce_to_s3.html} | 765 +-
.../amazon/aws/transfers/sftp_to_s3.html} | 728 +-
.../providers/amazon/aws/transfers/sql_to_s3.html} | 833 +-
.../amazon/aws/triggers/redshift_cluster.html} | 771 +-
.../airflow/providers/amazon/aws/utils.html} | 696 +-
.../amazon/aws/utils/connection_wrapper.html | 1315 +++
.../providers/amazon/aws/utils/eks_get_token.html} | 703 +-
.../providers/amazon/aws/utils/emailer.html} | 689 +-
.../airflow/providers/amazon/aws/utils/rds.html} | 660 +-
.../providers/amazon/aws/utils/redshift.html} | 684 +-
.../providers/amazon/aws/utils/sagemaker.html} | 662 +-
.../airflow/providers/amazon/aws/utils/tags.html} | 672 +-
.../providers/amazon/aws/utils/waiter.html} | 720 +-
.../providers/amazon/aws/waiters/base_waiter.html} | 679 +-
.../8.0.0/_modules}/index.html | 806 +-
.../providers/amazon/aws/example_appflow.html} | 764 +-
.../providers/amazon/aws/example_athena.html} | 818 +-
.../providers/amazon/aws/example_batch.html} | 903 +-
.../amazon/aws/example_cloudformation.html} | 748 +-
.../providers/amazon/aws/example_datasync.html} | 878 +-
.../system/providers/amazon/aws/example_dms.html} | 1064 +-
.../providers/amazon/aws/example_dynamodb.html} | 745 +-
.../amazon/aws/example_dynamodb_to_s3.html} | 794 +-
.../system/providers/amazon/aws/example_ec2.html} | 807 +-
.../system/providers/amazon/aws/example_ecs.html} | 858 +-
.../providers/amazon/aws/example_ecs_fargate.html} | 777 +-
.../amazon/aws/example_eks_templated.html} | 794 +-
.../aws/example_eks_with_fargate_in_one_step.html} | 792 +-
.../aws/example_eks_with_fargate_profile.html} | 828 +-
.../example_eks_with_nodegroup_in_one_step.html} | 806 +-
.../amazon/aws/example_eks_with_nodegroups.html} | 851 +-
.../system/providers/amazon/aws/example_emr.html} | 857 +-
.../providers/amazon/aws/example_emr_eks.html} | 933 +-
.../aws/example_emr_notebook_execution.html} | 755 +-
.../amazon/aws/example_emr_serverless.html} | 795 +-
.../providers/amazon/aws/example_ftp_to_s3.html} | 715 +-
.../providers/amazon/aws/example_gcs_to_s3.html} | 714 +-
.../amazon/aws/example_glacier_to_gcs.html} | 751 +-
.../system/providers/amazon/aws/example_glue.html} | 843 +-
.../aws/example_google_api_sheets_to_s3.html} | 725 +-
.../aws/example_google_api_youtube_to_s3.html} | 836 +-
.../amazon/aws/example_hive_to_dynamodb.html} | 790 +-
.../amazon/aws/example_imap_attachment_to_s3.html} | 732 +-
.../providers/amazon/aws/example_lambda.html} | 770 +-
.../providers/amazon/aws/example_local_to_s3.html} | 735 +-
.../providers/amazon/aws/example_mongo_to_s3.html} | 726 +-
.../providers/amazon/aws/example_quicksight.html} | 860 +-
.../providers/amazon/aws/example_rds_event.html} | 763 +-
.../providers/amazon/aws/example_rds_export.html} | 820 +-
.../amazon/aws/example_rds_instance.html} | 755 +-
.../amazon/aws/example_rds_snapshot.html} | 782 +-
.../providers/amazon/aws/example_redshift.html} | 916 +-
.../amazon/aws/example_redshift_s3_transfers.html} | 910 +-
.../system/providers/amazon/aws/example_s3.html} | 909 +-
.../providers/amazon/aws/example_s3_to_ftp.html} | 714 +-
.../providers/amazon/aws/example_s3_to_sftp.html} | 714 +-
.../providers/amazon/aws/example_s3_to_sql.html} | 913 +-
.../providers/amazon/aws/example_sagemaker.html | 1536 +++
.../amazon/aws/example_sagemaker_endpoint.html} | 928 +-
.../amazon/aws/example_salesforce_to_s3.html} | 720 +-
.../providers/amazon/aws/example_sftp_to_s3.html} | 714 +-
.../system/providers/amazon/aws/example_sns.html} | 719 +-
.../providers/amazon/aws/example_sql_to_s3.html} | 864 +-
.../system/providers/amazon/aws/example_sqs.html} | 744 +-
.../amazon/aws/example_step_functions.html} | 754 +-
.../tests/system/providers/amazon/aws/utils.html} | 974 +-
.../system/providers/amazon/aws/utils/ec2.html} | 787 +-
.../8.0.0/_partials/prerequisite_tasks.html} | 616 +-
.../providers/amazon/aws/exceptions/index.rst.txt | 40 +
.../amazon/aws/hooks/appflow/index.rst.txt | 60 +
.../amazon/aws/hooks/athena/index.rst.txt | 158 +
.../amazon/aws/hooks/base_aws/index.rst.txt | 384 +
.../amazon/aws/hooks/batch_client/index.rst.txt | 412 +
.../amazon/aws/hooks/batch_waiters/index.rst.txt | 179 +
.../amazon/aws/hooks/cloud_formation/index.rst.txt | 70 +
.../amazon/aws/hooks/datasync/index.rst.txt | 199 +
.../providers/amazon/aws/hooks/dms/index.rst.txt | 149 +
.../amazon/aws/hooks/dynamodb/index.rst.txt | 54 +
.../providers/amazon/aws/hooks/ec2/index.rst.txt | 130 +
.../providers/amazon/aws/hooks/ecr/index.rst.txt | 94 +
.../providers/amazon/aws/hooks/ecs/index.rst.txt | 272 +
.../providers/amazon/aws/hooks/eks/index.rst.txt | 405 +
.../elasticache_replication_group/index.rst.txt | 139 +
.../providers/amazon/aws/hooks/emr/index.rst.txt | 274 +
.../amazon/aws/hooks/glacier/index.rst.txt | 66 +
.../providers/amazon/aws/hooks/glue/index.rst.txt | 177 +
.../amazon/aws/hooks/glue_catalog/index.rst.txt | 143 +
.../amazon/aws/hooks/glue_crawler/index.rst.txt | 114 +
.../providers/amazon/aws/hooks/index.rst.txt | 50 +
.../amazon/aws/hooks/kinesis/index.rst.txt | 50 +
.../amazon/aws/hooks/lambda_function/index.rst.txt | 100 +
.../providers/amazon/aws/hooks/logs/index.rst.txt | 73 +
.../amazon/aws/hooks/quicksight/index.rst.txt | 98 +
.../providers/amazon/aws/hooks/rds/index.rst.txt | 195 +
.../aws/hooks/redshift_cluster/index.rst.txt | 172 +
.../amazon/aws/hooks/redshift_data/index.rst.txt | 77 +
.../amazon/aws/hooks/redshift_sql/index.rst.txt | 91 +
.../providers/amazon/aws/hooks/s3/index.rst.txt | 532 +
.../amazon/aws/hooks/sagemaker/index.rst.txt | 611 ++
.../amazon/aws/hooks/secrets_manager/index.rst.txt | 54 +
.../providers/amazon/aws/hooks/ses/index.rst.txt | 63 +
.../providers/amazon/aws/hooks/sns/index.rst.txt | 59 +
.../providers/amazon/aws/hooks/sqs/index.rst.txt | 65 +
.../providers/amazon/aws/hooks/ssm/index.rst.txt | 45 +
.../amazon/aws/hooks/step_function/index.rst.txt | 57 +
.../providers/amazon/aws/hooks/sts/index.rst.txt | 41 +
.../airflow/providers/amazon/aws/index.rst.txt | 33 +
.../amazon/aws/links/base_aws/index.rst.txt | 81 +
.../providers/amazon/aws/links/batch/index.rst.txt | 84 +
.../providers/amazon/aws/links/emr/index.rst.txt | 62 +
.../providers/amazon/aws/links/glue/index.rst.txt | 40 +
.../providers/amazon/aws/links/index.rst.txt | 19 +
.../providers/amazon/aws/links/logs/index.rst.txt | 48 +
.../aws/log/cloudwatch_task_handler/index.rst.txt | 69 +
.../airflow/providers/amazon/aws/log/index.rst.txt | 16 +
.../amazon/aws/log/s3_task_handler/index.rst.txt | 102 +
.../amazon/aws/operators/appflow/index.rst.txt | 228 +
.../amazon/aws/operators/athena/index.rst.txt | 80 +
.../amazon/aws/operators/batch/index.rst.txt | 193 +
.../aws/operators/cloud_formation/index.rst.txt | 111 +
.../amazon/aws/operators/datasync/index.rst.txt | 143 +
.../amazon/aws/operators/dms/index.rst.txt | 240 +
.../amazon/aws/operators/ec2/index.rst.txt | 177 +
.../amazon/aws/operators/ecs/index.rst.txt | 280 +
.../amazon/aws/operators/eks/index.rst.txt | 437 +
.../amazon/aws/operators/emr/index.rst.txt | 599 +
.../amazon/aws/operators/glacier/index.rst.txt | 82 +
.../amazon/aws/operators/glue/index.rst.txt | 81 +
.../aws/operators/glue_crawler/index.rst.txt | 60 +
.../providers/amazon/aws/operators/index.rst.txt | 37 +
.../aws/operators/lambda_function/index.rst.txt | 112 +
.../amazon/aws/operators/quicksight/index.rst.txt | 79 +
.../amazon/aws/operators/rds/index.rst.txt | 340 +
.../aws/operators/redshift_cluster/index.rst.txt | 325 +
.../aws/operators/redshift_data/index.rst.txt | 79 +
.../amazon/aws/operators/s3/index.rst.txt | 577 +
.../amazon/aws/operators/sagemaker/index.rst.txt | 662 ++
.../amazon/aws/operators/sns/index.rst.txt | 66 +
.../amazon/aws/operators/sqs/index.rst.txt | 68 +
.../aws/operators/step_function/index.rst.txt | 105 +
.../providers/amazon/aws/secrets/index.rst.txt | 16 +
.../aws/secrets/secrets_manager/index.rst.txt | 136 +
.../aws/secrets/systems_manager/index.rst.txt | 104 +
.../amazon/aws/sensors/athena/index.rst.txt | 81 +
.../amazon/aws/sensors/batch/index.rst.txt | 160 +
.../aws/sensors/cloud_formation/index.rst.txt | 99 +
.../providers/amazon/aws/sensors/dms/index.rst.txt | 86 +
.../amazon/aws/sensors/dynamodb/index.rst.txt | 43 +
.../providers/amazon/aws/sensors/ec2/index.rst.txt | 64 +
.../providers/amazon/aws/sensors/ecs/index.rst.txt | 138 +
.../providers/amazon/aws/sensors/eks/index.rst.txt | 228 +
.../providers/amazon/aws/sensors/emr/index.rst.txt | 418 +
.../amazon/aws/sensors/glacier/index.rst.txt | 80 +
.../amazon/aws/sensors/glue/index.rst.txt | 49 +
.../sensors/glue_catalog_partition/index.rst.txt | 66 +
.../amazon/aws/sensors/glue_crawler/index.rst.txt | 53 +
.../providers/amazon/aws/sensors/index.rst.txt | 35 +
.../aws/sensors/lambda_function/index.rst.txt | 55 +
.../amazon/aws/sensors/quicksight/index.rst.txt | 58 +
.../providers/amazon/aws/sensors/rds/index.rst.txt | 98 +
.../aws/sensors/redshift_cluster/index.rst.txt | 52 +
.../providers/amazon/aws/sensors/s3/index.rst.txt | 143 +
.../amazon/aws/sensors/sagemaker/index.rst.txt | 374 +
.../providers/amazon/aws/sensors/sqs/index.rst.txt | 103 +
.../amazon/aws/sensors/step_function/index.rst.txt | 83 +
.../amazon/aws/transfers/base/index.rst.txt | 47 +
.../aws/transfers/dynamodb_to_s3/index.rst.txt | 80 +
.../aws/transfers/exasol_to_s3/index.rst.txt | 76 +
.../amazon/aws/transfers/ftp_to_s3/index.rst.txt | 69 +
.../amazon/aws/transfers/gcs_to_s3/index.rst.txt | 94 +
.../aws/transfers/glacier_to_gcs/index.rst.txt | 65 +
.../aws/transfers/google_api_to_s3/index.rst.txt | 105 +
.../aws/transfers/hive_to_dynamodb/index.rst.txt | 78 +
.../transfers/imap_attachment_to_s3/index.rst.txt | 59 +
.../providers/amazon/aws/transfers/index.rst.txt | 33 +
.../amazon/aws/transfers/local_to_s3/index.rst.txt | 73 +
.../amazon/aws/transfers/mongo_to_s3/index.rst.txt | 80 +
.../aws/transfers/redshift_to_s3/index.rst.txt | 98 +
.../amazon/aws/transfers/s3_to_ftp/index.rst.txt | 54 +
.../aws/transfers/s3_to_redshift/index.rst.txt | 95 +
.../amazon/aws/transfers/s3_to_sftp/index.rst.txt | 60 +
.../amazon/aws/transfers/s3_to_sql/index.rst.txt | 80 +
.../aws/transfers/salesforce_to_s3/index.rst.txt | 76 +
.../amazon/aws/transfers/sftp_to_s3/index.rst.txt | 64 +
.../amazon/aws/transfers/sql_to_s3/index.rst.txt | 112 +
.../providers/amazon/aws/triggers/index.rst.txt | 15 +
.../aws/triggers/redshift_cluster/index.rst.txt | 94 +
.../aws/utils/connection_wrapper/index.rst.txt | 163 +
.../amazon/aws/utils/eks_get_token/index.rst.txt | 43 +
.../amazon/aws/utils/emailer/index.rst.txt | 29 +
.../providers/amazon/aws/utils/index.rst.txt | 60 +
.../providers/amazon/aws/utils/rds/index.rst.txt | 38 +
.../amazon/aws/utils/redshift/index.rst.txt | 41 +
.../amazon/aws/utils/sagemaker/index.rst.txt | 41 +
.../providers/amazon/aws/utils/tags/index.rst.txt | 31 +
.../amazon/aws/utils/waiter/index.rst.txt | 58 +
.../amazon/aws/waiters/base_waiter/index.rst.txt | 29 +
.../providers/amazon/aws/waiters/index.rst.txt | 15 +
.../_api/airflow/providers/amazon/index.rst.txt | 15 +
.../amazon/aws/example_appflow/index.rst.txt | 26 +
.../amazon/aws/example_athena/index.rst.txt | 81 +
.../amazon/aws/example_batch/index.rst.txt | 98 +
.../aws/example_cloudformation/index.rst.txt | 30 +
.../amazon/aws/example_datasync/index.rst.txt | 88 +
.../providers/amazon/aws/example_dms/index.rst.txt | 119 +
.../amazon/aws/example_dynamodb/index.rst.txt | 69 +
.../aws/example_dynamodb_to_s3/index.rst.txt | 79 +
.../providers/amazon/aws/example_ec2/index.rst.txt | 64 +
.../providers/amazon/aws/example_ecs/index.rst.txt | 62 +
.../amazon/aws/example_ecs_fargate/index.rst.txt | 82 +
.../amazon/aws/example_eks_templated/index.rst.txt | 26 +
.../index.rst.txt | 41 +
.../example_eks_with_fargate_profile/index.rst.txt | 45 +
.../index.rst.txt | 66 +
.../aws/example_eks_with_nodegroups/index.rst.txt | 66 +
.../providers/amazon/aws/example_emr/index.rst.txt | 91 +
.../amazon/aws/example_emr_eks/index.rst.txt | 117 +
.../example_emr_notebook_execution/index.rst.txt | 36 +
.../aws/example_emr_serverless/index.rst.txt | 31 +
.../amazon/aws/example_ftp_to_s3/index.rst.txt | 26 +
.../amazon/aws/example_gcs_to_s3/index.rst.txt | 26 +
.../aws/example_glacier_to_gcs/index.rst.txt | 54 +
.../amazon/aws/example_glue/index.rst.txt | 110 +
.../example_google_api_sheets_to_s3/index.rst.txt | 44 +
.../example_google_api_youtube_to_s3/index.rst.txt | 127 +
.../aws/example_hive_to_dynamodb/index.rst.txt | 96 +
.../example_imap_attachment_to_s3/index.rst.txt | 42 +
.../amazon/aws/example_lambda/index.rst.txt | 81 +
.../amazon/aws/example_local_to_s3/index.rst.txt | 66 +
.../amazon/aws/example_mongo_to_s3/index.rst.txt | 36 +
.../amazon/aws/example_quicksight/index.rst.txt | 98 +
.../amazon/aws/example_rds_event/index.rst.txt | 54 +
.../amazon/aws/example_rds_export/index.rst.txt | 62 +
.../amazon/aws/example_rds_instance/index.rst.txt | 36 +
.../amazon/aws/example_rds_snapshot/index.rst.txt | 26 +
.../amazon/aws/example_redshift/index.rst.txt | 87 +
.../example_redshift_s3_transfers/index.rst.txt | 126 +
.../providers/amazon/aws/example_s3/index.rst.txt | 67 +
.../amazon/aws/example_s3_to_ftp/index.rst.txt | 26 +
.../amazon/aws/example_s3_to_sftp/index.rst.txt | 26 +
.../amazon/aws/example_s3_to_sql/index.rst.txt | 114 +
.../amazon/aws/example_sagemaker/index.rst.txt | 138 +
.../aws/example_sagemaker_endpoint/index.rst.txt | 105 +
.../aws/example_salesforce_to_s3/index.rst.txt | 32 +
.../amazon/aws/example_sftp_to_s3/index.rst.txt | 26 +
.../providers/amazon/aws/example_sns/index.rst.txt | 54 +
.../amazon/aws/example_sql_to_s3/index.rst.txt | 102 +
.../providers/amazon/aws/example_sqs/index.rst.txt | 54 +
.../aws/example_step_functions/index.rst.txt | 65 +
.../system/providers/amazon/aws/index.rst.txt | 73 +
.../providers/amazon/aws/utils/ec2/index.rst.txt | 65 +
.../providers/amazon/aws/utils/index.rst.txt | 203 +
.../tests/system/providers/amazon/index.rst.txt | 15 +
.../_sources/_partials/prerequisite_tasks.rst.txt | 34 +
.../8.0.0/_sources/commits.rst.txt | 1213 +++
.../8.0.0/_sources/connections/aws.rst.txt | 820 ++
.../8.0.0/_sources/connections/emr.rst.txt | 43 +
.../8.0.0/_sources/connections/index.rst.txt | 28 +
.../8.0.0/_sources/connections/redshift.rst.txt | 91 +
.../8.0.0/_sources/deferrable.rst.txt | 33 +
.../8.0.0/_sources/example-dags.rst.txt | 23 +
.../8.0.0/_sources/index.rst.txt | 148 +
.../installing-providers-from-sources.rst.txt | 18 +
.../logging/cloud-watch-task-handlers.rst.txt | 38 +
.../8.0.0/_sources/logging/index.rst.txt | 25 +
.../8.0.0/_sources/logging/s3-task-handler.rst.txt | 140 +
.../8.0.0/_sources/operators/appflow.rst.txt | 146 +
.../8.0.0/_sources/operators/athena.rst.txt | 75 +
.../8.0.0/_sources/operators/batch.rst.txt | 111 +
.../_sources/operators/cloudformation.rst.txt | 99 +
.../8.0.0/_sources/operators/datasync.rst.txt | 114 +
.../8.0.0/_sources/operators/dms.rst.txt | 133 +
.../8.0.0/_sources/operators/dynamodb.rst.txt | 55 +
.../8.0.0/_sources/operators/ec2.rst.txt | 109 +
.../8.0.0/_sources/operators/ecs.rst.txt | 262 +
.../8.0.0/_sources/operators/eks.rst.txt | 247 +
.../8.0.0/_sources/operators/emr/emr.rst.txt | 205 +
.../8.0.0/_sources/operators/emr/emr_eks.rst.txt | 124 +
.../_sources/operators/emr/emr_serverless.rst.txt | 127 +
.../8.0.0/_sources/operators/emr/index.rst.txt | 28 +
.../8.0.0/_sources/operators/glue.rst.txt | 107 +
.../8.0.0/_sources/operators/index.rst.txt | 31 +
.../8.0.0/_sources/operators/lambda.rst.txt | 85 +
.../8.0.0/_sources/operators/quicksight.rst.txt | 69 +
.../8.0.0/_sources/operators/rds.rst.txt | 252 +
.../_sources/operators/redshift/index.rst.txt | 28 +
.../operators/redshift/redshift_cluster.rst.txt | 141 +
.../operators/redshift/redshift_data.rst.txt | 55 +
.../operators/redshift/redshift_sql.rst.txt | 53 +
.../8.0.0/_sources/operators/s3/glacier.rst.txt | 83 +
.../8.0.0/_sources/operators/s3/index.rst.txt | 28 +
.../8.0.0/_sources/operators/s3/s3.rst.txt | 271 +
.../8.0.0/_sources/operators/sagemaker.rst.txt | 316 +
.../8.0.0/_sources/operators/sns.rst.txt | 55 +
.../8.0.0/_sources/operators/sqs.rst.txt | 74 +
.../_sources/operators/step_functions.rst.txt | 82 +
.../secrets-backends/aws-secrets-manager.rst.txt | 170 +
.../aws-ssm-parameter-store.rst.txt | 136 +
.../8.0.0/_sources/secrets-backends/index.rst.txt | 27 +
.../8.0.0/_sources/transfer/dynamodb_to_s3.rst.txt | 70 +
.../8.0.0/_sources/transfer/ftp_to_s3.rst.txt | 53 +
.../8.0.0/_sources/transfer/gcs_to_s3.rst.txt | 52 +
.../8.0.0/_sources/transfer/glacier_to_gcs.rst.txt | 55 +
.../_sources/transfer/google_api_to_s3.rst.txt | 81 +
.../_sources/transfer/hive_to_dynamodb.rst.txt | 57 +
.../transfer/imap_attachment_to_s3.rst.txt | 51 +
.../8.0.0/_sources/transfer/index.rst.txt | 28 +
.../8.0.0/_sources/transfer/local_to_s3.rst.txt | 54 +
.../8.0.0/_sources/transfer/mongo_to_s3.rst.txt | 59 +
.../8.0.0/_sources/transfer/redshift_to_s3.rst.txt | 59 +
.../8.0.0/_sources/transfer/s3_to_ftp.rst.txt | 54 +
.../8.0.0/_sources/transfer/s3_to_redshift.rst.txt | 67 +
.../8.0.0/_sources/transfer/s3_to_sftp.rst.txt | 55 +
.../8.0.0/_sources/transfer/s3_to_sql.rst.txt | 65 +
.../_sources/transfer/salesforce_to_s3.rst.txt | 51 +
.../8.0.0/_sources/transfer/sftp_to_s3.rst.txt | 54 +
.../8.0.0/_sources/transfer/sql_to_s3.rst.txt | 69 +
.../8.0.0/_static/_gen/css/main-custom.min.css | 1 +
.../8.0.0/_static/_gen/css/main.min.css | 7 +
.../8.0.0/_static/_gen/js/docs.js | 1 +
.../_sphinx_javascript_frameworks_compat.js | 134 +
.../8.0.0/_static/basic.css | 899 ++
.../8.0.0/_static/check-solid.svg | 4 +
.../8.0.0/_static/clipboard.min.js | 7 +
.../8.0.0/_static/copy-button.svg | 5 +
.../8.0.0/_static/copybutton.css | 94 +
.../8.0.0/_static/copybutton.js | 248 +
.../8.0.0/_static/copybutton_funcs.js | 73 +
.../8.0.0/_static/doctools.js | 156 +
.../8.0.0/_static/documentation_options.js | 14 +
.../8.0.0/_static/file.png | Bin 0 -> 286 bytes
.../8.0.0/_static/graphviz.css | 19 +
.../8.0.0/_static/jquery-3.6.0.js | 10881 +++++++++++++++++++
.../8.0.0/_static/jquery.js | 2 +
.../8.0.0/_static/js/globaltoc.js | 24 +
.../8.0.0/_static/language_data.js | 199 +
.../8.0.0/_static/minus.png | Bin 0 -> 90 bytes
.../8.0.0/_static/pin_32.png | Bin 0 -> 1201 bytes
.../8.0.0/_static/plus.png | Bin 0 -> 90 bytes
.../8.0.0/_static/pygments.css | 74 +
.../8.0.0/_static/searchtools.js | 566 +
.../8.0.0/_static/sphinx_highlight.js | 144 +
.../8.0.0/_static/underscore-1.13.1.js | 2042 ++++
.../8.0.0/_static/underscore.js | 6 +
.../8.0.0/commits.html | 4958 +++++++++
.../8.0.0/connections/aws.html} | 1278 ++-
.../8.0.0/connections/emr.html} | 623 +-
.../8.0.0/connections}/index.html | 616 +-
.../8.0.0/connections/redshift.html} | 657 +-
.../8.0.0/deferrable.html} | 581 +-
.../8.0.0/example-dags.html} | 569 +-
.../8.0.0/genindex.html | 6759 ++++++++++++
.../8.0.0/index.html | 2219 ++++
.../8.0.0/installing-providers-from-sources.html} | 639 +-
.../8.0.0/logging/cloud-watch-task-handlers.html} | 624 +-
.../8.0.0/logging}/index.html | 615 +-
.../8.0.0/logging/s3-task-handler.html} | 690 +-
.../8.0.0/objects.inv | Bin 0 -> 23716 bytes
.../8.0.0/operators/appflow.html} | 723 +-
.../8.0.0/operators/athena.html} | 653 +-
.../8.0.0/operators/batch.html} | 685 +-
.../8.0.0/operators/cloudformation.html} | 668 +-
.../8.0.0/operators/datasync.html} | 709 +-
.../8.0.0/operators/dms.html} | 705 +-
.../8.0.0/operators/dynamodb.html} | 647 +-
.../8.0.0/operators/ec2.html} | 677 +-
.../8.0.0/operators/ecs.html} | 856 +-
.../8.0.0/operators/eks.html} | 873 +-
.../8.0.0/operators/emr.html | 1 +
.../8.0.0/operators/emr/emr.html} | 807 +-
.../8.0.0/operators/emr/emr_eks.html} | 701 +-
.../8.0.0/operators/emr/emr_serverless.html} | 700 +-
.../8.0.0/operators/emr}/index.html | 618 +-
.../8.0.0/operators/emr_eks.html | 1 +
.../8.0.0/operators/emr_serverless.html | 1 +
.../8.0.0/operators/glacier.html | 1 +
.../8.0.0/operators/glue.html} | 675 +-
.../8.0.0/operators/google_api_to_s3_transfer.html | 1 +
.../8.0.0/operators/imap_attachment_to_s3.html | 1 +
.../8.0.0/operators}/index.html | 634 +-
.../8.0.0/operators/lambda.html} | 664 +-
.../8.0.0/operators/quicksight.html} | 649 +-
.../8.0.0/operators/rds.html} | 843 +-
.../8.0.0/operators/redshift}/index.html | 618 +-
.../operators/redshift/redshift_cluster.html} | 719 +-
.../8.0.0/operators/redshift/redshift_data.html} | 655 +-
.../8.0.0/operators/redshift/redshift_sql.html} | 642 +-
.../8.0.0/operators/redshift_cluster.html | 1 +
.../8.0.0/operators/redshift_data.html | 1 +
.../8.0.0/operators/redshift_sql.html | 1 +
.../8.0.0/operators/s3.html | 1 +
.../8.0.0/operators/s3/glacier.html} | 655 +-
.../8.0.0/operators/s3}/index.html | 617 +-
.../8.0.0/operators/s3/s3.html} | 870 +-
.../8.0.0/operators/s3_to_redshift.html | 1 +
.../8.0.0/operators/sagemaker.html} | 935 +-
.../8.0.0/operators/salesforce_to_s3.html | 1 +
.../8.0.0/operators/sns.html} | 640 +-
.../8.0.0/operators/sqs.html} | 667 +-
.../8.0.0/operators/sqs_publish.html | 1 +
.../8.0.0/operators/step_functions.html} | 652 +-
.../operators/transfer/dynamodb_to_s3.html.html | 1 +
.../8.0.0/operators/transfer/ftp_to_s3.html.html | 1 +
.../8.0.0/operators/transfer/gcs_to_s3.html.html | 1 +
.../8.0.0/operators/transfer/glacier_to_gcs.html | 1 +
.../8.0.0/operators/transfer/google_api_to_s3.html | 1 +
.../8.0.0/operators/transfer/hive_to_dynamodb.html | 1 +
.../operators/transfer/imap_attachment_to_s3.html | 1 +
.../8.0.0/operators/transfer/local_to_s3.html | 1 +
.../8.0.0/operators/transfer/mongo_to_s3.html | 1 +
.../8.0.0/operators/transfer/redshift_to_s3.html | 1 +
.../8.0.0/operators/transfer/s3_to_ftp.html | 1 +
.../8.0.0/operators/transfer/s3_to_redshift.html | 1 +
.../8.0.0/operators/transfer/s3_to_sftp.html | 1 +
.../8.0.0/operators/transfer/s3_to_sql.html | 1 +
.../8.0.0/operators/transfer/salesforce_to_s3.html | 1 +
.../8.0.0/operators/transfer/sftp_to_s3.html | 1 +
.../8.0.0/operators/transfer/sql_to_s3.html | 1 +
.../8.0.0/py-modindex.html} | 1543 ++-
.../8.0.0/search.html} | 614 +-
.../8.0.0/searchindex.js | 1 +
.../secrets-backends/aws-secrets-manager.html} | 707 +-
.../secrets-backends/aws-ssm-parameter-store.html} | 681 +-
.../8.0.0/secrets-backends}/index.html | 613 +-
.../8.0.0/transfer/dynamodb_to_s3.html} | 674 +-
.../8.0.0/transfer/ftp_to_s3.html} | 638 +-
.../8.0.0/transfer/gcs_to_s3.html} | 637 +-
.../8.0.0/transfer/glacier_to_gcs.html} | 647 +-
.../8.0.0/transfer/google_api_to_s3.html} | 689 +-
.../8.0.0/transfer/hive_to_dynamodb.html} | 642 +-
.../8.0.0/transfer/imap_attachment_to_s3.html} | 639 +-
.../8.0.0/transfer}/index.html | 630 +-
.../8.0.0/transfer/local_to_s3.html} | 639 +-
.../8.0.0/transfer/mongo_to_s3.html} | 647 +-
.../8.0.0/transfer/redshift_to_s3.html} | 644 +-
.../8.0.0/transfer/s3_to_ftp.html} | 638 +-
.../8.0.0/transfer/s3_to_redshift.html} | 656 +-
.../8.0.0/transfer/s3_to_sftp.html} | 639 +-
.../8.0.0/transfer/s3_to_sql.html} | 676 +-
.../8.0.0/transfer/salesforce_to_s3.html} | 639 +-
.../8.0.0/transfer/sftp_to_s3.html} | 637 +-
.../8.0.0/transfer/sql_to_s3.html} | 658 +-
.../apache-airflow-providers-amazon/stable.txt | 2 +-
.../_sources/index.rst.txt | 46 +-
docs-archive/apache-airflow-providers/index.html | 39 +-
.../apache-airflow-providers/packages-ref.html | 125 +-
.../apache-airflow-providers/searchindex.js | 2 +-
743 files changed, 198387 insertions(+), 197586 deletions(-)
diff --git a/docs-archive/apache-airflow-providers-amazon/8.0.0/.buildinfo b/docs-archive/apache-airflow-providers-amazon/8.0.0/.buildinfo
new file mode 100644
index 0000000000..06828ea508
--- /dev/null
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/.buildinfo
@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: ad02ca7d3a3070d94413a650124ec341
+tags: 645f666f9bcd5a90fca523b33c5a78b7
diff --git a/docs-archive/apache-airflow-providers/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/exceptions/index.html
similarity index 60%
copy from docs-archive/apache-airflow-providers/index.html
copy to docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/exceptions/index.html
index 70eef65adf..eaf400448f 100644
--- a/docs-archive/apache-airflow-providers/index.html
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/exceptions/index.html
@@ -9,18 +9,20 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Provider packages — apache-airflow-providers Documentation</title>
- <link rel="stylesheet" href="_static/_gen/css/main.min.css" type="text/css" />
- <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main.min.css" />
- <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main-custom.min.css" />
- <link rel="shortcut icon" href="_static/pin_32.png"/>
- <link rel="index" title="Index" href="genindex.html" />
- <link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Providers packages reference" href="packages-ref.html" />
-
- <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers/stable/index.html" />
+ <title>airflow.providers.amazon.aws.exceptions — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../search.html" />
+ <link rel="next" title="tests.system.providers.amazon" href="../../../../../tests/system/providers/amazon/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.waiters.base_waiter" href="../waiters/base_waiter/index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/exceptions/index.html" />
<!-- Matomo -->
<script>
@@ -282,8 +284,21 @@
<div class="content-drawer">
<div class="td-sidebar">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -313,14 +328,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -401,8 +434,21 @@
<div class="td-sidebar desktop-only d-print-none">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -432,14 +478,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -528,9 +592,13 @@
<ul class="breadcrumb">
- <li class="breadcrumb-item"><a href="#" class="icon icon-home"> Home</a></li>
+ <li class="breadcrumb-item"><a href="../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
- <li class="breadcrumb-item"><a href="index.html"> Provider packages</a></li>
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.exceptions</span></code></a></li>
</ul>
</div>
@@ -541,440 +609,56 @@
<div class="bodywrapper">
<div class="body" role="main">
- <blockquote>
-<div></div></blockquote>
-<div class="section" id="provider-packages">
-<h1>Provider packages<a class="headerlink" href="#provider-packages" title="Permalink to this heading">¶</a></h1>
-<p>Apache Airflow 2 is built in modular way. The “Core” of Apache Airflow provides core scheduler
-functionality which allow you to write some basic tasks, but the capabilities of Apache Airflow can
-be extended by installing additional packages, called <code class="docutils literal notranslate"><span class="pre">providers</span></code>.</p>
-<p>Providers can contain operators, hooks, sensor, and transfer operators to communicate with a
-multitude of external systems, but they can also extend Airflow core with new capabilities.</p>
-<p>You can install those provider packages separately in order to interface with a given service. The providers
-for <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> are designed in the way that you can write your own providers easily. The
-<code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span> <span class="pre">Community</span></code> develops and maintain more than 60 provider packages, but you are free to
-develop your own providers - the providers you build have exactly the same capability as the providers
-written by the community, so you can release and share those providers with others.</p>
-<p>The full list of community managed providers is available at
-<a class="reference external" href="https://airflow.apache.org/docs/#providers-packages-docs-apache-airflow-providers-index-html">Providers Index</a>.</p>
-<p>You can also see index of all community provider’s operators and hooks in
-<a class="reference internal" href="operators-and-hooks-ref/index.html"><span class="doc">Operators and Hooks Reference</span></a></p>
-</div>
-<div class="section" id="extending-airflow-core-functionality">
-<h1>Extending Airflow core functionality<a class="headerlink" href="#extending-airflow-core-functionality" title="Permalink to this heading">¶</a></h1>
-<p>Providers give you the capability of extending core Airflow with extra capabilities. The Core airflow
-provides basic and solid functionality of scheduling, the providers extend its capabilities. Here we
-describe all the custom capabilities.</p>
-<p>Airflow automatically discovers which providers add those additional capabilities and, once you install
-provider package and re-start Airflow, those become automatically available to Airflow Users.</p>
-<p>The summary of the core functionalities that can be extended are available in
-<a class="reference internal" href="core-extensions/index.html"><span class="doc">Core Extensions</span></a>.</p>
-<div class="section" id="auth-backends">
-<h2>Auth backends<a class="headerlink" href="#auth-backends" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom authentication backends, that allow you to configure the way how your
-web server authenticates your users, integrating it with public or private authentication services.</p>
-<p>You can see all the authentication backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/auth-backends.html"><span class="doc">Auth backends</span></a></p>
-</div>
-<div class="section" id="custom-connections">
-<h2>Custom connections<a class="headerlink" href="#custom-connections" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom connection types, extending connection form and handling custom form field
-behaviour for the connections defined by the provider.</p>
-<p>You can see all custom connections available via community-managed providers in
-<a class="reference internal" href="core-extensions/connections.html"><span class="doc">Connections</span></a>.</p>
-</div>
-<div class="section" id="extra-links">
-<h2>Extra links<a class="headerlink" href="#extra-links" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add extra custom links to operators delivered by the provider. Those will be visible in
-task details view of the task.</p>
-<p>You can see all the extra links available via community-managed providers in
-<a class="reference internal" href="core-extensions/extra-links.html"><span class="doc">Extra Links</span></a>.</p>
-</div>
-<div class="section" id="logging">
-<h2>Logging<a class="headerlink" href="#logging" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add additional task logging capabilities. By default <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> saves logs for
-tasks locally and make them available to Airflow UI via internal http server. However, providers
-can add extra logging capabilities, where Airflow Logs can be written to a remote service and
-retrieved from those services.</p>
-<p>You can see all task loggers available via community-managed providers in
-<a class="reference internal" href="core-extensions/logging.html"><span class="doc">Writing logs</span></a>.</p>
-</div>
-<div class="section" id="secret-backends">
-<h2>Secret backends<a class="headerlink" href="#secret-backends" title="Permalink to this heading">¶</a></h2>
-<p>Airflow has the capability of reading connections, variables and configuration from Secret Backends rather
-than from its own Database.</p>
-<p>You can see all secret backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/secrets-backends.html"><span class="doc">Secret backends</span></a>.</p>
-</div>
-<div class="section" id="notifications">
-<h2>Notifications<a class="headerlink" href="#notifications" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom notifications, that allow you to configure the way how you would like to receive
-notifications about the status of your tasks/DAGs.</p>
-<p>You can see all the notifications available via community-managed providers in
-<a class="reference internal" href="core-extensions/notifications.html"><span class="doc">Notifications</span></a>.</p>
-</div>
-</div>
-<div class="section" id="installing-and-upgrading-providers">
-<h1>Installing and upgrading providers<a class="headerlink" href="#installing-and-upgrading-providers" title="Permalink to this heading">¶</a></h1>
-<p>Separate provider packages give the possibilities that were not available in 1.10:</p>
-<ol class="arabic simple">
-<li><p>You can upgrade to latest version of particular providers without the need of Apache Airflow core upgrade.</p></li>
-<li><p>You can downgrade to previous version of particular provider in case the new version introduces
-some problems, without impacting the main Apache Airflow core package.</p></li>
-<li><p>You can release and upgrade/downgrade provider packages incrementally, independent from each other. This
-means that you can incrementally validate each of the provider package update in your environment,
-following the usual tests you have in your environment.</p></li>
-</ol>
-</div>
-<div class="section" id="types-of-providers">
-<h1>Types of providers<a class="headerlink" href="#types-of-providers" title="Permalink to this heading">¶</a></h1>
-<p>Providers have the same capacity - no matter if they are provided by the community or if they are
-third-party providers. This chapter explains how community managed providers are versioned and released
-and how you can create your own providers.</p>
-<div class="section" id="community-maintained-providers">
-<h2>Community maintained providers<a class="headerlink" href="#community-maintained-providers" title="Permalink to this heading">¶</a></h2>
-<p>From the point of view of the community, Airflow is delivered in multiple, separate packages.
-The core of Airflow scheduling system is delivered as <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package and there are more than
-60 provider packages which can be installed separately as so called <code class="docutils literal notranslate"><span class="pre">Airflow</span> <span class="pre">Provider</span> <span class="pre">packages</span></code>.
-Those packages are available as <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers</span></code> packages - for example there is an
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> or <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> package).</p>
-<p>Community maintained providers are released and versioned separately from the Airflow releases. We are
-following the <a class="reference external" href="https://semver.org/">Semver</a> versioning scheme for the packages. Some versions of the
-provider packages might depend on particular versions of Airflow, but the general approach we have is that
-unless there is a good reason, new version of providers should work with recent versions of Airflow 2.x.
-Details will vary per-provider and if there is a limitation for particular version of particular provider,
-constraining the Airflow version used, it will be included as limitation of dependencies in the provider
-package.</p>
-<p>Each community provider has corresponding extra which can be used when installing airflow to install the
-provider together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> - for example you can install airflow with those extras:
-<code class="docutils literal notranslate"><span class="pre">apache-airflow[google,amazon]</span></code> (with correct constraints -see <a class="reference external" href="/docs/apache-airflow/stable/installation/index.html" title="(in apache-airflow v2.6.0.dev0)"><span>Installation</span></a>) and you
-will install the appropriate versions of the <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> and
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> packages together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>.</p>
-<p>Some of the community providers have cross-provider dependencies as well. Those are not required
-dependencies, they might simply enable certain features (for example transfer operators often create
-dependency between different providers. Again, the general approach here is that the providers are backwards
-compatible, including cross-dependencies. Any kind of breaking changes and requirements on particular versions of other
-provider packages are automatically documented in the release notes of every provider.</p>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>For Airflow 1.10 We also provided <code class="docutils literal notranslate"><span class="pre">apache-airflow-backport-providers</span></code> packages that could be installed
-with those versions Those were the same providers as for 2.0 but automatically back-ported to work for
-Airflow 1.10. The last release of backport providers was done on March 17, 2021 and the backport
-providers will no longer be released, since Airflow 1.10 has reached End-Of-Life as of June 17, 2021.</p>
-</div>
-<p>If you want to contribute to <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>, you can see how to build and extend community
-managed providers in <a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a>.</p>
-</div>
-<div class="section" id="custom-provider-packages">
-<span id="providers-community-maintained-providers"></span><h2>Custom provider packages<a class="headerlink" href="#custom-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p>You can develop and release your own providers. Your custom operators, hooks, sensors, transfer operators
-can be packaged together in a standard airflow package and installed using the same mechanisms.
-Moreover they can also use the same mechanisms to extend the Airflow Core with auth backends,
-custom connections, logging, secret backends and extra operator links as described in the previous chapter.</p>
-</div>
-</div>
-<div class="section" id="how-to-create-your-own-provider">
-<h1>How to create your own provider<a class="headerlink" href="#how-to-create-your-own-provider" title="Permalink to this heading">¶</a></h1>
-<p>As mentioned in the <a class="reference external" href="http://airflow.apache.org/docs/apache-airflow-providers/index.html">Providers</a>
-documentation, custom providers can extend Airflow core - they can add extra links to operators as well
-as custom connections. You can use build your own providers and install them as packages if you would like
-to use the mechanism for your own, custom providers.</p>
-<p>Adding a provider to Airflow is just a matter of building a Python package and adding the right meta-data to
-the package. We are using standard mechanism of python to define
-<a class="reference external" href="https://docs.python.org/3/library/importlib.metadata.html#entry-points">entry points</a> . Your package
-needs to define appropriate entry-point <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> which has to point to a callable
-implemented by your package and return a dictionary containing the list of discoverable capabilities
-of your package. The dictionary has to follow the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json">json-schema specification</a>.</p>
-<p>Most of the schema provides extension point for the documentation (which you might want to also use for
-your own purpose) but the important fields from the extensibility point of view are those:</p>
-<p>Displaying package information in CLI/API:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">package-name</span></code> - Name of the package for the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code> - Human-friendly name of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">description</span></code> - Additional description of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">version</span></code> - List of versions of the package (in reverse-chronological order). The first version in the
-list is the current package version. It is taken from the version of package installed, not from the
-provider_info information.</p></li>
-</ul>
-<p>Exposing customized functionality to the Airflow’s core:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">extra-links</span></code> - this field should contain the list of all operator class names that are adding extra links
-capability. See <a class="reference external" href="/docs/apache-airflow/stable/howto/define-extra-link.html" title="(in apache-airflow v2.6.0.dev0)"><span>Define an operator extra link</span></a> for description of how to add extra link
-capability to the operators of yours.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">connection-types</span></code> - this field should contain the list of all connection types together with hook
-class names implementing those custom connection types (providing custom extra fields and
-custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
-<code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> (deprecated) - this field should contain the list of all hook class names that provide
-custom connection types with custom extra fields and field behaviour. The <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array
-is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
-providers are targeting Airflow 2.2.0+ you do not have to include the <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array, if
-you want to also target earlier versions of Airflow 2, you should include both <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> and
-<code class="docutils literal notranslate"><span class="pre">connection-types</span></code> arrays. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details.</p></li>
-</ul>
-<p>When your providers are installed you can query the installed providers and their capabilities with the
-<code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">providers</span></code> command. This way you can verify if your providers are properly recognized and whether
-they define the extensions properly. See <a class="reference external" href="/docs/apache-airflow/stable/cli-and-env-variables-ref.html" title="(in apache-airflow v2.6.0.dev0)"><span>Command Line Interface and Environment Variables Reference</span></a> for details of available CLI
-sub-commands.</p>
-<p>When you write your own provider, consider following the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#naming-conventions-for-provider-packages">Naming conventions for provider packages</a></p>
-</div>
-<div class="section" id="faq-for-airflow-and-providers">
-<h1>FAQ for Airflow and Providers<a class="headerlink" href="#faq-for-airflow-and-providers" title="Permalink to this heading">¶</a></h1>
-<div class="section" id="upgrading-airflow-2-0-and-providers">
-<h2>Upgrading Airflow 2.0 and Providers<a class="headerlink" href="#upgrading-airflow-2-0-and-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When upgrading to a new Airflow version such as 2.0, but possibly 2.0.1 and beyond, is the best practice
-to also upgrade provider packages at the same time?</strong></p>
-<p>It depends on your use case. If you have automated or semi-automated verification of your installation,
-that you can run a new version of Airflow including all provider packages, then definitely go for it.
-If you rely more on manual testing, it is advised that you upgrade in stages. Depending on your choice
-you can either upgrade all used provider packages first, and then upgrade Airflow Core or the other way
-round. The first approach - when you first upgrade all providers is probably safer, as you can do it
-incrementally, step-by-step replacing provider by provider in your environment.</p>
-</div>
-<div class="section" id="customizing-provider-packages">
-<h2>Customizing Provider Packages<a class="headerlink" href="#customizing-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an older version of my provider package which we have lightly customized and is working
-fine with my MSSQL installation. I am upgrading my Airflow version. Do I need to upgrade my provider,
-or can I keep it as it is?</strong></p>
-<p>It depends on the scope of customization. There is no need to upgrade the provider packages to later
-versions unless you want to upgrade to Airflow version that introduces backwards-incompatible changes.
-Generally speaking, with Airflow 2 we are following the <a class="reference external" href="https://semver.org/">Semver</a> approach where
-we will introduce backwards-incompatible changes in Major releases, so all your modifications (as long
-as you have not used internal Airflow classes) should work for All Airflow 2.* versions.</p>
-</div>
-<div class="section" id="creating-your-own-providers">
-<h2>Creating your own providers<a class="headerlink" href="#creating-your-own-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When I write my own provider, do I need to do anything special to make it available to others?</strong></p>
-<p>You do not need to do anything special besides creating the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point
-returning properly formatted meta-data - dictionary with <code class="docutils literal notranslate"><span class="pre">extra-links</span></code> and <code class="docutils literal notranslate"><span class="pre">connection-types</span></code> fields
-(and deprecated <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> field if you are also targeting versions of Airflow before 2.2.0).</p>
-<p>Anyone who runs airflow in an environment that has your Python package installed will be able to use the
-package as a provider package.</p>
-<p><strong>What do I need to do to turn a package into a provider?</strong></p>
-<p>You need to do the following to turn an existing Python package into a provider (see below for examples):</p>
-<ul class="simple">
-<li><p>Add the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point in the <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> - this tells airflow where to get
-the required provider metadata</p></li>
-<li><p>Create the function that you refer to in the first step as part of your package: this functions returns a
-dictionary that contains all meta-data about your provider package</p></li>
-<li><p>If you want Airflow to link to documentation of your Provider in the providers page, make sure
-to add “project-url/documentation” <a class="reference external" href="https://peps.python.org/pep-0621/#example">metadata</a> to your package.
-This will also add link to your documentation in PyPI.</p></li>
-<li><p>note that the dictionary should be compliant with <code class="docutils literal notranslate"><span class="pre">airflow/provider_info.schema.json</span></code> JSON-schema
-specification. The community-managed providers have more fields there that are used to build
-documentation, but the requirement for runtime information only contains several fields which are defined
-in the schema:</p></li>
-</ul>
-<div class="example-block-wrapper compound">
-<p class="example-header compound-first"><span class="example-title">airflow/provider_info.schema.json</span></p>
-<div class="compound-last highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"$schema"</span><span class="p">:</span><span class="w"> </span><span class="s2">"http://json-schema.org/draft-07/schema#"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"package-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Package name available under which the package is available in the PyPI repository."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Provider name"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Information about the package in RST format"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-names"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class names that provide connection types to core (deprecated by connection-types)"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"deprecated"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"The hook-class-names property has been deprecated in favour of connection-types which is more performant version allowing to only import individual Hooks rather than all hooks at once"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"deprecatedVersion"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.2.0"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"connection-types"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Map of connection types mapped to hook class names."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"connection-type"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Type of connection defined by the provider"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class name that implements the connection type"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"connection-type"</span><span class="p">,</span><span class="w"> </span><span class="s2">"hook-class-name"</span><span class="p">]</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"extra-links"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Operator class names that provide extra link functionality"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"secrets-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Secrets Backend class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"logging"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Logging Task Handlers class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"auth-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"API Auth Backend module names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"notifications"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Notification class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"task-decorators"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Apply custom decorators to the TaskFlow API. Can be accessed by users via '@task.<name>'"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"path"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
-<span class="w"> </span><span class="s2">"name"</span><span class="p">,</span>
-<span class="w"> </span><span class="s2">"description"</span>
-<span class="w"> </span><span class="p">]</span>
-<span class="p">}</span>
-</pre></div>
-</div>
-</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>:</p>
-<div class="highlight-cfg notranslate"><div class="highlight"><pre><span></span><span class="k">[options.entry_points]</span>
-<span class="c1"># the function get_provider_info is defined in myproviderpackage.somemodule</span>
-<span class="na">apache_airflow_provider</span><span class="o">=</span>
-<span class="w"> </span><span class="na">provider_info</span><span class="o">=</span><span class="s">myproviderpackage.somemodule:get_provider_info</span>
-</pre></div>
-</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">myproviderpackage/somemodule.py</span></code>:</p>
-<div class="highlight-Python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_provider_info</span><span class="p">():</span>
- <span class="k">return</span> <span class="p">{</span>
- <span class="s2">"package-name"</span><span class="p">:</span> <span class="s2">"my-package-name"</span><span class="p">,</span>
- <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"name"</span><span class="p">,</span>
- <span class="s2">"description"</span><span class="p">:</span> <span class="s2">"a description"</span><span class="p">,</span>
- <span class="s2">"hook-class-names"</span><span class="p">:</span> <span class="p">[</span>
- <span class="s2">"myproviderpackage.hooks.source.SourceHook"</span><span class="p">,</span>
- <span class="p">],</span>
- <span class="p">}</span>
-</pre></div>
-</div>
-<p><strong>How do provider packages work under the hood?</strong></p>
-<p>When running Airflow with your provider package, there will be (at least) three components to your airflow installation:</p>
-<ul class="simple">
-<li><p>The installation itself (for example, a <code class="docutils literal notranslate"><span class="pre">venv</span></code> where you installed airflow with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">apache-airflow</span></code>)
-together with the related files (e.g. <code class="docutils literal notranslate"><span class="pre">dags</span></code> folder)</p></li>
-<li><p>The <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package</p></li>
-<li><p>Your own <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package that is independent of <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> or your airflow installation, which
-can be a local Python package (that you install via <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span> <span class="pre">/path/to/my-package</span></code>), a normal pip package
-(<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">myproviderpackage</span></code>), or any other type of Python package</p></li>
-</ul>
-<p>In the <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package you need to add the entry point and provide the appropriate metadata as described above.
-If you have done that, airflow does the following at runtime:</p>
-<ul class="simple">
-<li><p>Loop through ALL packages installed in your environment / <code class="docutils literal notranslate"><span class="pre">venv</span></code></p></li>
-<li><p>For each package, if the package’s <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> has a section <code class="docutils literal notranslate"><span class="pre">[options.entry_points]</span></code>, and if that section has a value
-for <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code>, then get the value for <code class="docutils literal notranslate"><span class="pre">provider_info</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code></p></li>
-<li><p>That value works like an import statement: <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code> translates to something like
-<code class="docutils literal notranslate"><span class="pre">from</span> <span class="pre">myproviderpackage.somemodule</span> <span class="pre">import</span> <span class="pre">get_provider_info</span></code>, and the <code class="docutils literal notranslate"><span class="pre">get_provider_info</span></code> that is being imported should be a
-callable, i.e. a function</p></li>
-<li><p>This function should return a dictionary with metadata</p></li>
-<li><p>If you have custom connection types as part of your package, that metadata will including a field called <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>
-which should be a list of strings of your custom hooks - those strings should also be in an import-like format, e.g.
-<code class="docutils literal notranslate"><span class="pre">myproviderpackage.hooks.source.SourceHook</span></code> means that there is a class <code class="docutils literal notranslate"><span class="pre">SourceHook</span></code> in <code class="docutils literal notranslate"><span class="pre">myproviderpackage/hooks/source.py</span></code>
-- airflow then imports these hooks and looks for the functions <code class="docutils literal notranslate"><span class="pre">get_ui_field_behaviour</span></code> and <code class="docutils literal notranslate"><span class="pre">get_connection_form_widgets</span></code>
-(both optional) as well as the attributes <code class="docutils literal notranslate"><span class="pre">conn_type</span></code> and <code class="docutils literal notranslate"><span class="pre">hook_name</span></code> to create the custom connection type in the airflow UI</p></li>
-</ul>
-<p><strong>Should I name my provider specifically or should it be created in ``airflow.providers`` package?</strong></p>
-<p>We have quite a number (>60) of providers managed by the community and we are going to maintain them
-together with Apache Airflow. All those providers have well-defined structured and follow the
-naming conventions we defined and they are all in <code class="docutils literal notranslate"><span class="pre">airflow.providers</span></code> package. If your intention is
-to contribute your provider, then you should follow those conventions and make a PR to Apache Airflow
-to contribute to it. But you are free to use any package name as long as there are no conflicts with other
-names, so preferably choose package that is in your “domain”.</p>
-<p><strong>Is there a convention for a connection id and type?</strong></p>
-<p>Very good question. Glad that you asked. We usually follow the convention <code class="docutils literal notranslate"><span class="pre"><NAME>_default</span></code> for connection
-id and just <code class="docutils literal notranslate"><span class="pre"><NAME></span></code> for connection type. Few examples:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">google_cloud_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">google_cloud_platform</span></code> type</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">aws_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">aws</span></code> type</p></li>
-</ul>
-<p>You should follow this convention. It is important, to use unique names for connection type,
-so it should be unique for your provider. If two providers try to add connection with the same type
-only one of them will succeed.</p>
-<p><strong>Can I contribute my own provider to Apache Airflow?</strong></p>
-<p>Of course, but it’s better to check at developer’s mailing list whether such contribution will be accepted by
-the Community, before investing time to make the provider compliant with community requirements.
-The Community only accepts providers that are generic enough, are well documented, fully covered by tests
-and with capabilities of being tested by people in the community. So we might not always be in the
-position to accept such contributions.</p>
-<p>After you think that your provider matches the expected values above, you can read
-<a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a> to check all prerequisites for a new
-community Provider and discuss it at the <a class="reference external" href="http://airflow.apache.org/community/">Devlist</a>.</p>
-<p>However, in case you have your own, specific provider, which you can maintain on your own or by your
-team, you are free to publish the providers in whatever form you find appropriate. The custom and
-community-managed providers have exactly the same capabilities.</p>
-<p><strong>Can I advertise my own provider to Apache Airflow users and share it with others as package in PyPI?</strong></p>
-<p>Absolutely! We have an <a class="reference external" href="https://airflow.apache.org/ecosystem/">Ecosystem</a> area on our website where
-we share non-community managed extensions and work for Airflow. Feel free to make a PR to the page and
-add we will evaluate and merge it when we see that such provider can be useful for the community of
-Airflow users.</p>
-<p><strong>Can I charge for the use of my provider?</strong></p>
-<p>This is something that is outside of our control and domain. As an Apache project, we are
-commercial-friendly and there are many businesses built around Apache Airflow and many other
-Apache projects. As a community, we provide all the software for free and this will never
-change. What 3rd-party developers are doing is not under control of Apache Airflow community.</p>
-</div>
-<div class="section" id="using-backport-providers-in-airflow-1-10">
-<h2>Using Backport Providers in Airflow 1.10<a class="headerlink" href="#using-backport-providers-in-airflow-1-10" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an Airflow version (1.10.12) running and it is stable. However, because of a Cloud provider change,
-I would like to upgrade the provider package. If I don’t need to upgrade the Airflow version anymore,
-how do I know that this provider version is compatible with my Airflow version?</strong></p>
-<p>We have Backport Providers are compatible with 1.10 but they stopped being released on
-March 17, 2021. Since then, no new changes to providers for Airflow 2.0 are going to be
-released as backport packages. It’s the highest time to upgrade to Airflow 2.0.</p>
-<p>When it comes to compatibility of providers with different Airflow 2 versions, each
-provider package will keep its own dependencies, and while we expect those providers to be generally
-backwards-compatible, particular versions of particular providers might introduce dependencies on
-specific Airflow versions.</p>
-<div class="toctree-wrapper compound">
-</div>
+ <div class="section" id="module-airflow.providers.amazon.aws.exceptions">
+<span id="airflow-providers-amazon-aws-exceptions"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.exceptions" title="airflow.providers.amazon.aws.exceptions"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.exceptions</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.exceptions" title="Permalink to this heading">¶</a></h1>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<dl class="py exception">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.exceptions.EcsTaskFailToStart">
+<em class="property"><span class="pre">exception</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.exceptions.</span></span><span class="sig-name descname"><span class="pre">EcsTaskFailToStart</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">message</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../_modules/airfl [...]
+<dd><p>Bases: <a class="reference external" href="https://docs.python.org/3/library/exceptions.html#Exception" title="(in Python v3.11)"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Exception</span></code></a></p>
+<p>Raise when ECS tasks fail to start AFTER processing the request.</p>
+<dl class="field-list simple">
+</dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.exceptions.EcsTaskFailToStart.__reduce__">
+<span class="sig-name descname"><span class="pre">__reduce__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../_modules/airflow/providers/amazon/aws/exceptions.html#EcsTaskFailToStart.__reduce__"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.exceptions.EcsTaskFailToStart.__reduce__" title="Permalink to this definition">¶</a></dt>
+<dd><p>Helper for pickle.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py exception">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.exceptions.EcsOperatorError">
+<em class="property"><span class="pre">exception</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.exceptions.</span></span><span class="sig-name descname"><span class="pre">EcsOperatorError</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">failures</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">message</span></span></em><span class="sig- [...]
+<dd><p>Bases: <a class="reference external" href="https://docs.python.org/3/library/exceptions.html#Exception" title="(in Python v3.11)"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Exception</span></code></a></p>
+<p>Raise when ECS cannot handle the request.</p>
+<dl class="field-list simple">
+</dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.exceptions.EcsOperatorError.__reduce__">
+<span class="sig-name descname"><span class="pre">__reduce__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../_modules/airflow/providers/amazon/aws/exceptions.html#EcsOperatorError.__reduce__"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.exceptions.EcsOperatorError.__reduce__" title="Permalink to this definition">¶</a></dt>
+<dd><p>Helper for pickle.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py exception">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.exceptions.S3HookUriParseFailure">
+<em class="property"><span class="pre">exception</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.exceptions.</span></span><span class="sig-name descname"><span class="pre">S3HookUriParseFailure</span></span><a class="reference internal" href="../../../../../../_modules/airflow/providers/amazon/aws/exceptions.html#S3HookUriParseFailure"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class [...]
+<dd><p>Bases: <a class="reference external" href="/docs/apache-airflow/stable/_api/airflow/exceptions/index.html#airflow.exceptions.AirflowException" title="(in apache-airflow v2.6.0.dev0)"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.exceptions.AirflowException</span></code></a></p>
+<p>When parse_s3_url fails to parse URL, this error is thrown.</p>
+</dd></dl>
+
</div>
</div>
- <div class="pager" role="navigation" aria-label="related navigation"><button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Previous</button>
- <a rel="next" title="Providers packages reference" href="packages-ref.html" >
+ <div class="pager" role="navigation" aria-label="related navigation">
+ <a rel="prev" title="airflow.providers.amazon.aws.waiters.base_waiter" href="../waiters/base_waiter/index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="p">Previous</button>
+ </a>
+ <a rel="next" title="tests.system.providers.amazon" href="../../../../../tests/system/providers/amazon/index.html" >
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="n">Next</button>
</a>
</div>
@@ -1044,28 +728,19 @@ specific Airflow versions.</p>
<nav class="wy-nav-side-toc">
<div class="wy-menu-vertical">
<ul>
-<li><a class="reference internal" href="#">Provider packages</a></li>
-<li><a class="reference internal" href="#extending-airflow-core-functionality">Extending Airflow core functionality</a><ul>
-<li><a class="reference internal" href="#auth-backends">Auth backends</a></li>
-<li><a class="reference internal" href="#custom-connections">Custom connections</a></li>
-<li><a class="reference internal" href="#extra-links">Extra links</a></li>
-<li><a class="reference internal" href="#logging">Logging</a></li>
-<li><a class="reference internal" href="#secret-backends">Secret backends</a></li>
-<li><a class="reference internal" href="#notifications">Notifications</a></li>
+<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.exceptions</span></code></a><ul>
+<li><a class="reference internal" href="#module-contents">Module Contents</a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.exceptions.EcsTaskFailToStart"><code class="docutils literal notranslate"><span class="pre">EcsTaskFailToStart</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.exceptions.EcsTaskFailToStart.__reduce__"><code class="docutils literal notranslate"><span class="pre">EcsTaskFailToStart.__reduce__()</span></code></a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.exceptions.EcsOperatorError"><code class="docutils literal notranslate"><span class="pre">EcsOperatorError</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.exceptions.EcsOperatorError.__reduce__"><code class="docutils literal notranslate"><span class="pre">EcsOperatorError.__reduce__()</span></code></a></li>
</ul>
</li>
-<li><a class="reference internal" href="#installing-and-upgrading-providers">Installing and upgrading providers</a></li>
-<li><a class="reference internal" href="#types-of-providers">Types of providers</a><ul>
-<li><a class="reference internal" href="#community-maintained-providers">Community maintained providers</a></li>
-<li><a class="reference internal" href="#custom-provider-packages">Custom provider packages</a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.exceptions.S3HookUriParseFailure"><code class="docutils literal notranslate"><span class="pre">S3HookUriParseFailure</span></code></a></li>
</ul>
</li>
-<li><a class="reference internal" href="#how-to-create-your-own-provider">How to create your own provider</a></li>
-<li><a class="reference internal" href="#faq-for-airflow-and-providers">FAQ for Airflow and Providers</a><ul>
-<li><a class="reference internal" href="#upgrading-airflow-2-0-and-providers">Upgrading Airflow 2.0 and Providers</a></li>
-<li><a class="reference internal" href="#customizing-provider-packages">Customizing Provider Packages</a></li>
-<li><a class="reference internal" href="#creating-your-own-providers">Creating your own providers</a></li>
-<li><a class="reference internal" href="#using-backport-providers-in-airflow-1-10">Using Backport Providers in Airflow 1.10</a></li>
</ul>
</li>
</ul>
@@ -1090,7 +765,7 @@ specific Airflow versions.</p>
<div class="base-layout--button">
- <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers/index.rst" rel="nofollow">
+ <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-amazon/_api/airflow/providers/amazon/aws/exceptions/index.rst" rel="nofollow">
<button class="btn-hollow btn-brown btn-with-icon with-box-shadow button-fixed">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
@@ -1224,19 +899,19 @@ specific Airflow versions.</p>
</span>
</div>
</footer>
- <script type="text/javascript" src="_static/_gen/js/docs.js"></script>
- <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
+ <script type="text/javascript" src="../../../../../../_static/_gen/js/docs.js"></script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../../../" src="../../../../../../_static/documentation_options.js"></script>
<script src="https://code.jquery.com/jquery-3.4.1.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js" integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy" crossorigin="anonymous"></script>
- <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
- <script src="_static/jquery.js"></script>
- <script src="_static/underscore.js"></script>
- <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
- <script src="_static/doctools.js"></script>
- <script src="_static/sphinx_highlight.js"></script>
- <script src="_static/clipboard.min.js"></script>
- <script src="_static/copybutton.js"></script>
- <script src="_static/js/globaltoc.js"></script>
+ <script data-url_root="../../../../../../" id="documentation_options" src="../../../../../../_static/documentation_options.js"></script>
+ <script src="../../../../../../_static/jquery.js"></script>
+ <script src="../../../../../../_static/underscore.js"></script>
+ <script src="../../../../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
+ <script src="../../../../../../_static/doctools.js"></script>
+ <script src="../../../../../../_static/sphinx_highlight.js"></script>
+ <script src="../../../../../../_static/clipboard.min.js"></script>
+ <script src="../../../../../../_static/copybutton.js"></script>
+ <script src="../../../../../../_static/js/globaltoc.js"></script>
</body>
</html>
\ No newline at end of file
diff --git a/docs-archive/apache-airflow-providers/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/appflow/index.html
similarity index 60%
copy from docs-archive/apache-airflow-providers/index.html
copy to docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/appflow/index.html
index 70eef65adf..277fa44915 100644
--- a/docs-archive/apache-airflow-providers/index.html
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/appflow/index.html
@@ -9,18 +9,20 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Provider packages — apache-airflow-providers Documentation</title>
- <link rel="stylesheet" href="_static/_gen/css/main.min.css" type="text/css" />
- <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main.min.css" />
- <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main-custom.min.css" />
- <link rel="shortcut icon" href="_static/pin_32.png"/>
- <link rel="index" title="Index" href="genindex.html" />
- <link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Providers packages reference" href="packages-ref.html" />
-
- <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers/stable/index.html" />
+ <title>airflow.providers.amazon.aws.hooks.appflow — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../../search.html" />
+ <link rel="next" title="airflow.providers.amazon.aws.hooks.athena" href="../athena/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.hooks" href="../index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/hooks/appflow/index.html" />
<!-- Matomo -->
<script>
@@ -282,8 +284,21 @@
<div class="content-drawer">
<div class="td-sidebar">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -313,14 +328,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -401,8 +434,21 @@
<div class="td-sidebar desktop-only d-print-none">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -432,14 +478,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -528,9 +592,15 @@
<ul class="breadcrumb">
- <li class="breadcrumb-item"><a href="#" class="icon icon-home"> Home</a></li>
+ <li class="breadcrumb-item"><a href="../../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks</span></code></a></li>
- <li class="breadcrumb-item"><a href="index.html"> Provider packages</a></li>
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.appflow</span></code></a></li>
</ul>
</div>
@@ -541,440 +611,101 @@
<div class="bodywrapper">
<div class="body" role="main">
- <blockquote>
-<div></div></blockquote>
-<div class="section" id="provider-packages">
-<h1>Provider packages<a class="headerlink" href="#provider-packages" title="Permalink to this heading">¶</a></h1>
-<p>Apache Airflow 2 is built in modular way. The “Core” of Apache Airflow provides core scheduler
-functionality which allow you to write some basic tasks, but the capabilities of Apache Airflow can
-be extended by installing additional packages, called <code class="docutils literal notranslate"><span class="pre">providers</span></code>.</p>
-<p>Providers can contain operators, hooks, sensor, and transfer operators to communicate with a
-multitude of external systems, but they can also extend Airflow core with new capabilities.</p>
-<p>You can install those provider packages separately in order to interface with a given service. The providers
-for <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> are designed in the way that you can write your own providers easily. The
-<code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span> <span class="pre">Community</span></code> develops and maintain more than 60 provider packages, but you are free to
-develop your own providers - the providers you build have exactly the same capability as the providers
-written by the community, so you can release and share those providers with others.</p>
-<p>The full list of community managed providers is available at
-<a class="reference external" href="https://airflow.apache.org/docs/#providers-packages-docs-apache-airflow-providers-index-html">Providers Index</a>.</p>
-<p>You can also see index of all community provider’s operators and hooks in
-<a class="reference internal" href="operators-and-hooks-ref/index.html"><span class="doc">Operators and Hooks Reference</span></a></p>
-</div>
-<div class="section" id="extending-airflow-core-functionality">
-<h1>Extending Airflow core functionality<a class="headerlink" href="#extending-airflow-core-functionality" title="Permalink to this heading">¶</a></h1>
-<p>Providers give you the capability of extending core Airflow with extra capabilities. The Core airflow
-provides basic and solid functionality of scheduling, the providers extend its capabilities. Here we
-describe all the custom capabilities.</p>
-<p>Airflow automatically discovers which providers add those additional capabilities and, once you install
-provider package and re-start Airflow, those become automatically available to Airflow Users.</p>
-<p>The summary of the core functionalities that can be extended are available in
-<a class="reference internal" href="core-extensions/index.html"><span class="doc">Core Extensions</span></a>.</p>
-<div class="section" id="auth-backends">
-<h2>Auth backends<a class="headerlink" href="#auth-backends" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom authentication backends, that allow you to configure the way how your
-web server authenticates your users, integrating it with public or private authentication services.</p>
-<p>You can see all the authentication backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/auth-backends.html"><span class="doc">Auth backends</span></a></p>
-</div>
-<div class="section" id="custom-connections">
-<h2>Custom connections<a class="headerlink" href="#custom-connections" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom connection types, extending connection form and handling custom form field
-behaviour for the connections defined by the provider.</p>
-<p>You can see all custom connections available via community-managed providers in
-<a class="reference internal" href="core-extensions/connections.html"><span class="doc">Connections</span></a>.</p>
-</div>
-<div class="section" id="extra-links">
-<h2>Extra links<a class="headerlink" href="#extra-links" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add extra custom links to operators delivered by the provider. Those will be visible in
-task details view of the task.</p>
-<p>You can see all the extra links available via community-managed providers in
-<a class="reference internal" href="core-extensions/extra-links.html"><span class="doc">Extra Links</span></a>.</p>
-</div>
-<div class="section" id="logging">
-<h2>Logging<a class="headerlink" href="#logging" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add additional task logging capabilities. By default <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> saves logs for
-tasks locally and make them available to Airflow UI via internal http server. However, providers
-can add extra logging capabilities, where Airflow Logs can be written to a remote service and
-retrieved from those services.</p>
-<p>You can see all task loggers available via community-managed providers in
-<a class="reference internal" href="core-extensions/logging.html"><span class="doc">Writing logs</span></a>.</p>
-</div>
-<div class="section" id="secret-backends">
-<h2>Secret backends<a class="headerlink" href="#secret-backends" title="Permalink to this heading">¶</a></h2>
-<p>Airflow has the capability of reading connections, variables and configuration from Secret Backends rather
-than from its own Database.</p>
-<p>You can see all secret backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/secrets-backends.html"><span class="doc">Secret backends</span></a>.</p>
-</div>
-<div class="section" id="notifications">
-<h2>Notifications<a class="headerlink" href="#notifications" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom notifications, that allow you to configure the way how you would like to receive
-notifications about the status of your tasks/DAGs.</p>
-<p>You can see all the notifications available via community-managed providers in
-<a class="reference internal" href="core-extensions/notifications.html"><span class="doc">Notifications</span></a>.</p>
-</div>
-</div>
-<div class="section" id="installing-and-upgrading-providers">
-<h1>Installing and upgrading providers<a class="headerlink" href="#installing-and-upgrading-providers" title="Permalink to this heading">¶</a></h1>
-<p>Separate provider packages give the possibilities that were not available in 1.10:</p>
-<ol class="arabic simple">
-<li><p>You can upgrade to latest version of particular providers without the need of Apache Airflow core upgrade.</p></li>
-<li><p>You can downgrade to previous version of particular provider in case the new version introduces
-some problems, without impacting the main Apache Airflow core package.</p></li>
-<li><p>You can release and upgrade/downgrade provider packages incrementally, independent from each other. This
-means that you can incrementally validate each of the provider package update in your environment,
-following the usual tests you have in your environment.</p></li>
-</ol>
-</div>
-<div class="section" id="types-of-providers">
-<h1>Types of providers<a class="headerlink" href="#types-of-providers" title="Permalink to this heading">¶</a></h1>
-<p>Providers have the same capacity - no matter if they are provided by the community or if they are
-third-party providers. This chapter explains how community managed providers are versioned and released
-and how you can create your own providers.</p>
-<div class="section" id="community-maintained-providers">
-<h2>Community maintained providers<a class="headerlink" href="#community-maintained-providers" title="Permalink to this heading">¶</a></h2>
-<p>From the point of view of the community, Airflow is delivered in multiple, separate packages.
-The core of Airflow scheduling system is delivered as <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package and there are more than
-60 provider packages which can be installed separately as so called <code class="docutils literal notranslate"><span class="pre">Airflow</span> <span class="pre">Provider</span> <span class="pre">packages</span></code>.
-Those packages are available as <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers</span></code> packages - for example there is an
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> or <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> package).</p>
-<p>Community maintained providers are released and versioned separately from the Airflow releases. We are
-following the <a class="reference external" href="https://semver.org/">Semver</a> versioning scheme for the packages. Some versions of the
-provider packages might depend on particular versions of Airflow, but the general approach we have is that
-unless there is a good reason, new version of providers should work with recent versions of Airflow 2.x.
-Details will vary per-provider and if there is a limitation for particular version of particular provider,
-constraining the Airflow version used, it will be included as limitation of dependencies in the provider
-package.</p>
-<p>Each community provider has corresponding extra which can be used when installing airflow to install the
-provider together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> - for example you can install airflow with those extras:
-<code class="docutils literal notranslate"><span class="pre">apache-airflow[google,amazon]</span></code> (with correct constraints -see <a class="reference external" href="/docs/apache-airflow/stable/installation/index.html" title="(in apache-airflow v2.6.0.dev0)"><span>Installation</span></a>) and you
-will install the appropriate versions of the <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> and
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> packages together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>.</p>
-<p>Some of the community providers have cross-provider dependencies as well. Those are not required
-dependencies, they might simply enable certain features (for example transfer operators often create
-dependency between different providers. Again, the general approach here is that the providers are backwards
-compatible, including cross-dependencies. Any kind of breaking changes and requirements on particular versions of other
-provider packages are automatically documented in the release notes of every provider.</p>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>For Airflow 1.10 We also provided <code class="docutils literal notranslate"><span class="pre">apache-airflow-backport-providers</span></code> packages that could be installed
-with those versions Those were the same providers as for 2.0 but automatically back-ported to work for
-Airflow 1.10. The last release of backport providers was done on March 17, 2021 and the backport
-providers will no longer be released, since Airflow 1.10 has reached End-Of-Life as of June 17, 2021.</p>
-</div>
-<p>If you want to contribute to <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>, you can see how to build and extend community
-managed providers in <a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a>.</p>
-</div>
-<div class="section" id="custom-provider-packages">
-<span id="providers-community-maintained-providers"></span><h2>Custom provider packages<a class="headerlink" href="#custom-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p>You can develop and release your own providers. Your custom operators, hooks, sensors, transfer operators
-can be packaged together in a standard airflow package and installed using the same mechanisms.
-Moreover they can also use the same mechanisms to extend the Airflow Core with auth backends,
-custom connections, logging, secret backends and extra operator links as described in the previous chapter.</p>
-</div>
-</div>
-<div class="section" id="how-to-create-your-own-provider">
-<h1>How to create your own provider<a class="headerlink" href="#how-to-create-your-own-provider" title="Permalink to this heading">¶</a></h1>
-<p>As mentioned in the <a class="reference external" href="http://airflow.apache.org/docs/apache-airflow-providers/index.html">Providers</a>
-documentation, custom providers can extend Airflow core - they can add extra links to operators as well
-as custom connections. You can use build your own providers and install them as packages if you would like
-to use the mechanism for your own, custom providers.</p>
-<p>Adding a provider to Airflow is just a matter of building a Python package and adding the right meta-data to
-the package. We are using standard mechanism of python to define
-<a class="reference external" href="https://docs.python.org/3/library/importlib.metadata.html#entry-points">entry points</a> . Your package
-needs to define appropriate entry-point <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> which has to point to a callable
-implemented by your package and return a dictionary containing the list of discoverable capabilities
-of your package. The dictionary has to follow the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json">json-schema specification</a>.</p>
-<p>Most of the schema provides extension point for the documentation (which you might want to also use for
-your own purpose) but the important fields from the extensibility point of view are those:</p>
-<p>Displaying package information in CLI/API:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">package-name</span></code> - Name of the package for the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code> - Human-friendly name of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">description</span></code> - Additional description of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">version</span></code> - List of versions of the package (in reverse-chronological order). The first version in the
-list is the current package version. It is taken from the version of package installed, not from the
-provider_info information.</p></li>
-</ul>
-<p>Exposing customized functionality to the Airflow’s core:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">extra-links</span></code> - this field should contain the list of all operator class names that are adding extra links
-capability. See <a class="reference external" href="/docs/apache-airflow/stable/howto/define-extra-link.html" title="(in apache-airflow v2.6.0.dev0)"><span>Define an operator extra link</span></a> for description of how to add extra link
-capability to the operators of yours.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">connection-types</span></code> - this field should contain the list of all connection types together with hook
-class names implementing those custom connection types (providing custom extra fields and
-custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
-<code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> (deprecated) - this field should contain the list of all hook class names that provide
-custom connection types with custom extra fields and field behaviour. The <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array
-is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
-providers are targeting Airflow 2.2.0+ you do not have to include the <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array, if
-you want to also target earlier versions of Airflow 2, you should include both <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> and
-<code class="docutils literal notranslate"><span class="pre">connection-types</span></code> arrays. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details.</p></li>
-</ul>
-<p>When your providers are installed you can query the installed providers and their capabilities with the
-<code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">providers</span></code> command. This way you can verify if your providers are properly recognized and whether
-they define the extensions properly. See <a class="reference external" href="/docs/apache-airflow/stable/cli-and-env-variables-ref.html" title="(in apache-airflow v2.6.0.dev0)"><span>Command Line Interface and Environment Variables Reference</span></a> for details of available CLI
-sub-commands.</p>
-<p>When you write your own provider, consider following the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#naming-conventions-for-provider-packages">Naming conventions for provider packages</a></p>
-</div>
-<div class="section" id="faq-for-airflow-and-providers">
-<h1>FAQ for Airflow and Providers<a class="headerlink" href="#faq-for-airflow-and-providers" title="Permalink to this heading">¶</a></h1>
-<div class="section" id="upgrading-airflow-2-0-and-providers">
-<h2>Upgrading Airflow 2.0 and Providers<a class="headerlink" href="#upgrading-airflow-2-0-and-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When upgrading to a new Airflow version such as 2.0, but possibly 2.0.1 and beyond, is the best practice
-to also upgrade provider packages at the same time?</strong></p>
-<p>It depends on your use case. If you have automated or semi-automated verification of your installation,
-that you can run a new version of Airflow including all provider packages, then definitely go for it.
-If you rely more on manual testing, it is advised that you upgrade in stages. Depending on your choice
-you can either upgrade all used provider packages first, and then upgrade Airflow Core or the other way
-round. The first approach - when you first upgrade all providers is probably safer, as you can do it
-incrementally, step-by-step replacing provider by provider in your environment.</p>
-</div>
-<div class="section" id="customizing-provider-packages">
-<h2>Customizing Provider Packages<a class="headerlink" href="#customizing-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an older version of my provider package which we have lightly customized and is working
-fine with my MSSQL installation. I am upgrading my Airflow version. Do I need to upgrade my provider,
-or can I keep it as it is?</strong></p>
-<p>It depends on the scope of customization. There is no need to upgrade the provider packages to later
-versions unless you want to upgrade to Airflow version that introduces backwards-incompatible changes.
-Generally speaking, with Airflow 2 we are following the <a class="reference external" href="https://semver.org/">Semver</a> approach where
-we will introduce backwards-incompatible changes in Major releases, so all your modifications (as long
-as you have not used internal Airflow classes) should work for All Airflow 2.* versions.</p>
-</div>
-<div class="section" id="creating-your-own-providers">
-<h2>Creating your own providers<a class="headerlink" href="#creating-your-own-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When I write my own provider, do I need to do anything special to make it available to others?</strong></p>
-<p>You do not need to do anything special besides creating the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point
-returning properly formatted meta-data - dictionary with <code class="docutils literal notranslate"><span class="pre">extra-links</span></code> and <code class="docutils literal notranslate"><span class="pre">connection-types</span></code> fields
-(and deprecated <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> field if you are also targeting versions of Airflow before 2.2.0).</p>
-<p>Anyone who runs airflow in an environment that has your Python package installed will be able to use the
-package as a provider package.</p>
-<p><strong>What do I need to do to turn a package into a provider?</strong></p>
-<p>You need to do the following to turn an existing Python package into a provider (see below for examples):</p>
+ <div class="section" id="module-airflow.providers.amazon.aws.hooks.appflow">
+<span id="airflow-providers-amazon-aws-hooks-appflow"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.hooks.appflow" title="airflow.providers.amazon.aws.hooks.appflow"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.appflow</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.hooks.appflow" title="Permalink to this heading">¶</a></h1>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<div class="section" id="classes">
+<h3>Classes<a class="headerlink" href="#classes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.appflow.AppflowHook" title="airflow.providers.amazon.aws.hooks.appflow.AppflowHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">AppflowHook</span></code></a></p></td>
+<td><p>Interact with Amazon Appflow.</p></td>
+</tr>
+</tbody>
+</table>
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.appflow.AppflowHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.appflow.</span></span><span class="sig-name descname"><span class="pre">AppflowHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</ [...]
+<dd><p>Bases: <a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p>
+<p>Interact with Amazon Appflow.
+Provide thin wrapper around <a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/appflow.html#Appflow.Client" title="(in Boto3 v1.26.120)"><code class="xref py py-class docutils literal notranslate"><span class="pre">boto3.client("appflow")</span></code></a>.</p>
+<p>Additional arguments (such as <code class="docutils literal notranslate"><span class="pre">aws_conn_id</span></code>) may be specified and
+are passed down to the underlying AwsBaseHook.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Add the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point in the <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> - this tells airflow where to get
-the required provider metadata</p></li>
-<li><p>Create the function that you refer to in the first step as part of your package: this functions returns a
-dictionary that contains all meta-data about your provider package</p></li>
-<li><p>If you want Airflow to link to documentation of your Provider in the providers page, make sure
-to add “project-url/documentation” <a class="reference external" href="https://peps.python.org/pep-0621/#example">metadata</a> to your package.
-This will also add link to your documentation in PyPI.</p></li>
-<li><p>note that the dictionary should be compliant with <code class="docutils literal notranslate"><span class="pre">airflow/provider_info.schema.json</span></code> JSON-schema
-specification. The community-managed providers have more fields there that are used to build
-documentation, but the requirement for runtime information only contains several fields which are defined
-in the schema:</p></li>
+<li><p><a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-class docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p></li>
+<li><p><a class="reference external" href="https://docs.aws.amazon.com/appflow/1.0/APIReference/Welcome.html">Amazon Appflow API Reference</a></p></li>
</ul>
-<div class="example-block-wrapper compound">
-<p class="example-header compound-first"><span class="example-title">airflow/provider_info.schema.json</span></p>
-<div class="compound-last highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"$schema"</span><span class="p">:</span><span class="w"> </span><span class="s2">"http://json-schema.org/draft-07/schema#"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"package-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Package name available under which the package is available in the PyPI repository."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Provider name"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Information about the package in RST format"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-names"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class names that provide connection types to core (deprecated by connection-types)"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"deprecated"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"The hook-class-names property has been deprecated in favour of connection-types which is more performant version allowing to only import individual Hooks rather than all hooks at once"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"deprecatedVersion"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.2.0"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"connection-types"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Map of connection types mapped to hook class names."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"connection-type"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Type of connection defined by the provider"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class name that implements the connection type"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"connection-type"</span><span class="p">,</span><span class="w"> </span><span class="s2">"hook-class-name"</span><span class="p">]</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"extra-links"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Operator class names that provide extra link functionality"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"secrets-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Secrets Backend class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"logging"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Logging Task Handlers class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"auth-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"API Auth Backend module names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"notifications"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Notification class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"task-decorators"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Apply custom decorators to the TaskFlow API. Can be accessed by users via '@task.<name>'"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"path"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
-<span class="w"> </span><span class="s2">"name"</span><span class="p">,</span>
-<span class="w"> </span><span class="s2">"description"</span>
-<span class="w"> </span><span class="p">]</span>
-<span class="p">}</span>
-</pre></div>
-</div>
-</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>:</p>
-<div class="highlight-cfg notranslate"><div class="highlight"><pre><span></span><span class="k">[options.entry_points]</span>
-<span class="c1"># the function get_provider_info is defined in myproviderpackage.somemodule</span>
-<span class="na">apache_airflow_provider</span><span class="o">=</span>
-<span class="w"> </span><span class="na">provider_info</span><span class="o">=</span><span class="s">myproviderpackage.somemodule:get_provider_info</span>
-</pre></div>
-</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">myproviderpackage/somemodule.py</span></code>:</p>
-<div class="highlight-Python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_provider_info</span><span class="p">():</span>
- <span class="k">return</span> <span class="p">{</span>
- <span class="s2">"package-name"</span><span class="p">:</span> <span class="s2">"my-package-name"</span><span class="p">,</span>
- <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"name"</span><span class="p">,</span>
- <span class="s2">"description"</span><span class="p">:</span> <span class="s2">"a description"</span><span class="p">,</span>
- <span class="s2">"hook-class-names"</span><span class="p">:</span> <span class="p">[</span>
- <span class="s2">"myproviderpackage.hooks.source.SourceHook"</span><span class="p">,</span>
- <span class="p">],</span>
- <span class="p">}</span>
-</pre></div>
</div>
-<p><strong>How do provider packages work under the hood?</strong></p>
-<p>When running Airflow with your provider package, there will be (at least) three components to your airflow installation:</p>
-<ul class="simple">
-<li><p>The installation itself (for example, a <code class="docutils literal notranslate"><span class="pre">venv</span></code> where you installed airflow with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">apache-airflow</span></code>)
-together with the related files (e.g. <code class="docutils literal notranslate"><span class="pre">dags</span></code> folder)</p></li>
-<li><p>The <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package</p></li>
-<li><p>Your own <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package that is independent of <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> or your airflow installation, which
-can be a local Python package (that you install via <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span> <span class="pre">/path/to/my-package</span></code>), a normal pip package
-(<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">myproviderpackage</span></code>), or any other type of Python package</p></li>
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.appflow.AppflowHook.conn">
+<span class="sig-name descname"><span class="pre">conn</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/appflow.html#AppflowHook.conn"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.appflow.AppflowHook.conn" title="Permalink to this definition">¶</a></dt>
+<dd><p>Get the underlying boto3 Appflow client (cached)</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.appflow.AppflowHook.run_flow">
+<span class="sig-name descname"><span class="pre">run_flow</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">flow_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">poll_interval</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">20</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">wait_for_completion</span></span><span class= [...]
+<dd><p>Execute an AppFlow run.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>flow_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – The flow name</p></li>
+<li><p><strong>poll_interval</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a>) – Time (seconds) to wait between two consecutive calls to check the run status</p></li>
+<li><p><strong>wait_for_completion</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a>) – whether to wait for the run to end to return</p></li>
</ul>
-<p>In the <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package you need to add the entry point and provide the appropriate metadata as described above.
-If you have done that, airflow does the following at runtime:</p>
-<ul class="simple">
-<li><p>Loop through ALL packages installed in your environment / <code class="docutils literal notranslate"><span class="pre">venv</span></code></p></li>
-<li><p>For each package, if the package’s <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> has a section <code class="docutils literal notranslate"><span class="pre">[options.entry_points]</span></code>, and if that section has a value
-for <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code>, then get the value for <code class="docutils literal notranslate"><span class="pre">provider_info</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code></p></li>
-<li><p>That value works like an import statement: <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code> translates to something like
-<code class="docutils literal notranslate"><span class="pre">from</span> <span class="pre">myproviderpackage.somemodule</span> <span class="pre">import</span> <span class="pre">get_provider_info</span></code>, and the <code class="docutils literal notranslate"><span class="pre">get_provider_info</span></code> that is being imported should be a
-callable, i.e. a function</p></li>
-<li><p>This function should return a dictionary with metadata</p></li>
-<li><p>If you have custom connection types as part of your package, that metadata will including a field called <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>
-which should be a list of strings of your custom hooks - those strings should also be in an import-like format, e.g.
-<code class="docutils literal notranslate"><span class="pre">myproviderpackage.hooks.source.SourceHook</span></code> means that there is a class <code class="docutils literal notranslate"><span class="pre">SourceHook</span></code> in <code class="docutils literal notranslate"><span class="pre">myproviderpackage/hooks/source.py</span></code>
-- airflow then imports these hooks and looks for the functions <code class="docutils literal notranslate"><span class="pre">get_ui_field_behaviour</span></code> and <code class="docutils literal notranslate"><span class="pre">get_connection_form_widgets</span></code>
-(both optional) as well as the attributes <code class="docutils literal notranslate"><span class="pre">conn_type</span></code> and <code class="docutils literal notranslate"><span class="pre">hook_name</span></code> to create the custom connection type in the airflow UI</p></li>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>The run execution ID</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.appflow.AppflowHook.update_flow_filter">
+<span class="sig-name descname"><span class="pre">update_flow_filter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">flow_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">filter_tasks</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">set_trigger_ondemand</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span [...]
+<dd><p>Update the flow task filter.
+All filters will be removed if an empty array is passed to filter_tasks.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>flow_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – The flow name</p></li>
+<li><p><strong>filter_tasks</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#list" title="(in Python v3.11)"><em>list</em></a><em>[</em><em>mypy_boto3_appflow.type_defs.TaskTypeDef</em><em>]</em>) – List flow tasks to be added</p></li>
+<li><p><strong>set_trigger_ondemand</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a>) – If True, set the trigger to on-demand; otherwise, keep the trigger as is</p></li>
</ul>
-<p><strong>Should I name my provider specifically or should it be created in ``airflow.providers`` package?</strong></p>
-<p>We have quite a number (>60) of providers managed by the community and we are going to maintain them
-together with Apache Airflow. All those providers have well-defined structured and follow the
-naming conventions we defined and they are all in <code class="docutils literal notranslate"><span class="pre">airflow.providers</span></code> package. If your intention is
-to contribute your provider, then you should follow those conventions and make a PR to Apache Airflow
-to contribute to it. But you are free to use any package name as long as there are no conflicts with other
-names, so preferably choose package that is in your “domain”.</p>
-<p><strong>Is there a convention for a connection id and type?</strong></p>
-<p>Very good question. Glad that you asked. We usually follow the convention <code class="docutils literal notranslate"><span class="pre"><NAME>_default</span></code> for connection
-id and just <code class="docutils literal notranslate"><span class="pre"><NAME></span></code> for connection type. Few examples:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">google_cloud_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">google_cloud_platform</span></code> type</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">aws_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">aws</span></code> type</p></li>
-</ul>
-<p>You should follow this convention. It is important, to use unique names for connection type,
-so it should be unique for your provider. If two providers try to add connection with the same type
-only one of them will succeed.</p>
-<p><strong>Can I contribute my own provider to Apache Airflow?</strong></p>
-<p>Of course, but it’s better to check at developer’s mailing list whether such contribution will be accepted by
-the Community, before investing time to make the provider compliant with community requirements.
-The Community only accepts providers that are generic enough, are well documented, fully covered by tests
-and with capabilities of being tested by people in the community. So we might not always be in the
-position to accept such contributions.</p>
-<p>After you think that your provider matches the expected values above, you can read
-<a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a> to check all prerequisites for a new
-community Provider and discuss it at the <a class="reference external" href="http://airflow.apache.org/community/">Devlist</a>.</p>
-<p>However, in case you have your own, specific provider, which you can maintain on your own or by your
-team, you are free to publish the providers in whatever form you find appropriate. The custom and
-community-managed providers have exactly the same capabilities.</p>
-<p><strong>Can I advertise my own provider to Apache Airflow users and share it with others as package in PyPI?</strong></p>
-<p>Absolutely! We have an <a class="reference external" href="https://airflow.apache.org/ecosystem/">Ecosystem</a> area on our website where
-we share non-community managed extensions and work for Airflow. Feel free to make a PR to the page and
-add we will evaluate and merge it when we see that such provider can be useful for the community of
-Airflow users.</p>
-<p><strong>Can I charge for the use of my provider?</strong></p>
-<p>This is something that is outside of our control and domain. As an Apache project, we are
-commercial-friendly and there are many businesses built around Apache Airflow and many other
-Apache projects. As a community, we provide all the software for free and this will never
-change. What 3rd-party developers are doing is not under control of Apache Airflow community.</p>
-</div>
-<div class="section" id="using-backport-providers-in-airflow-1-10">
-<h2>Using Backport Providers in Airflow 1.10<a class="headerlink" href="#using-backport-providers-in-airflow-1-10" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an Airflow version (1.10.12) running and it is stable. However, because of a Cloud provider change,
-I would like to upgrade the provider package. If I don’t need to upgrade the Airflow version anymore,
-how do I know that this provider version is compatible with my Airflow version?</strong></p>
-<p>We have Backport Providers are compatible with 1.10 but they stopped being released on
-March 17, 2021. Since then, no new changes to providers for Airflow 2.0 are going to be
-released as backport packages. It’s the highest time to upgrade to Airflow 2.0.</p>
-<p>When it comes to compatibility of providers with different Airflow 2 versions, each
-provider package will keep its own dependencies, and while we expect those providers to be generally
-backwards-compatible, particular versions of particular providers might introduce dependencies on
-specific Airflow versions.</p>
-<div class="toctree-wrapper compound">
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>None</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p>None</p>
+</dd>
+</dl>
+</dd></dl>
+
+</dd></dl>
+
</div>
</div>
</div>
- <div class="pager" role="navigation" aria-label="related navigation"><button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Previous</button>
- <a rel="next" title="Providers packages reference" href="packages-ref.html" >
+ <div class="pager" role="navigation" aria-label="related navigation">
+ <a rel="prev" title="airflow.providers.amazon.aws.hooks" href="../index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="p">Previous</button>
+ </a>
+ <a rel="next" title="airflow.providers.amazon.aws.hooks.athena" href="../athena/index.html" >
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="n">Next</button>
</a>
</div>
@@ -1044,28 +775,19 @@ specific Airflow versions.</p>
<nav class="wy-nav-side-toc">
<div class="wy-menu-vertical">
<ul>
-<li><a class="reference internal" href="#">Provider packages</a></li>
-<li><a class="reference internal" href="#extending-airflow-core-functionality">Extending Airflow core functionality</a><ul>
-<li><a class="reference internal" href="#auth-backends">Auth backends</a></li>
-<li><a class="reference internal" href="#custom-connections">Custom connections</a></li>
-<li><a class="reference internal" href="#extra-links">Extra links</a></li>
-<li><a class="reference internal" href="#logging">Logging</a></li>
-<li><a class="reference internal" href="#secret-backends">Secret backends</a></li>
-<li><a class="reference internal" href="#notifications">Notifications</a></li>
+<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.appflow</span></code></a><ul>
+<li><a class="reference internal" href="#module-contents">Module Contents</a><ul>
+<li><a class="reference internal" href="#classes">Classes</a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.appflow.AppflowHook"><code class="docutils literal notranslate"><span class="pre">AppflowHook</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.appflow.AppflowHook.conn"><code class="docutils literal notranslate"><span class="pre">AppflowHook.conn()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.appflow.AppflowHook.run_flow"><code class="docutils literal notranslate"><span class="pre">AppflowHook.run_flow()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.appflow.AppflowHook.update_flow_filter"><code class="docutils literal notranslate"><span class="pre">AppflowHook.update_flow_filter()</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
-<li><a class="reference internal" href="#installing-and-upgrading-providers">Installing and upgrading providers</a></li>
-<li><a class="reference internal" href="#types-of-providers">Types of providers</a><ul>
-<li><a class="reference internal" href="#community-maintained-providers">Community maintained providers</a></li>
-<li><a class="reference internal" href="#custom-provider-packages">Custom provider packages</a></li>
</ul>
</li>
-<li><a class="reference internal" href="#how-to-create-your-own-provider">How to create your own provider</a></li>
-<li><a class="reference internal" href="#faq-for-airflow-and-providers">FAQ for Airflow and Providers</a><ul>
-<li><a class="reference internal" href="#upgrading-airflow-2-0-and-providers">Upgrading Airflow 2.0 and Providers</a></li>
-<li><a class="reference internal" href="#customizing-provider-packages">Customizing Provider Packages</a></li>
-<li><a class="reference internal" href="#creating-your-own-providers">Creating your own providers</a></li>
-<li><a class="reference internal" href="#using-backport-providers-in-airflow-1-10">Using Backport Providers in Airflow 1.10</a></li>
</ul>
</li>
</ul>
@@ -1090,7 +812,7 @@ specific Airflow versions.</p>
<div class="base-layout--button">
- <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers/index.rst" rel="nofollow">
+ <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-amazon/_api/airflow/providers/amazon/aws/hooks/appflow/index.rst" rel="nofollow">
<button class="btn-hollow btn-brown btn-with-icon with-box-shadow button-fixed">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
@@ -1224,19 +946,19 @@ specific Airflow versions.</p>
</span>
</div>
</footer>
- <script type="text/javascript" src="_static/_gen/js/docs.js"></script>
- <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
+ <script type="text/javascript" src="../../../../../../../_static/_gen/js/docs.js"></script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../../../../" src="../../../../../../../_static/documentation_options.js"></script>
<script src="https://code.jquery.com/jquery-3.4.1.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js" integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy" crossorigin="anonymous"></script>
- <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
- <script src="_static/jquery.js"></script>
- <script src="_static/underscore.js"></script>
- <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
- <script src="_static/doctools.js"></script>
- <script src="_static/sphinx_highlight.js"></script>
- <script src="_static/clipboard.min.js"></script>
- <script src="_static/copybutton.js"></script>
- <script src="_static/js/globaltoc.js"></script>
+ <script data-url_root="../../../../../../../" id="documentation_options" src="../../../../../../../_static/documentation_options.js"></script>
+ <script src="../../../../../../../_static/jquery.js"></script>
+ <script src="../../../../../../../_static/underscore.js"></script>
+ <script src="../../../../../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
+ <script src="../../../../../../../_static/doctools.js"></script>
+ <script src="../../../../../../../_static/sphinx_highlight.js"></script>
+ <script src="../../../../../../../_static/clipboard.min.js"></script>
+ <script src="../../../../../../../_static/copybutton.js"></script>
+ <script src="../../../../../../../_static/js/globaltoc.js"></script>
</body>
</html>
\ No newline at end of file
diff --git a/docs-archive/apache-airflow-providers/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/athena/index.html
similarity index 60%
copy from docs-archive/apache-airflow-providers/index.html
copy to docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/athena/index.html
index 70eef65adf..3f5f1f2c53 100644
--- a/docs-archive/apache-airflow-providers/index.html
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/athena/index.html
@@ -9,18 +9,20 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Provider packages — apache-airflow-providers Documentation</title>
- <link rel="stylesheet" href="_static/_gen/css/main.min.css" type="text/css" />
- <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main.min.css" />
- <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main-custom.min.css" />
- <link rel="shortcut icon" href="_static/pin_32.png"/>
- <link rel="index" title="Index" href="genindex.html" />
- <link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Providers packages reference" href="packages-ref.html" />
-
- <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers/stable/index.html" />
+ <title>airflow.providers.amazon.aws.hooks.athena — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../../search.html" />
+ <link rel="next" title="airflow.providers.amazon.aws.hooks.base_aws" href="../base_aws/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.hooks.appflow" href="../appflow/index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/hooks/athena/index.html" />
<!-- Matomo -->
<script>
@@ -282,8 +284,21 @@
<div class="content-drawer">
<div class="td-sidebar">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -313,14 +328,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -401,8 +434,21 @@
<div class="td-sidebar desktop-only d-print-none">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -432,14 +478,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -528,9 +592,15 @@
<ul class="breadcrumb">
- <li class="breadcrumb-item"><a href="#" class="icon icon-home"> Home</a></li>
+ <li class="breadcrumb-item"><a href="../../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks</span></code></a></li>
- <li class="breadcrumb-item"><a href="index.html"> Provider packages</a></li>
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.athena</span></code></a></li>
</ul>
</div>
@@ -541,440 +611,232 @@
<div class="bodywrapper">
<div class="body" role="main">
- <blockquote>
-<div></div></blockquote>
-<div class="section" id="provider-packages">
-<h1>Provider packages<a class="headerlink" href="#provider-packages" title="Permalink to this heading">¶</a></h1>
-<p>Apache Airflow 2 is built in modular way. The “Core” of Apache Airflow provides core scheduler
-functionality which allow you to write some basic tasks, but the capabilities of Apache Airflow can
-be extended by installing additional packages, called <code class="docutils literal notranslate"><span class="pre">providers</span></code>.</p>
-<p>Providers can contain operators, hooks, sensor, and transfer operators to communicate with a
-multitude of external systems, but they can also extend Airflow core with new capabilities.</p>
-<p>You can install those provider packages separately in order to interface with a given service. The providers
-for <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> are designed in the way that you can write your own providers easily. The
-<code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span> <span class="pre">Community</span></code> develops and maintain more than 60 provider packages, but you are free to
-develop your own providers - the providers you build have exactly the same capability as the providers
-written by the community, so you can release and share those providers with others.</p>
-<p>The full list of community managed providers is available at
-<a class="reference external" href="https://airflow.apache.org/docs/#providers-packages-docs-apache-airflow-providers-index-html">Providers Index</a>.</p>
-<p>You can also see index of all community provider’s operators and hooks in
-<a class="reference internal" href="operators-and-hooks-ref/index.html"><span class="doc">Operators and Hooks Reference</span></a></p>
-</div>
-<div class="section" id="extending-airflow-core-functionality">
-<h1>Extending Airflow core functionality<a class="headerlink" href="#extending-airflow-core-functionality" title="Permalink to this heading">¶</a></h1>
-<p>Providers give you the capability of extending core Airflow with extra capabilities. The Core airflow
-provides basic and solid functionality of scheduling, the providers extend its capabilities. Here we
-describe all the custom capabilities.</p>
-<p>Airflow automatically discovers which providers add those additional capabilities and, once you install
-provider package and re-start Airflow, those become automatically available to Airflow Users.</p>
-<p>The summary of the core functionalities that can be extended are available in
-<a class="reference internal" href="core-extensions/index.html"><span class="doc">Core Extensions</span></a>.</p>
-<div class="section" id="auth-backends">
-<h2>Auth backends<a class="headerlink" href="#auth-backends" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom authentication backends, that allow you to configure the way how your
-web server authenticates your users, integrating it with public or private authentication services.</p>
-<p>You can see all the authentication backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/auth-backends.html"><span class="doc">Auth backends</span></a></p>
-</div>
-<div class="section" id="custom-connections">
-<h2>Custom connections<a class="headerlink" href="#custom-connections" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom connection types, extending connection form and handling custom form field
-behaviour for the connections defined by the provider.</p>
-<p>You can see all custom connections available via community-managed providers in
-<a class="reference internal" href="core-extensions/connections.html"><span class="doc">Connections</span></a>.</p>
-</div>
-<div class="section" id="extra-links">
-<h2>Extra links<a class="headerlink" href="#extra-links" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add extra custom links to operators delivered by the provider. Those will be visible in
-task details view of the task.</p>
-<p>You can see all the extra links available via community-managed providers in
-<a class="reference internal" href="core-extensions/extra-links.html"><span class="doc">Extra Links</span></a>.</p>
-</div>
-<div class="section" id="logging">
-<h2>Logging<a class="headerlink" href="#logging" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add additional task logging capabilities. By default <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> saves logs for
-tasks locally and make them available to Airflow UI via internal http server. However, providers
-can add extra logging capabilities, where Airflow Logs can be written to a remote service and
-retrieved from those services.</p>
-<p>You can see all task loggers available via community-managed providers in
-<a class="reference internal" href="core-extensions/logging.html"><span class="doc">Writing logs</span></a>.</p>
-</div>
-<div class="section" id="secret-backends">
-<h2>Secret backends<a class="headerlink" href="#secret-backends" title="Permalink to this heading">¶</a></h2>
-<p>Airflow has the capability of reading connections, variables and configuration from Secret Backends rather
-than from its own Database.</p>
-<p>You can see all secret backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/secrets-backends.html"><span class="doc">Secret backends</span></a>.</p>
-</div>
-<div class="section" id="notifications">
-<h2>Notifications<a class="headerlink" href="#notifications" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom notifications, that allow you to configure the way how you would like to receive
-notifications about the status of your tasks/DAGs.</p>
-<p>You can see all the notifications available via community-managed providers in
-<a class="reference internal" href="core-extensions/notifications.html"><span class="doc">Notifications</span></a>.</p>
-</div>
-</div>
-<div class="section" id="installing-and-upgrading-providers">
-<h1>Installing and upgrading providers<a class="headerlink" href="#installing-and-upgrading-providers" title="Permalink to this heading">¶</a></h1>
-<p>Separate provider packages give the possibilities that were not available in 1.10:</p>
-<ol class="arabic simple">
-<li><p>You can upgrade to latest version of particular providers without the need of Apache Airflow core upgrade.</p></li>
-<li><p>You can downgrade to previous version of particular provider in case the new version introduces
-some problems, without impacting the main Apache Airflow core package.</p></li>
-<li><p>You can release and upgrade/downgrade provider packages incrementally, independent from each other. This
-means that you can incrementally validate each of the provider package update in your environment,
-following the usual tests you have in your environment.</p></li>
-</ol>
-</div>
-<div class="section" id="types-of-providers">
-<h1>Types of providers<a class="headerlink" href="#types-of-providers" title="Permalink to this heading">¶</a></h1>
-<p>Providers have the same capacity - no matter if they are provided by the community or if they are
-third-party providers. This chapter explains how community managed providers are versioned and released
-and how you can create your own providers.</p>
-<div class="section" id="community-maintained-providers">
-<h2>Community maintained providers<a class="headerlink" href="#community-maintained-providers" title="Permalink to this heading">¶</a></h2>
-<p>From the point of view of the community, Airflow is delivered in multiple, separate packages.
-The core of Airflow scheduling system is delivered as <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package and there are more than
-60 provider packages which can be installed separately as so called <code class="docutils literal notranslate"><span class="pre">Airflow</span> <span class="pre">Provider</span> <span class="pre">packages</span></code>.
-Those packages are available as <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers</span></code> packages - for example there is an
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> or <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> package).</p>
-<p>Community maintained providers are released and versioned separately from the Airflow releases. We are
-following the <a class="reference external" href="https://semver.org/">Semver</a> versioning scheme for the packages. Some versions of the
-provider packages might depend on particular versions of Airflow, but the general approach we have is that
-unless there is a good reason, new version of providers should work with recent versions of Airflow 2.x.
-Details will vary per-provider and if there is a limitation for particular version of particular provider,
-constraining the Airflow version used, it will be included as limitation of dependencies in the provider
-package.</p>
-<p>Each community provider has corresponding extra which can be used when installing airflow to install the
-provider together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> - for example you can install airflow with those extras:
-<code class="docutils literal notranslate"><span class="pre">apache-airflow[google,amazon]</span></code> (with correct constraints -see <a class="reference external" href="/docs/apache-airflow/stable/installation/index.html" title="(in apache-airflow v2.6.0.dev0)"><span>Installation</span></a>) and you
-will install the appropriate versions of the <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> and
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> packages together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>.</p>
-<p>Some of the community providers have cross-provider dependencies as well. Those are not required
-dependencies, they might simply enable certain features (for example transfer operators often create
-dependency between different providers. Again, the general approach here is that the providers are backwards
-compatible, including cross-dependencies. Any kind of breaking changes and requirements on particular versions of other
-provider packages are automatically documented in the release notes of every provider.</p>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>For Airflow 1.10 We also provided <code class="docutils literal notranslate"><span class="pre">apache-airflow-backport-providers</span></code> packages that could be installed
-with those versions Those were the same providers as for 2.0 but automatically back-ported to work for
-Airflow 1.10. The last release of backport providers was done on March 17, 2021 and the backport
-providers will no longer be released, since Airflow 1.10 has reached End-Of-Life as of June 17, 2021.</p>
-</div>
-<p>If you want to contribute to <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>, you can see how to build and extend community
-managed providers in <a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a>.</p>
-</div>
-<div class="section" id="custom-provider-packages">
-<span id="providers-community-maintained-providers"></span><h2>Custom provider packages<a class="headerlink" href="#custom-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p>You can develop and release your own providers. Your custom operators, hooks, sensors, transfer operators
-can be packaged together in a standard airflow package and installed using the same mechanisms.
-Moreover they can also use the same mechanisms to extend the Airflow Core with auth backends,
-custom connections, logging, secret backends and extra operator links as described in the previous chapter.</p>
-</div>
-</div>
-<div class="section" id="how-to-create-your-own-provider">
-<h1>How to create your own provider<a class="headerlink" href="#how-to-create-your-own-provider" title="Permalink to this heading">¶</a></h1>
-<p>As mentioned in the <a class="reference external" href="http://airflow.apache.org/docs/apache-airflow-providers/index.html">Providers</a>
-documentation, custom providers can extend Airflow core - they can add extra links to operators as well
-as custom connections. You can use build your own providers and install them as packages if you would like
-to use the mechanism for your own, custom providers.</p>
-<p>Adding a provider to Airflow is just a matter of building a Python package and adding the right meta-data to
-the package. We are using standard mechanism of python to define
-<a class="reference external" href="https://docs.python.org/3/library/importlib.metadata.html#entry-points">entry points</a> . Your package
-needs to define appropriate entry-point <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> which has to point to a callable
-implemented by your package and return a dictionary containing the list of discoverable capabilities
-of your package. The dictionary has to follow the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json">json-schema specification</a>.</p>
-<p>Most of the schema provides extension point for the documentation (which you might want to also use for
-your own purpose) but the important fields from the extensibility point of view are those:</p>
-<p>Displaying package information in CLI/API:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">package-name</span></code> - Name of the package for the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code> - Human-friendly name of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">description</span></code> - Additional description of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">version</span></code> - List of versions of the package (in reverse-chronological order). The first version in the
-list is the current package version. It is taken from the version of package installed, not from the
-provider_info information.</p></li>
+ <div class="section" id="module-airflow.providers.amazon.aws.hooks.athena">
+<span id="airflow-providers-amazon-aws-hooks-athena"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.hooks.athena" title="airflow.providers.amazon.aws.hooks.athena"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.athena</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.hooks.athena" title="Permalink to this heading">¶</a></h1>
+<p>This module contains AWS Athena hook.</p>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<div class="section" id="classes">
+<h3>Classes<a class="headerlink" href="#classes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook" title="airflow.providers.amazon.aws.hooks.athena.AthenaHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">AthenaHook</span></code></a></p></td>
+<td><p>Interact with Amazon Athena.</p></td>
+</tr>
+</tbody>
+</table>
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.athena.</span></span><span class="sig-name descname"><span class="pre">AthenaHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">sleep_ [...]
+<dd><p>Bases: <a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p>
+<p>Interact with Amazon Athena.
+Provide thick wrapper around <a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena.html#Athena.Client" title="(in Boto3 v1.26.120)"><code class="xref py py-class docutils literal notranslate"><span class="pre">boto3.client("athena")</span></code></a>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>sleep_time</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a>) – Time (in seconds) to wait between two consecutive calls to check query status on Athena</p></li>
+<li><p><strong>log_query</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a>) – Whether to log athena query and other execution params when it’s executed.
+Defaults to <em>True</em>.</p></li>
</ul>
-<p>Exposing customized functionality to the Airflow’s core:</p>
+</dd>
+</dl>
+<p>Additional arguments (such as <code class="docutils literal notranslate"><span class="pre">aws_conn_id</span></code>) may be specified and
+are passed down to the underlying AwsBaseHook.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">extra-links</span></code> - this field should contain the list of all operator class names that are adding extra links
-capability. See <a class="reference external" href="/docs/apache-airflow/stable/howto/define-extra-link.html" title="(in apache-airflow v2.6.0.dev0)"><span>Define an operator extra link</span></a> for description of how to add extra link
-capability to the operators of yours.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">connection-types</span></code> - this field should contain the list of all connection types together with hook
-class names implementing those custom connection types (providing custom extra fields and
-custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
-<code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> (deprecated) - this field should contain the list of all hook class names that provide
-custom connection types with custom extra fields and field behaviour. The <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array
-is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
-providers are targeting Airflow 2.2.0+ you do not have to include the <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array, if
-you want to also target earlier versions of Airflow 2, you should include both <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> and
-<code class="docutils literal notranslate"><span class="pre">connection-types</span></code> arrays. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details.</p></li>
+<li><p><a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-class docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p></li>
</ul>
-<p>When your providers are installed you can query the installed providers and their capabilities with the
-<code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">providers</span></code> command. This way you can verify if your providers are properly recognized and whether
-they define the extensions properly. See <a class="reference external" href="/docs/apache-airflow/stable/cli-and-env-variables-ref.html" title="(in apache-airflow v2.6.0.dev0)"><span>Command Line Interface and Environment Variables Reference</span></a> for details of available CLI
-sub-commands.</p>
-<p>When you write your own provider, consider following the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#naming-conventions-for-provider-packages">Naming conventions for provider packages</a></p>
-</div>
-<div class="section" id="faq-for-airflow-and-providers">
-<h1>FAQ for Airflow and Providers<a class="headerlink" href="#faq-for-airflow-and-providers" title="Permalink to this heading">¶</a></h1>
-<div class="section" id="upgrading-airflow-2-0-and-providers">
-<h2>Upgrading Airflow 2.0 and Providers<a class="headerlink" href="#upgrading-airflow-2-0-and-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When upgrading to a new Airflow version such as 2.0, but possibly 2.0.1 and beyond, is the best practice
-to also upgrade provider packages at the same time?</strong></p>
-<p>It depends on your use case. If you have automated or semi-automated verification of your installation,
-that you can run a new version of Airflow including all provider packages, then definitely go for it.
-If you rely more on manual testing, it is advised that you upgrade in stages. Depending on your choice
-you can either upgrade all used provider packages first, and then upgrade Airflow Core or the other way
-round. The first approach - when you first upgrade all providers is probably safer, as you can do it
-incrementally, step-by-step replacing provider by provider in your environment.</p>
</div>
-<div class="section" id="customizing-provider-packages">
-<h2>Customizing Provider Packages<a class="headerlink" href="#customizing-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an older version of my provider package which we have lightly customized and is working
-fine with my MSSQL installation. I am upgrading my Airflow version. Do I need to upgrade my provider,
-or can I keep it as it is?</strong></p>
-<p>It depends on the scope of customization. There is no need to upgrade the provider packages to later
-versions unless you want to upgrade to Airflow version that introduces backwards-incompatible changes.
-Generally speaking, with Airflow 2 we are following the <a class="reference external" href="https://semver.org/">Semver</a> approach where
-we will introduce backwards-incompatible changes in Major releases, so all your modifications (as long
-as you have not used internal Airflow classes) should work for All Airflow 2.* versions.</p>
-</div>
-<div class="section" id="creating-your-own-providers">
-<h2>Creating your own providers<a class="headerlink" href="#creating-your-own-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When I write my own provider, do I need to do anything special to make it available to others?</strong></p>
-<p>You do not need to do anything special besides creating the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point
-returning properly formatted meta-data - dictionary with <code class="docutils literal notranslate"><span class="pre">extra-links</span></code> and <code class="docutils literal notranslate"><span class="pre">connection-types</span></code> fields
-(and deprecated <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> field if you are also targeting versions of Airflow before 2.2.0).</p>
-<p>Anyone who runs airflow in an environment that has your Python package installed will be able to use the
-package as a provider package.</p>
-<p><strong>What do I need to do to turn a package into a provider?</strong></p>
-<p>You need to do the following to turn an existing Python package into a provider (see below for examples):</p>
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.INTERMEDIATE_STATES">
+<span class="sig-name descname"><span class="pre">INTERMEDIATE_STATES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('QUEUED',</span> <span class="pre">'RUNNING')</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/athena.html#AthenaHook.INTERMEDIATE_STATES"><span class="viewcode-link"><span class="pre">[source]</span></span></ [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.FAILURE_STATES">
+<span class="sig-name descname"><span class="pre">FAILURE_STATES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('FAILED',</span> <span class="pre">'CANCELLED')</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/athena.html#AthenaHook.FAILURE_STATES"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a cla [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.SUCCESS_STATES">
+<span class="sig-name descname"><span class="pre">SUCCESS_STATES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('SUCCEEDED',)</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/athena.html#AthenaHook.SUCCESS_STATES"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.pro [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.TERMINAL_STATES">
+<span class="sig-name descname"><span class="pre">TERMINAL_STATES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('SUCCEEDED',</span> <span class="pre">'FAILED',</span> <span class="pre">'CANCELLED')</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/athena.html#AthenaHook.TERMINAL_STATES"><span class="viewcode-link"><span clas [...]
+<dd></dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.run_query">
+<span class="sig-name descname"><span class="pre">run_query</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">query</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">query_context</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">result_configuration</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">client_request_token</span></span><span class="o"><span class= [...]
+<dd><p>Run Presto query on athena with provided config and return submitted query_execution_id</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Add the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point in the <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> - this tells airflow where to get
-the required provider metadata</p></li>
-<li><p>Create the function that you refer to in the first step as part of your package: this functions returns a
-dictionary that contains all meta-data about your provider package</p></li>
-<li><p>If you want Airflow to link to documentation of your Provider in the providers page, make sure
-to add “project-url/documentation” <a class="reference external" href="https://peps.python.org/pep-0621/#example">metadata</a> to your package.
-This will also add link to your documentation in PyPI.</p></li>
-<li><p>note that the dictionary should be compliant with <code class="docutils literal notranslate"><span class="pre">airflow/provider_info.schema.json</span></code> JSON-schema
-specification. The community-managed providers have more fields there that are used to build
-documentation, but the requirement for runtime information only contains several fields which are defined
-in the schema:</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/client/start_query_execution.html#Athena.Client.start_query_execution" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Athena.Client.start_query_execution()</span></code></a></p></li>
</ul>
-<div class="example-block-wrapper compound">
-<p class="example-header compound-first"><span class="example-title">airflow/provider_info.schema.json</span></p>
-<div class="compound-last highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"$schema"</span><span class="p">:</span><span class="w"> </span><span class="s2">"http://json-schema.org/draft-07/schema#"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"package-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Package name available under which the package is available in the PyPI repository."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Provider name"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Information about the package in RST format"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-names"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class names that provide connection types to core (deprecated by connection-types)"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"deprecated"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"The hook-class-names property has been deprecated in favour of connection-types which is more performant version allowing to only import individual Hooks rather than all hooks at once"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"deprecatedVersion"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.2.0"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"connection-types"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Map of connection types mapped to hook class names."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"connection-type"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Type of connection defined by the provider"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class name that implements the connection type"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"connection-type"</span><span class="p">,</span><span class="w"> </span><span class="s2">"hook-class-name"</span><span class="p">]</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"extra-links"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Operator class names that provide extra link functionality"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"secrets-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Secrets Backend class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"logging"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Logging Task Handlers class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"auth-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"API Auth Backend module names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"notifications"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Notification class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"task-decorators"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Apply custom decorators to the TaskFlow API. Can be accessed by users via '@task.<name>'"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"path"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
-<span class="w"> </span><span class="s2">"name"</span><span class="p">,</span>
-<span class="w"> </span><span class="s2">"description"</span>
-<span class="w"> </span><span class="p">]</span>
-<span class="p">}</span>
-</pre></div>
</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>query</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Presto query to run</p></li>
+<li><p><strong>query_context</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a><em>[</em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em>, </em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em>]</em>) – Context in which query [...]
+<li><p><strong>result_configuration</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a><em>[</em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em>, </em><em>Any</em><em>]</em>) – Dict with path to store results in and config related to encryption</p></li>
+<li><p><strong>client_request_token</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – Unique token created by user to avoid multiple executions of same query</p></li>
+<li><p><strong>workgroup</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Athena workgroup name, when not specified, will be ‘primary’</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.check_query_status">
+<span class="sig-name descname"><span class="pre">check_query_status</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">query_execution_id</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/athena.html#AthenaHook.check_query_status"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow. [...]
+<dd><p>Fetch the status of submitted athena query. Returns None or one of valid query states.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/client/get_query_execution.html#Athena.Client.get_query_execution" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Athena.Client.get_query_execution()</span></code></a></p></li>
+</ul>
</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>:</p>
-<div class="highlight-cfg notranslate"><div class="highlight"><pre><span></span><span class="k">[options.entry_points]</span>
-<span class="c1"># the function get_provider_info is defined in myproviderpackage.somemodule</span>
-<span class="na">apache_airflow_provider</span><span class="o">=</span>
-<span class="w"> </span><span class="na">provider_info</span><span class="o">=</span><span class="s">myproviderpackage.somemodule:get_provider_info</span>
-</pre></div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>query_execution_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Id of submitted athena query</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.get_state_change_reason">
+<span class="sig-name descname"><span class="pre">get_state_change_reason</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">query_execution_id</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/athena.html#AthenaHook.get_state_change_reason"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href= [...]
+<dd><p>Fetch the reason for a state change (e.g. error message). Returns None or reason string.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/client/get_query_execution.html#Athena.Client.get_query_execution" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Athena.Client.get_query_execution()</span></code></a></p></li>
+</ul>
</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">myproviderpackage/somemodule.py</span></code>:</p>
-<div class="highlight-Python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_provider_info</span><span class="p">():</span>
- <span class="k">return</span> <span class="p">{</span>
- <span class="s2">"package-name"</span><span class="p">:</span> <span class="s2">"my-package-name"</span><span class="p">,</span>
- <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"name"</span><span class="p">,</span>
- <span class="s2">"description"</span><span class="p">:</span> <span class="s2">"a description"</span><span class="p">,</span>
- <span class="s2">"hook-class-names"</span><span class="p">:</span> <span class="p">[</span>
- <span class="s2">"myproviderpackage.hooks.source.SourceHook"</span><span class="p">,</span>
- <span class="p">],</span>
- <span class="p">}</span>
-</pre></div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>query_execution_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Id of submitted athena query</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.get_query_results">
+<span class="sig-name descname"><span class="pre">get_query_results</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">query_execution_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">next_token_id</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_results</span></span> [...]
+<dd><p>Fetch submitted athena query results. returns none if query is in intermediate state or
+failed/cancelled state else dict of query output</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/client/get_query_results.html#Athena.Client.get_query_results" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Athena.Client.get_query_results()</span></code></a></p></li>
+</ul>
</div>
-<p><strong>How do provider packages work under the hood?</strong></p>
-<p>When running Airflow with your provider package, there will be (at least) three components to your airflow installation:</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>query_execution_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Id of submitted athena query</p></li>
+<li><p><strong>next_token_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – The token that specifies where to start pagination.</p></li>
+<li><p><strong>max_results</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a>) – The maximum number of results (rows) to return in this request.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.get_query_results_paginator">
+<span class="sig-name descname"><span class="pre">get_query_results_paginator</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">query_execution_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_items</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">page_size</span></s [...]
+<dd><p>Fetch submitted athena query results. returns none if query is in intermediate state or
+failed/cancelled state else a paginator to iterate through pages of results. If you
+wish to get all results at once, call build_full_result() on the returned PageIterator</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>The installation itself (for example, a <code class="docutils literal notranslate"><span class="pre">venv</span></code> where you installed airflow with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">apache-airflow</span></code>)
-together with the related files (e.g. <code class="docutils literal notranslate"><span class="pre">dags</span></code> folder)</p></li>
-<li><p>The <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package</p></li>
-<li><p>Your own <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package that is independent of <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> or your airflow installation, which
-can be a local Python package (that you install via <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span> <span class="pre">/path/to/my-package</span></code>), a normal pip package
-(<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">myproviderpackage</span></code>), or any other type of Python package</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/paginator/GetQueryResults.html#Athena.Paginator.GetQueryResults" title="(in Boto3 v1.26.120)"><code class="xref py py-class docutils literal notranslate"><span class="pre">Athena.Paginator.GetQueryResults</span></code></a></p></li>
+</ul>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>query_execution_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Id of submitted athena query</p></li>
+<li><p><strong>max_items</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><em>None</em>) – The total number of items to return.</p></li>
+<li><p><strong>page_size</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><em>None</em>) – The size of each page.</p></li>
+<li><p><strong>starting_token</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – A token to specify where to start paginating.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.poll_query_status">
+<span class="sig-name descname"><span class="pre">poll_query_status</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">query_execution_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_polling_attempts</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../.. [...]
+<dd><p>Poll the status of submitted athena query until query state reaches final state.
+Returns one of the final states</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>query_execution_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Id of submitted athena query</p></li>
+<li><p><strong>max_polling_attempts</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><em>None</em>) – Number of times to poll for query state before function exits</p></li>
</ul>
-<p>In the <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package you need to add the entry point and provide the appropriate metadata as described above.
-If you have done that, airflow does the following at runtime:</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.get_output_location">
+<span class="sig-name descname"><span class="pre">get_output_location</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">query_execution_id</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/athena.html#AthenaHook.get_output_location"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflo [...]
+<dd><p>Function to get the output location of the query results
+in s3 uri format.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Loop through ALL packages installed in your environment / <code class="docutils literal notranslate"><span class="pre">venv</span></code></p></li>
-<li><p>For each package, if the package’s <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> has a section <code class="docutils literal notranslate"><span class="pre">[options.entry_points]</span></code>, and if that section has a value
-for <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code>, then get the value for <code class="docutils literal notranslate"><span class="pre">provider_info</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code></p></li>
-<li><p>That value works like an import statement: <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code> translates to something like
-<code class="docutils literal notranslate"><span class="pre">from</span> <span class="pre">myproviderpackage.somemodule</span> <span class="pre">import</span> <span class="pre">get_provider_info</span></code>, and the <code class="docutils literal notranslate"><span class="pre">get_provider_info</span></code> that is being imported should be a
-callable, i.e. a function</p></li>
-<li><p>This function should return a dictionary with metadata</p></li>
-<li><p>If you have custom connection types as part of your package, that metadata will including a field called <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>
-which should be a list of strings of your custom hooks - those strings should also be in an import-like format, e.g.
-<code class="docutils literal notranslate"><span class="pre">myproviderpackage.hooks.source.SourceHook</span></code> means that there is a class <code class="docutils literal notranslate"><span class="pre">SourceHook</span></code> in <code class="docutils literal notranslate"><span class="pre">myproviderpackage/hooks/source.py</span></code>
-- airflow then imports these hooks and looks for the functions <code class="docutils literal notranslate"><span class="pre">get_ui_field_behaviour</span></code> and <code class="docutils literal notranslate"><span class="pre">get_connection_form_widgets</span></code>
-(both optional) as well as the attributes <code class="docutils literal notranslate"><span class="pre">conn_type</span></code> and <code class="docutils literal notranslate"><span class="pre">hook_name</span></code> to create the custom connection type in the airflow UI</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/client/get_query_execution.html#Athena.Client.get_query_execution" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Athena.Client.get_query_execution()</span></code></a></p></li>
</ul>
-<p><strong>Should I name my provider specifically or should it be created in ``airflow.providers`` package?</strong></p>
-<p>We have quite a number (>60) of providers managed by the community and we are going to maintain them
-together with Apache Airflow. All those providers have well-defined structured and follow the
-naming conventions we defined and they are all in <code class="docutils literal notranslate"><span class="pre">airflow.providers</span></code> package. If your intention is
-to contribute your provider, then you should follow those conventions and make a PR to Apache Airflow
-to contribute to it. But you are free to use any package name as long as there are no conflicts with other
-names, so preferably choose package that is in your “domain”.</p>
-<p><strong>Is there a convention for a connection id and type?</strong></p>
-<p>Very good question. Glad that you asked. We usually follow the convention <code class="docutils literal notranslate"><span class="pre"><NAME>_default</span></code> for connection
-id and just <code class="docutils literal notranslate"><span class="pre"><NAME></span></code> for connection type. Few examples:</p>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>query_execution_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Id of submitted athena query</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.athena.AthenaHook.stop_query">
+<span class="sig-name descname"><span class="pre">stop_query</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">query_execution_id</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/athena.html#AthenaHook.stop_query"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon [...]
+<dd><p>Cancel the submitted athena query</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">google_cloud_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">google_cloud_platform</span></code> type</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">aws_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">aws</span></code> type</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/client/stop_query_execution.html#Athena.Client.stop_query_execution" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Athena.Client.stop_query_execution()</span></code></a></p></li>
</ul>
-<p>You should follow this convention. It is important, to use unique names for connection type,
-so it should be unique for your provider. If two providers try to add connection with the same type
-only one of them will succeed.</p>
-<p><strong>Can I contribute my own provider to Apache Airflow?</strong></p>
-<p>Of course, but it’s better to check at developer’s mailing list whether such contribution will be accepted by
-the Community, before investing time to make the provider compliant with community requirements.
-The Community only accepts providers that are generic enough, are well documented, fully covered by tests
-and with capabilities of being tested by people in the community. So we might not always be in the
-position to accept such contributions.</p>
-<p>After you think that your provider matches the expected values above, you can read
-<a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a> to check all prerequisites for a new
-community Provider and discuss it at the <a class="reference external" href="http://airflow.apache.org/community/">Devlist</a>.</p>
-<p>However, in case you have your own, specific provider, which you can maintain on your own or by your
-team, you are free to publish the providers in whatever form you find appropriate. The custom and
-community-managed providers have exactly the same capabilities.</p>
-<p><strong>Can I advertise my own provider to Apache Airflow users and share it with others as package in PyPI?</strong></p>
-<p>Absolutely! We have an <a class="reference external" href="https://airflow.apache.org/ecosystem/">Ecosystem</a> area on our website where
-we share non-community managed extensions and work for Airflow. Feel free to make a PR to the page and
-add we will evaluate and merge it when we see that such provider can be useful for the community of
-Airflow users.</p>
-<p><strong>Can I charge for the use of my provider?</strong></p>
-<p>This is something that is outside of our control and domain. As an Apache project, we are
-commercial-friendly and there are many businesses built around Apache Airflow and many other
-Apache projects. As a community, we provide all the software for free and this will never
-change. What 3rd-party developers are doing is not under control of Apache Airflow community.</p>
</div>
-<div class="section" id="using-backport-providers-in-airflow-1-10">
-<h2>Using Backport Providers in Airflow 1.10<a class="headerlink" href="#using-backport-providers-in-airflow-1-10" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an Airflow version (1.10.12) running and it is stable. However, because of a Cloud provider change,
-I would like to upgrade the provider package. If I don’t need to upgrade the Airflow version anymore,
-how do I know that this provider version is compatible with my Airflow version?</strong></p>
-<p>We have Backport Providers are compatible with 1.10 but they stopped being released on
-March 17, 2021. Since then, no new changes to providers for Airflow 2.0 are going to be
-released as backport packages. It’s the highest time to upgrade to Airflow 2.0.</p>
-<p>When it comes to compatibility of providers with different Airflow 2 versions, each
-provider package will keep its own dependencies, and while we expect those providers to be generally
-backwards-compatible, particular versions of particular providers might introduce dependencies on
-specific Airflow versions.</p>
-<div class="toctree-wrapper compound">
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>query_execution_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Id of submitted athena query</p>
+</dd>
+</dl>
+</dd></dl>
+
+</dd></dl>
+
</div>
</div>
</div>
- <div class="pager" role="navigation" aria-label="related navigation"><button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Previous</button>
- <a rel="next" title="Providers packages reference" href="packages-ref.html" >
+ <div class="pager" role="navigation" aria-label="related navigation">
+ <a rel="prev" title="airflow.providers.amazon.aws.hooks.appflow" href="../appflow/index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="p">Previous</button>
+ </a>
+ <a rel="next" title="airflow.providers.amazon.aws.hooks.base_aws" href="../base_aws/index.html" >
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="n">Next</button>
</a>
</div>
@@ -1044,28 +906,28 @@ specific Airflow versions.</p>
<nav class="wy-nav-side-toc">
<div class="wy-menu-vertical">
<ul>
-<li><a class="reference internal" href="#">Provider packages</a></li>
-<li><a class="reference internal" href="#extending-airflow-core-functionality">Extending Airflow core functionality</a><ul>
-<li><a class="reference internal" href="#auth-backends">Auth backends</a></li>
-<li><a class="reference internal" href="#custom-connections">Custom connections</a></li>
-<li><a class="reference internal" href="#extra-links">Extra links</a></li>
-<li><a class="reference internal" href="#logging">Logging</a></li>
-<li><a class="reference internal" href="#secret-backends">Secret backends</a></li>
-<li><a class="reference internal" href="#notifications">Notifications</a></li>
+<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.athena</span></code></a><ul>
+<li><a class="reference internal" href="#module-contents">Module Contents</a><ul>
+<li><a class="reference internal" href="#classes">Classes</a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook"><code class="docutils literal notranslate"><span class="pre">AthenaHook</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.INTERMEDIATE_STATES"><code class="docutils literal notranslate"><span class="pre">AthenaHook.INTERMEDIATE_STATES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.FAILURE_STATES"><code class="docutils literal notranslate"><span class="pre">AthenaHook.FAILURE_STATES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.SUCCESS_STATES"><code class="docutils literal notranslate"><span class="pre">AthenaHook.SUCCESS_STATES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.TERMINAL_STATES"><code class="docutils literal notranslate"><span class="pre">AthenaHook.TERMINAL_STATES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.run_query"><code class="docutils literal notranslate"><span class="pre">AthenaHook.run_query()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.check_query_status"><code class="docutils literal notranslate"><span class="pre">AthenaHook.check_query_status()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.get_state_change_reason"><code class="docutils literal notranslate"><span class="pre">AthenaHook.get_state_change_reason()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.get_query_results"><code class="docutils literal notranslate"><span class="pre">AthenaHook.get_query_results()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.get_query_results_paginator"><code class="docutils literal notranslate"><span class="pre">AthenaHook.get_query_results_paginator()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.poll_query_status"><code class="docutils literal notranslate"><span class="pre">AthenaHook.poll_query_status()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.get_output_location"><code class="docutils literal notranslate"><span class="pre">AthenaHook.get_output_location()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.athena.AthenaHook.stop_query"><code class="docutils literal notranslate"><span class="pre">AthenaHook.stop_query()</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
-<li><a class="reference internal" href="#installing-and-upgrading-providers">Installing and upgrading providers</a></li>
-<li><a class="reference internal" href="#types-of-providers">Types of providers</a><ul>
-<li><a class="reference internal" href="#community-maintained-providers">Community maintained providers</a></li>
-<li><a class="reference internal" href="#custom-provider-packages">Custom provider packages</a></li>
</ul>
</li>
-<li><a class="reference internal" href="#how-to-create-your-own-provider">How to create your own provider</a></li>
-<li><a class="reference internal" href="#faq-for-airflow-and-providers">FAQ for Airflow and Providers</a><ul>
-<li><a class="reference internal" href="#upgrading-airflow-2-0-and-providers">Upgrading Airflow 2.0 and Providers</a></li>
-<li><a class="reference internal" href="#customizing-provider-packages">Customizing Provider Packages</a></li>
-<li><a class="reference internal" href="#creating-your-own-providers">Creating your own providers</a></li>
-<li><a class="reference internal" href="#using-backport-providers-in-airflow-1-10">Using Backport Providers in Airflow 1.10</a></li>
</ul>
</li>
</ul>
@@ -1090,7 +952,7 @@ specific Airflow versions.</p>
<div class="base-layout--button">
- <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers/index.rst" rel="nofollow">
+ <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-amazon/_api/airflow/providers/amazon/aws/hooks/athena/index.rst" rel="nofollow">
<button class="btn-hollow btn-brown btn-with-icon with-box-shadow button-fixed">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
@@ -1224,19 +1086,19 @@ specific Airflow versions.</p>
</span>
</div>
</footer>
- <script type="text/javascript" src="_static/_gen/js/docs.js"></script>
- <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
+ <script type="text/javascript" src="../../../../../../../_static/_gen/js/docs.js"></script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../../../../" src="../../../../../../../_static/documentation_options.js"></script>
<script src="https://code.jquery.com/jquery-3.4.1.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js" integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy" crossorigin="anonymous"></script>
- <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
- <script src="_static/jquery.js"></script>
- <script src="_static/underscore.js"></script>
- <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
- <script src="_static/doctools.js"></script>
- <script src="_static/sphinx_highlight.js"></script>
- <script src="_static/clipboard.min.js"></script>
- <script src="_static/copybutton.js"></script>
- <script src="_static/js/globaltoc.js"></script>
+ <script data-url_root="../../../../../../../" id="documentation_options" src="../../../../../../../_static/documentation_options.js"></script>
+ <script src="../../../../../../../_static/jquery.js"></script>
+ <script src="../../../../../../../_static/underscore.js"></script>
+ <script src="../../../../../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
+ <script src="../../../../../../../_static/doctools.js"></script>
+ <script src="../../../../../../../_static/sphinx_highlight.js"></script>
+ <script src="../../../../../../../_static/clipboard.min.js"></script>
+ <script src="../../../../../../../_static/copybutton.js"></script>
+ <script src="../../../../../../../_static/js/globaltoc.js"></script>
</body>
</html>
\ No newline at end of file
diff --git a/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/base_aws/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/base_aws/index.html
new file mode 100644
index 0000000000..7ea269a4bc
--- /dev/null
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/base_aws/index.html
@@ -0,0 +1,1453 @@
+
+
+
+
+<!DOCTYPE html>
+<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
+ <html xmlns="http://www.w3.org/1999/xhtml" lang="en">
+<head>
+ <meta charset="utf-8" />
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>airflow.providers.amazon.aws.hooks.base_aws — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../../search.html" />
+ <link rel="next" title="airflow.providers.amazon.aws.hooks.batch_client" href="../batch_client/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.hooks.athena" href="../athena/index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/hooks/base_aws/index.html" />
+
+ <!-- Matomo -->
+ <script>
+ var _paq = window._paq = window._paq || [];
+ /* tracker methods like "setCustomDimension" should be called before "trackPageView" */
+ /* We explicitly disable cookie tracking to avoid privacy issues */
+ _paq.push(['disableCookies']);
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="https://analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '13']);
+ var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+ <!-- End Matomo Code -->
+
+
+</head><body class="td-section">
+
+
+<header>
+ <nav class="js-navbar-scroll navbar">
+ <div class="navbar__icon-container">
+ <a href="/">
+ <svg xmlns="http://www.w3.org/2000/svg" width="155.314" height="60" viewBox="0 0 155.314 60">
+ <defs>
+ <clipPath id="clip-path">
+ <path id="Rectangle_1" d="M0 0h155.314v60H0z" fill="none" data-name="Rectangle 1"></path>
+ </clipPath>
+ </defs>
+ <g id="logo" transform="translate(-1305 -780.355)">
+ <g id="Group_2" clip-path="url(#clip-path)" data-name="Group 2" transform="translate(1305 780.355)">
+ <g id="Group_1" data-name="Group 1" transform="translate(.486 .486)">
+ <path id="Path_1" d="M1307.562 880.867l28.187-28.893a.521.521 0 0 0 .063-.666c-1.714-2.393-4.877-2.808-6.049-4.416-3.472-4.763-4.353-7.459-5.845-7.292a.456.456 0 0 0-.271.143l-10.182 10.438c-5.858 6-6.7 19.225-6.852 30.3a.552.552 0 0 0 .949.386z" fill="#017cee" data-name="Path 1" transform="translate(-1306.613 -822.232)"></path>
+ <path id="Path_2" d="M1405.512 908.489l-28.893-28.189a.521.521 0 0 0-.667-.063c-2.393 1.715-2.808 4.877-4.416 6.049-4.763 3.472-7.459 4.353-7.292 5.845a.456.456 0 0 0 .143.27l10.438 10.182c6 5.858 19.225 6.7 30.3 6.852a.552.552 0 0 0 .387-.946z" fill="#00ad46" data-name="Path 2" transform="translate(-1346.876 -850.567)"></path>
+ <path id="Path_3" d="M1373.909 902.252c-3.28-3.2-4.8-9.53 1.486-22.583-10.219 4.567-13.8 10.57-12.039 12.289z" fill="#04d659" data-name="Path 3" transform="translate(-1345.96 -850.233)"></path>
+ <path id="Path_4" d="M1433.132 782.359l-28.186 28.893a.52.52 0 0 0-.063.666c1.715 2.393 4.876 2.808 6.049 4.416 3.472 4.763 4.354 7.459 5.845 7.292a.454.454 0 0 0 .271-.143l10.182-10.438c5.858-6 6.7-19.225 6.852-30.3a.553.553 0 0 0-.95-.386z" fill="#00c7d4" data-name="Path 4" transform="translate(-1375.21 -782.123)"></path>
+ <path id="Path_5" d="M1426.9 881.155c-3.2 3.28-9.53 4.8-22.584-1.486 4.567 10.219 10.57 13.8 12.289 12.039z" fill="#11e1ee" data-name="Path 5" transform="translate(-1374.875 -850.233)"></path>
+ <path id="Path_6" d="M1307 782.919l28.893 28.186a.521.521 0 0 0 .666.063c2.393-1.715 2.808-4.877 4.416-6.049 4.763-3.472 7.459-4.353 7.292-5.845a.459.459 0 0 0-.143-.271l-10.438-10.182c-6-5.858-19.225-6.7-30.3-6.852a.552.552 0 0 0-.386.95z" fill="#e43921" data-name="Path 6" transform="translate(-1306.766 -781.97)"></path>
+ <path id="Path_7" d="M1405.8 804.711c3.28 3.2 4.8 9.53-1.486 22.584 10.219-4.567 13.8-10.571 12.039-12.289z" fill-rule="evenodd" fill="#ff7557" data-name="Path 7" transform="translate(-1374.875 -797.859)"></path>
+ <path id="Path_8" d="M1329.355 849.266c3.2-3.28 9.53-4.8 22.584 1.486-4.567-10.219-10.57-13.8-12.289-12.039z" fill="#0cb6ff" data-name="Path 8" transform="translate(-1322.503 -821.316)"></path>
+ <circle id="Ellipse_1" cx="1.26" cy="1.26" r="1.26" fill="#4a4848" data-name="Ellipse 1" transform="translate(28.18 28.171)"></circle>
+ <path id="Path_9" d="M1527.558 827.347a.229.229 0 0 1-.223-.223.458.458 0 0 1 .011-.123l2.766-7.214a.346.346 0 0 1 .357-.245h.758a.348.348 0 0 1 .357.245l2.754 7.214.022.123a.228.228 0 0 1-.223.223h-.568a.288.288 0 0 1-.19-.056.352.352 0 0 1-.089-.134l-.613-1.583h-3.657l-.613 1.583a.317.317 0 0 1-.1.134.269.269 0 0 1-.178.056zm4.795-2.732l-1.505-3.958-1.505 3.958zm3.322 4.85a.258.258 0 0 1-.189-.078.241.241 0 0 1-.067-.178v-7.4a.241.241 0 0 1 .067-.178.258 [...]
+ <path id="Path_10" d="M1527.2 827.081l-.061.061zm-.056-.279l-.08-.031zm2.766-7.214l.08.031zm1.472 0l-.081.029zm2.754 7.214l.084-.015a.064.064 0 0 0 0-.015zm.022.123h.086v-.015zm-.067.156l.06.061zm-.914.011l-.061.061.006.005zm-.089-.134l.081-.027zm-.613-1.583l.08-.031a.086.086 0 0 0-.08-.055zm-3.657 0v-.086a.086.086 0 0 0-.08.055zm-.613 1.583l-.08-.031zm-.1.134l.055.066zm4.047-2.676v.086a.086.086 0 0 0 .08-.116zm-1.505-3.958l.08-.03a.086.086 0 0 0-.16 0zm-1 [...]
+ <path id="Path_11" d="M1519.066 884.011a.581.581 0 0 1-.567-.567 1.151 1.151 0 0 1 .028-.312l7.026-18.328a.881.881 0 0 1 .906-.623h1.926a.882.882 0 0 1 .907.623l7 18.328.057.312a.583.583 0 0 1-.567.567h-1.445a.735.735 0 0 1-.482-.142.9.9 0 0 1-.226-.34l-1.558-4.023h-9.292l-1.558 4.023a.8.8 0 0 1-.255.34.688.688 0 0 1-.453.142zm12.181-6.94l-3.824-10.056-3.823 10.055zm8.184-10.538a.592.592 0 0 1-.652-.651v-1.53a.714.714 0 0 1 .17-.482.656.656 0 0 1 .482-.2h1 [...]
+ </g>
+ </g>
+ </g>
+ </svg>
+
+ </a>
+ </div>
+ <div class="desktop-only navbar__menu-container">
+
+ <div class="navbar__menu-content" id="main_navbar">
+
+ <div class="navbar__links-container">
+
+ <a class="navbar__text-link" href="/community/">
+ Community
+ </a>
+
+ <a class="navbar__text-link" href="/meetups/">
+ Meetups
+ </a>
+
+ <a class="navbar__text-link" href="/docs/">
+ Documentation
+ </a>
+
+ <a class="navbar__text-link" href="/use-cases/">
+ Use-cases
+ </a>
+
+ <a class="navbar__text-link" href="/announcements/">
+ Announcements
+ </a>
+
+ <a class="navbar__text-link" href="/blog/">
+ Blog
+ </a>
+
+ <a class="navbar__text-link" href="/ecosystem/">
+ Ecosystem
+ </a>
+
+ </div>
+
+
+
+ </div>
+
+ </div>
+ <div class="mobile-only navbar__drawer-container">
+ <button class="navbar__toggle-button" id="navbar-toggle-button">
+
+ <div id="hamburger-icon" class="navbar__toggle-button--icon visible">
+ <svg xmlns="http://www.w3.org/2000/svg" width="26" height="20" viewBox="0 0 26 20">
+ <g id="Group_1294" data-name="Group 1294" transform="translate(-38.791 291)">
+ <g id="Group_1291" data-name="Group 1291" transform="translate(39 -291)">
+ <rect id="Rectangle_461" width="26" height="2" fill="#51504f" data-name="Rectangle 461" rx="1" transform="translate(-.209)"></rect>
+ </g>
+ <g id="Group_1292" data-name="Group 1292" transform="translate(39 -281.822)">
+ <rect id="Rectangle_462" width="26" height="2" fill="#51504f" data-name="Rectangle 462" rx="1" transform="translate(-.209 -.178)"></rect>
+ </g>
+ <g id="Group_1293" data-name="Group 1293" transform="translate(39 -272.644)">
+ <rect id="Rectangle_463" width="26" height="2" fill="#51504f" data-name="Rectangle 463" rx="1" transform="translate(-.209 -.356)"></rect>
+ </g>
+ </g>
+ </svg>
+
+ </div>
+
+ <div id="close-icon" class="navbar__toggle-button--icon">
+ <svg xmlns="http://www.w3.org/2000/svg" width="19.799" height="19.799" viewBox="0 0 19.799 19.799">
+ <g id="Group_1574" data-name="Group 1574" transform="translate(-41.892 290.899)">
+ <g id="Group_1291" data-name="Group 1291" transform="rotate(-45 -308.114 -187.077)">
+ <rect id="Rectangle_461" width="26" height="2" fill="#51504f" data-name="Rectangle 461" rx="1" transform="translate(-.209)"></rect>
+ </g>
+ <g id="Group_1292" data-name="Group 1292" transform="rotate(45 372.48 -93.011)">
+ <rect id="Rectangle_462" width="26" height="2" fill="#51504f" data-name="Rectangle 462" rx="1" transform="translate(-.209 -.178)"></rect>
+ </g>
+ </g>
+ </svg>
+
+ </div>
+
+ </button>
+ <div class="navbar__drawer" id="navbar-drawer">
+ <div class="navbar__menu-content" id="main_navbar">
+
+ <div class="navbar__links-container">
+
+ <a class="navbar__text-link" href="/community/">
+ Community
+ </a>
+
+ <a class="navbar__text-link" href="/meetups/">
+ Meetups
+ </a>
+
+ <a class="navbar__text-link" href="/docs/">
+ Documentation
+ </a>
+
+ <a class="navbar__text-link" href="/use-cases/">
+ Use-cases
+ </a>
+
+ <a class="navbar__text-link" href="/announcements/">
+ Announcements
+ </a>
+
+ <a class="navbar__text-link" href="/blog/">
+ Blog
+ </a>
+
+ <a class="navbar__text-link" href="/ecosystem/">
+ Ecosystem
+ </a>
+
+
+ </div>
+
+
+ </div>
+ </div>
+ </div>
+ </nav>
+
+</header>
+
+
+ <div class="roadmap container-fluid td-default base-layout">
+
+
+ <div class="content-drawer-wrapper">
+ <button class="content-drawer__toggle-button" id="content-open-button">
+
+ <div id="hamburger-icon" class="content-drawer__toggle-button--icon visible">
+ <svg xmlns="http://www.w3.org/2000/svg" width="20.005" height="13.879" viewBox="0 0 20.005 13.879">
+ <g id="Group_1619" data-name="Group 1619" transform="translate(271.132 -418.872)">
+ <g id="Group_1613" data-name="Group 1613" transform="translate(-266.229 431.045)">
+ <path id="Path_1337" d="M-218.5 513.215h-14.583a.259.259 0 0 1-.259-.26.259.259 0 0 1 .259-.259h14.583a.26.26 0 0 1 .259.259.259.259 0 0 1-.259.26z" fill="#51504f" data-name="Path 1337" transform="translate(233.345 -512.696)"></path>
+ </g>
+ <g id="Group_1614" data-name="Group 1614" transform="translate(-266.229 425.684)">
+ <path id="Path_1338" d="M-218.5 471.9h-14.583a.259.259 0 0 1-.259-.26.26.26 0 0 1 .259-.259h14.583a.26.26 0 0 1 .259.259.26.26 0 0 1-.259.26z" fill="#51504f" data-name="Path 1338" transform="translate(233.345 -471.376)"></path>
+ </g>
+ <g id="Group_1615" data-name="Group 1615" transform="translate(-266.229 420.323)">
+ <path id="Path_1339" d="M-218.5 430.574h-14.583a.259.259 0 0 1-.259-.259.259.259 0 0 1 .259-.259h14.583a.26.26 0 0 1 .259.259.259.259 0 0 1-.259.259z" fill="#51504f" data-name="Path 1339" transform="translate(233.345 -430.055)"></path>
+ </g>
+ <g id="Group_1616" data-name="Group 1616" transform="translate(-271.132 429.814)">
+ <path id="Path_1340" d="M-269.663 506.145a1.47 1.47 0 0 1-1.469-1.469 1.47 1.47 0 0 1 1.469-1.469 1.47 1.47 0 0 1 1.469 1.469 1.47 1.47 0 0 1-1.469 1.469zm0-2.418a.951.951 0 0 0-.95.95.951.951 0 0 0 .95.95.951.951 0 0 0 .95-.95.951.951 0 0 0-.95-.95z" fill="#51504f" data-name="Path 1340" transform="translate(271.132 -503.208)"></path>
+ </g>
+ <g id="Group_1617" data-name="Group 1617" transform="translate(-271.132 424.556)">
+ <path id="Path_1341" d="M-269.663 465.62a1.47 1.47 0 0 1-1.469-1.469 1.47 1.47 0 0 1 1.469-1.469 1.47 1.47 0 0 1 1.469 1.469 1.47 1.47 0 0 1-1.469 1.469zm0-2.418a.951.951 0 0 0-.95.95.951.951 0 0 0 .95.95.951.951 0 0 0 .95-.95.951.951 0 0 0-.95-.952z" fill="#51504f" data-name="Path 1341" transform="translate(271.132 -462.683)"></path>
+ </g>
+ <g id="Group_1618" data-name="Group 1618" transform="translate(-271.132 418.872)">
+ <path id="Path_1342" d="M-269.663 421.809a1.47 1.47 0 0 1-1.469-1.469 1.47 1.47 0 0 1 1.469-1.469 1.47 1.47 0 0 1 1.469 1.469 1.47 1.47 0 0 1-1.469 1.469zm0-2.418a.951.951 0 0 0-.95.95.951.951 0 0 0 .95.95.951.951 0 0 0 .95-.95.951.951 0 0 0-.95-.95z" fill="#51504f" data-name="Path 1342" transform="translate(271.132 -418.872)"></path>
+ </g>
+ </g>
+ </svg>
+
+ <span class="bodytext__mobile--brownish-grey">Content</span>
+ </div>
+
+ </button>
+ <nav id="content-navbar" class="navbar navbar--hidden">
+ <div class="navbar__icon-container">
+ <a href="/">
+ <svg xmlns="http://www.w3.org/2000/svg" width="155.314" height="60" viewBox="0 0 155.314 60">
+ <defs>
+ <clipPath id="clip-path">
+ <path id="Rectangle_1" d="M0 0h155.314v60H0z" fill="none" data-name="Rectangle 1"></path>
+ </clipPath>
+ </defs>
+ <g id="logo" transform="translate(-1305 -780.355)">
+ <g id="Group_2" clip-path="url(#clip-path)" data-name="Group 2" transform="translate(1305 780.355)">
+ <g id="Group_1" data-name="Group 1" transform="translate(.486 .486)">
+ <path id="Path_1" d="M1307.562 880.867l28.187-28.893a.521.521 0 0 0 .063-.666c-1.714-2.393-4.877-2.808-6.049-4.416-3.472-4.763-4.353-7.459-5.845-7.292a.456.456 0 0 0-.271.143l-10.182 10.438c-5.858 6-6.7 19.225-6.852 30.3a.552.552 0 0 0 .949.386z" fill="#017cee" data-name="Path 1" transform="translate(-1306.613 -822.232)"></path>
+ <path id="Path_2" d="M1405.512 908.489l-28.893-28.189a.521.521 0 0 0-.667-.063c-2.393 1.715-2.808 4.877-4.416 6.049-4.763 3.472-7.459 4.353-7.292 5.845a.456.456 0 0 0 .143.27l10.438 10.182c6 5.858 19.225 6.7 30.3 6.852a.552.552 0 0 0 .387-.946z" fill="#00ad46" data-name="Path 2" transform="translate(-1346.876 -850.567)"></path>
+ <path id="Path_3" d="M1373.909 902.252c-3.28-3.2-4.8-9.53 1.486-22.583-10.219 4.567-13.8 10.57-12.039 12.289z" fill="#04d659" data-name="Path 3" transform="translate(-1345.96 -850.233)"></path>
+ <path id="Path_4" d="M1433.132 782.359l-28.186 28.893a.52.52 0 0 0-.063.666c1.715 2.393 4.876 2.808 6.049 4.416 3.472 4.763 4.354 7.459 5.845 7.292a.454.454 0 0 0 .271-.143l10.182-10.438c5.858-6 6.7-19.225 6.852-30.3a.553.553 0 0 0-.95-.386z" fill="#00c7d4" data-name="Path 4" transform="translate(-1375.21 -782.123)"></path>
+ <path id="Path_5" d="M1426.9 881.155c-3.2 3.28-9.53 4.8-22.584-1.486 4.567 10.219 10.57 13.8 12.289 12.039z" fill="#11e1ee" data-name="Path 5" transform="translate(-1374.875 -850.233)"></path>
+ <path id="Path_6" d="M1307 782.919l28.893 28.186a.521.521 0 0 0 .666.063c2.393-1.715 2.808-4.877 4.416-6.049 4.763-3.472 7.459-4.353 7.292-5.845a.459.459 0 0 0-.143-.271l-10.438-10.182c-6-5.858-19.225-6.7-30.3-6.852a.552.552 0 0 0-.386.95z" fill="#e43921" data-name="Path 6" transform="translate(-1306.766 -781.97)"></path>
+ <path id="Path_7" d="M1405.8 804.711c3.28 3.2 4.8 9.53-1.486 22.584 10.219-4.567 13.8-10.571 12.039-12.289z" fill-rule="evenodd" fill="#ff7557" data-name="Path 7" transform="translate(-1374.875 -797.859)"></path>
+ <path id="Path_8" d="M1329.355 849.266c3.2-3.28 9.53-4.8 22.584 1.486-4.567-10.219-10.57-13.8-12.289-12.039z" fill="#0cb6ff" data-name="Path 8" transform="translate(-1322.503 -821.316)"></path>
+ <circle id="Ellipse_1" cx="1.26" cy="1.26" r="1.26" fill="#4a4848" data-name="Ellipse 1" transform="translate(28.18 28.171)"></circle>
+ <path id="Path_9" d="M1527.558 827.347a.229.229 0 0 1-.223-.223.458.458 0 0 1 .011-.123l2.766-7.214a.346.346 0 0 1 .357-.245h.758a.348.348 0 0 1 .357.245l2.754 7.214.022.123a.228.228 0 0 1-.223.223h-.568a.288.288 0 0 1-.19-.056.352.352 0 0 1-.089-.134l-.613-1.583h-3.657l-.613 1.583a.317.317 0 0 1-.1.134.269.269 0 0 1-.178.056zm4.795-2.732l-1.505-3.958-1.505 3.958zm3.322 4.85a.258.258 0 0 1-.189-.078.241.241 0 0 1-.067-.178v-7.4a.241.241 0 0 1 .067-.178 [...]
+ <path id="Path_10" d="M1527.2 827.081l-.061.061zm-.056-.279l-.08-.031zm2.766-7.214l.08.031zm1.472 0l-.081.029zm2.754 7.214l.084-.015a.064.064 0 0 0 0-.015zm.022.123h.086v-.015zm-.067.156l.06.061zm-.914.011l-.061.061.006.005zm-.089-.134l.081-.027zm-.613-1.583l.08-.031a.086.086 0 0 0-.08-.055zm-3.657 0v-.086a.086.086 0 0 0-.08.055zm-.613 1.583l-.08-.031zm-.1.134l.055.066zm4.047-2.676v.086a.086.086 0 0 0 .08-.116zm-1.505-3.958l.08-.03a.086.086 0 0 0-.16 0 [...]
+ <path id="Path_11" d="M1519.066 884.011a.581.581 0 0 1-.567-.567 1.151 1.151 0 0 1 .028-.312l7.026-18.328a.881.881 0 0 1 .906-.623h1.926a.882.882 0 0 1 .907.623l7 18.328.057.312a.583.583 0 0 1-.567.567h-1.445a.735.735 0 0 1-.482-.142.9.9 0 0 1-.226-.34l-1.558-4.023h-9.292l-1.558 4.023a.8.8 0 0 1-.255.34.688.688 0 0 1-.453.142zm12.181-6.94l-3.824-10.056-3.823 10.055zm8.184-10.538a.592.592 0 0 1-.652-.651v-1.53a.714.714 0 0 1 .17-.482.656.656 0 0 1 .482- [...]
+ </g>
+ </g>
+ </g>
+ </svg>
+
+ </a>
+ </div>
+
+ <div id="content-close-button">
+ <svg xmlns="http://www.w3.org/2000/svg" width="19.799" height="19.799" viewBox="0 0 19.799 19.799">
+ <g id="Group_1574" data-name="Group 1574" transform="translate(-41.892 290.899)">
+ <g id="Group_1291" data-name="Group 1291" transform="rotate(-45 -308.114 -187.077)">
+ <rect id="Rectangle_461" width="26" height="2" fill="#51504f" data-name="Rectangle 461" rx="1" transform="translate(-.209)"></rect>
+ </g>
+ <g id="Group_1292" data-name="Group 1292" transform="rotate(45 372.48 -93.011)">
+ <rect id="Rectangle_462" width="26" height="2" fill="#51504f" data-name="Rectangle 462" rx="1" transform="translate(-.209 -.178)"></rect>
+ </g>
+ </g>
+ </svg>
+
+ </div>
+
+ </nav>
+ <div class="content-drawer-container" id="content-drawer">
+ <div class="content-drawer">
+ <div class="td-sidebar">
+
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
+<div class="searchb-box">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
+ <input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+
+ <button class="search-form__button" type="submit">
+
+ <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 20 20">
+ <g id="Group_1579" data-name="Group 1579" transform="translate(-41.001 -41)">
+ <path id="Path_169" d="M71.415 64.687a7.215 7.215 0 1 0-6.729 6.728 7.222 7.222 0 0 0 6.729-6.728z" fill="none" data-name="Path 169" transform="translate(-14.277 -14.276)"></path>
+ <path id="Path_170" d="M60.863 59.8l-6.093-6.09a7.78 7.78 0 1 0-1.06 1.06l6.09 6.093a.468.468 0 0 0 .662 0l.4-.4a.468.468 0 0 0 .001-.663zM42.512 49.183a6.274 6.274 0 1 1 5.851 5.85 6.28 6.28 0 0 1-5.851-5.85z" fill="#51504f" data-name="Path 170"></path>
+ </g>
+ </svg>
+ </button>
+ </form>
+</div>
+
+
+<style>
+ .searchb-box {
+ margin-bottom: 26px;
+ }
+ .searchb-box .search-form {
+ width: 100%;
+ margin-top: 20px;
+ }
+</style>
+
+
+<div class="toctree" role="navigation" aria-label="main navigation">
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
+</ul>
+
+</div>
+
+<style type="text/css">
+ .toctree {
+ }
+ .toctree li {
+ list-style: none;
+ }
+ .toctree .caption {
+ font-family: Roboto;
+ font-size: 18px;
+ font-weight: bold;
+ font-stretch: normal;
+ font-style: normal;
+ line-height: 1.33;
+ letter-spacing: normal;
+ color: #51504f;
+ padding-bottom: 13px;
+ text-transform: uppercase;
+ margin-bottom: 0;
+ }
+ .toctree .current > a:not([href="#"]) {
+ color: #017cee;
+ }
+ .toctree > ul {
+ padding-left: 0;
+ }
+ .toctree ul {
+ padding-left: 15px;
+ display: none;
+ }
+ .toctree > ul,
+ .toctree li.current > ul {
+ display: block;
+ }
+
+ .toctree a .toctree-expand {
+ display: inline-block;
+ position: relative;
+ height: 1em;
+ }
+
+ .toctree a .toctree-expand:before {
+ position: absolute;
+ top: 6px;
+ left: -12px;
+ content: '►';
+ font-size: 7px;
+ }
+ .toctree .current > a > .toctree-expand:before {
+ content: '▼';
+ }
+
+ .toctree .current {
+ color: #017cee;
+ }
+ .toctree li {
+ font-family: Roboto;
+ font-size: 16px;
+ font-weight: normal;
+ font-stretch: normal;
+ font-style: normal;
+ line-height: 1.63;
+ letter-spacing: normal;
+ color: #707070;
+ }
+
+</style>
+ </div>
+ </div>
+ </div>
+ </div>
+
+ <div class="d-flex">
+
+
+ <div class="td-sidebar desktop-only d-print-none">
+
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
+<div class="searchb-box">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
+ <input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+
+ <button class="search-form__button" type="submit">
+
+ <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 20 20">
+ <g id="Group_1579" data-name="Group 1579" transform="translate(-41.001 -41)">
+ <path id="Path_169" d="M71.415 64.687a7.215 7.215 0 1 0-6.729 6.728 7.222 7.222 0 0 0 6.729-6.728z" fill="none" data-name="Path 169" transform="translate(-14.277 -14.276)"></path>
+ <path id="Path_170" d="M60.863 59.8l-6.093-6.09a7.78 7.78 0 1 0-1.06 1.06l6.09 6.093a.468.468 0 0 0 .662 0l.4-.4a.468.468 0 0 0 .001-.663zM42.512 49.183a6.274 6.274 0 1 1 5.851 5.85 6.28 6.28 0 0 1-5.851-5.85z" fill="#51504f" data-name="Path 170"></path>
+ </g>
+ </svg>
+ </button>
+ </form>
+</div>
+
+
+<style>
+ .searchb-box {
+ margin-bottom: 26px;
+ }
+ .searchb-box .search-form {
+ width: 100%;
+ margin-top: 20px;
+ }
+</style>
+
+
+<div class="toctree" role="navigation" aria-label="main navigation">
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
+</ul>
+
+</div>
+
+<style type="text/css">
+ .toctree {
+ }
+ .toctree li {
+ list-style: none;
+ }
+ .toctree .caption {
+ font-family: Roboto;
+ font-size: 18px;
+ font-weight: bold;
+ font-stretch: normal;
+ font-style: normal;
+ line-height: 1.33;
+ letter-spacing: normal;
+ color: #51504f;
+ padding-bottom: 13px;
+ text-transform: uppercase;
+ margin-bottom: 0;
+ }
+ .toctree .current > a:not([href="#"]) {
+ color: #017cee;
+ }
+ .toctree > ul {
+ padding-left: 0;
+ }
+ .toctree ul {
+ padding-left: 15px;
+ display: none;
+ }
+ .toctree > ul,
+ .toctree li.current > ul {
+ display: block;
+ }
+
+ .toctree a .toctree-expand {
+ display: inline-block;
+ position: relative;
+ height: 1em;
+ }
+
+ .toctree a .toctree-expand:before {
+ position: absolute;
+ top: 6px;
+ left: -12px;
+ content: '►';
+ font-size: 7px;
+ }
+ .toctree .current > a > .toctree-expand:before {
+ content: '▼';
+ }
+
+ .toctree .current {
+ color: #017cee;
+ }
+ .toctree li {
+ font-family: Roboto;
+ font-size: 16px;
+ font-weight: normal;
+ font-stretch: normal;
+ font-style: normal;
+ line-height: 1.63;
+ letter-spacing: normal;
+ color: #707070;
+ }
+
+</style>
+ </div>
+
+
+
+
+ <main class="col-12 col-md-9 col-xl-8" role="main">
+
+
+
+
+
+
+
+
+<div role="navigation" aria-label="breadcrumbs navigation" class="d-none d-md-block d-print-none">
+
+ <ul class="breadcrumb">
+
+ <li class="breadcrumb-item"><a href="../../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws</span></code></a></li>
+
+ </ul>
+</div>
+
+ <div class="rst-content">
+ <div class="document">
+ <div class="documentwrapper">
+ <div class="bodywrapper">
+ <div class="body" role="main">
+
+ <div class="section" id="module-airflow.providers.amazon.aws.hooks.base_aws">
+<span id="airflow-providers-amazon-aws-hooks-base-aws"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.hooks.base_aws" title="airflow.providers.amazon.aws.hooks.base_aws"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.hooks.base_aws" title="Permalink to this heading">¶</a></h1>
+<p>This module contains Base AWS Hook.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<p>For more information on how to use this hook, take a look at the guide:
+<a class="reference internal" href="../../../../../../../connections/aws.html#howto-connection-aws"><span class="std std-ref">Amazon Web Services Connection</span></a></p>
+</div>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<div class="section" id="classes">
+<h3>Classes<a class="headerlink" href="#classes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory" title="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory"><code class="xref py py-obj docutils literal notranslate"><span class="pre">BaseSessionFactory</span></code></a></p></td>
+<td><p>Base AWS Session Factory class to handle synchronous and async boto session creation.</p></td>
+</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">AwsGenericHook</span></code></a></p></td>
+<td><p>Generic class for interact with AWS.</p></td>
+</tr>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">AwsBaseHook</span></code></a></p></td>
+<td><p>Base class for interact with AWS.</p></td>
+</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseAsyncSessionFactory" title="airflow.providers.amazon.aws.hooks.base_aws.BaseAsyncSessionFactory"><code class="xref py py-obj docutils literal notranslate"><span class="pre">BaseAsyncSessionFactory</span></code></a></p></td>
+<td><p>Base AWS Session Factory class to handle aiobotocore session creation.</p></td>
+</tr>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseAsyncHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseAsyncHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">AwsBaseAsyncHook</span></code></a></p></td>
+<td><p>Interacts with AWS using aiobotocore asynchronously.</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+<div class="section" id="functions">
+<h3>Functions<a class="headerlink" href="#functions" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.resolve_session_factory" title="airflow.providers.amazon.aws.hooks.base_aws.resolve_session_factory"><code class="xref py py-obj docutils literal notranslate"><span class="pre">resolve_session_factory</span></code></a>()</p></td>
+<td><p>Resolves custom SessionFactory class</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+<div class="section" id="attributes">
+<h3>Attributes<a class="headerlink" href="#attributes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseAwsConnection" title="airflow.providers.amazon.aws.hooks.base_aws.BaseAwsConnection"><code class="xref py py-obj docutils literal notranslate"><span class="pre">BaseAwsConnection</span></code></a></p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.SessionFactory" title="airflow.providers.amazon.aws.hooks.base_aws.SessionFactory"><code class="xref py py-obj docutils literal notranslate"><span class="pre">SessionFactory</span></code></a></p></td>
+<td><p></p></td>
+</tr>
+</tbody>
+</table>
+<dl class="py data">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseAwsConnection">
+<span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.</span></span><span class="sig-name descname"><span class="pre">BaseAwsConnection</span></span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#BaseAwsConnection"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseAwsConnectio [...]
+<dd></dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.</span></span><span class="sig-name descname"><span class="pre">BaseSessionFactory</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">conn</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">region_name</span></span><span class="o"><spa [...]
+<dd><p>Bases: <code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.utils.log.logging_mixin.LoggingMixin</span></code></p>
+<p>Base AWS Session Factory class to handle synchronous and async boto session creation.
+It can handle most of the AWS supported authentication methods.</p>
+<p>User can also derive from this class to have full control of boto3 session
+creation or to support custom federation.</p>
+<p>Note: Not all features implemented for synchronous sessions are available for async
+sessions.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference internal" href="../../../../../../../connections/aws.html#howto-connection-aws-session-factory"><span class="std std-ref">Session Factory</span></a></p></li>
+</ul>
+</div>
+<dl class="field-list simple">
+</dl>
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.extra_config">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">extra_config</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><span class="pre">dict</span></a><span class="p"><span class="pre">[</span></span><a class="reference external" href="https [...]
+<dd><p>AWS Connection extra_config.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.region_name">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">region_name</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><span class="pre">str</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </s [...]
+<dd><p>AWS Region Name read-only property.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.config">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">config</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">Config</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/constants.html#None" title="(in Python v3.11)"> [...]
+<dd><p>Configuration for botocore client read-only property.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.role_arn">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">role_arn</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><span class="pre">str</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span [...]
+<dd><p>Assume Role ARN from AWS Connection</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.conn">
+<span class="sig-name descname"><span class="pre">conn</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#BaseSessionFactory.conn"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.conn" title="Permalink to this definition">¶</a></dt>
+<dd><p>Cached AWS Connection Wrapper.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.basic_session">
+<span class="sig-name descname"><span class="pre">basic_session</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#BaseSessionFactory.basic_session"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.basic_session" title="Permalink to thi [...]
+<dd><p>Cached property with basic boto3.session.Session.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.get_async_session">
+<span class="sig-name descname"><span class="pre">get_async_session</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#BaseSessionFactory.get_async_session"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.get_async_session" title="Perm [...]
+<dd></dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.create_session">
+<span class="sig-name descname"><span class="pre">create_session</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">deferrable</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#BaseSessionFactory.create_session">< [...]
+<dd><p>Create boto3 or aiobotocore Session from connection config.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.</span></span><span class="sig-name descname"><span class="pre">AwsGenericHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">aws_conn_id</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">default_co [...]
+<dd><p>Bases: <a class="reference external" href="/docs/apache-airflow/stable/_api/airflow/hooks/base/index.html#airflow.hooks.base.BaseHook" title="(in apache-airflow v2.6.0.dev0)"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.hooks.base.BaseHook</span></code></a>, <code class="xref py py-obj docutils literal notranslate"><span class="pre">Generic</span></code>[<a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseAwsC [...]
+<p>Generic class for interact with AWS.
+This class provide a thin wrapper around the boto3 python library.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>aws_conn_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – The Airflow connection used for AWS credentials.
+If this is None or empty then the default boto3 behaviour is used. If
+running Airflow in a distributed manner and aws_conn_id is None or
+empty, then default boto3 configuration would be used (and must be
+maintained on each worker node).</p></li>
+<li><p><strong>verify</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – Whether or not to verify SSL certificates. See:
+<a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html">https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html</a></p></li>
+<li><p><strong>region_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – AWS region_name. If not specified then the default boto3 behaviour is used.</p></li>
+<li><p><strong>client_type</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – Reference to <a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.client" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">boto3.client</span> <span [...]
+Mutually exclusive with <code class="docutils literal notranslate"><span class="pre">resource_type</span></code>.</p></li>
+<li><p><strong>resource_type</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – Reference to <a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.resource" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">boto3.resource</span> [...]
+Mutually exclusive with <code class="docutils literal notranslate"><span class="pre">client_type</span></code>.</p></li>
+<li><p><strong>config</strong> (<em>Config</em><em> | </em><em>None</em>) – Configuration for botocore client. See:
+<a class="reference external" href="https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html">https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html</a></p></li>
+</ul>
+</dd>
+</dl>
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.service_config">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">service_config</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><span class="pre">dict</span></a></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers [...]
+<dd><dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.region_name">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">region_name</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><span class="pre">str</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </s [...]
+<dd><p>AWS Region Name read-only property.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.config">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">config</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">botocore.config.Config</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.config"><span class="viewcode-link"><span class="pre">[source]< [...]
+<dd><p>Configuration for botocore client read-only property.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.verify">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">verify</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><span class="pre">bool</span></a><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </spa [...]
+<dd><p>Verify or not SSL certificates boto3 client/resource read-only property.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.async_conn">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">async_conn</span></span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.async_conn"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.async_conn" title="Permalink to t [...]
+<dd><p>Get an aiobotocore client to use for async operations.</p>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_region_name">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">conn_region_name</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><span class="pre">str</span></a></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers [...]
+<dd><p>Get actual AWS Region Name from Hook connection (cached).</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_partition">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">conn_partition</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><span class="pre">str</span></a></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/a [...]
+<dd><p>Get associated AWS Region Partition from Hook connection (cached).</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_name_attr">
+<span class="sig-name descname"><span class="pre">conn_name_attr</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'aws_conn_id'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.conn_name_attr"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflo [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.default_conn_name">
+<span class="sig-name descname"><span class="pre">default_conn_name</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'aws_default'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.default_conn_name"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="# [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_type">
+<span class="sig-name descname"><span class="pre">conn_type</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'aws'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.conn_type"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.hook_name">
+<span class="sig-name descname"><span class="pre">hook_name</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'Amazon</span> <span class="pre">Web</span> <span class="pre">Services'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.hook_name"><span class="viewcode-link"><span class="pre">[source]</spa [...]
+<dd></dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_config">
+<span class="sig-name descname"><span class="pre">conn_config</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.conn_config"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_config" title="Permalink to this definition"> [...]
+<dd><p>Get the Airflow Connection object and wrap it in helper (cached).</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_session">
+<span class="sig-name descname"><span class="pre">get_session</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">region_name</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">deferrable</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</sp [...]
+<dd><p>Get the underlying boto3.session.Session(region_name=region_name).</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_client_type">
+<span class="sig-name descname"><span class="pre">get_client_type</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">region_name</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">config</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</spa [...]
+<dd><p>Get the underlying boto3 client using boto3 session</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_resource_type">
+<span class="sig-name descname"><span class="pre">get_resource_type</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">region_name</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">config</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</s [...]
+<dd><p>Get the underlying boto3 resource using boto3 session</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn">
+<span class="sig-name descname"><span class="pre">conn</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.conn"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn" title="Permalink to this definition">¶</a></dt>
+<dd><p>Get the underlying boto3 client/resource (cached)</p>
+<dl class="field-list simple">
+<dt class="field-odd">Returns</dt>
+<dd class="field-odd"><p>boto3.client or boto3.resource</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p>BaseAwsConnection</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_client_meta">
+<span class="sig-name descname"><span class="pre">conn_client_meta</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.conn_client_meta"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_client_meta" title="Permalink to th [...]
+<dd><p>Get botocore client metadata from Hook connection (cached).</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_conn">
+<span class="sig-name descname"><span class="pre">get_conn</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.get_conn"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_conn" title="Permalink to this definition">¶</a></dt>
+<dd><p>Get the underlying boto3 client/resource (cached)</p>
+<p>Implemented so that caching works as intended. It exists for compatibility
+with subclasses that rely on a super().get_conn() method.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Returns</dt>
+<dd class="field-odd"><p>boto3.client or boto3.resource</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p>BaseAwsConnection</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_credentials">
+<span class="sig-name descname"><span class="pre">get_credentials</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">region_name</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.get_credentials"><sp [...]
+<dd><p>Get the underlying <cite>botocore.Credentials</cite> object.</p>
+<p>This contains the following authentication attributes: access_key, secret_key and token.
+By use this method also secret_key and token will mask in tasks logs.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.expand_role">
+<span class="sig-name descname"><span class="pre">expand_role</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">role</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">region_name</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow [...]
+<dd><p>If the IAM role is a role name, get the Amazon Resource Name (ARN) for the role.
+If IAM role is already an IAM role ARN, no change is made.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>role</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – IAM role name or ARN</p></li>
+<li><p><strong>region_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – Optional region name to get credentials for</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>IAM role ARN</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.retry">
+<em class="property"><span class="pre">static</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">retry</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">should_retry</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.retry"><span class="viewcode-link"><span class="pre">[source] [...]
+<dd><p>A decorator that provides a mechanism to repeat requests in response to exceeding a temporary quote
+limit.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_ui_field_behaviour">
+<em class="property"><span class="pre">static</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">get_ui_field_behaviour</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.get_ui_field_behaviour"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflo [...]
+<dd><p>Returns custom UI field behaviour for AWS Connection.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.test_connection">
+<span class="sig-name descname"><span class="pre">test_connection</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.test_connection"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.test_connection" title="Permalink to this [...]
+<dd><p>Tests the AWS connection by call AWS STS (Security Token Service) GetCallerIdentity API.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<p><a class="reference external" href="https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html">https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html</a></p>
+</div>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.waiter_path">
+<span class="sig-name descname"><span class="pre">waiter_path</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.waiter_path"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.waiter_path" title="Permalink to this definition"> [...]
+<dd><dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_waiter">
+<span class="sig-name descname"><span class="pre">get_waiter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">waiter_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">parameters</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">deferrable</span></span><span class="o"><s [...]
+<dd><p>First checks if there is a custom waiter with the provided waiter_name and
+uses that if it exists, otherwise it will check the service client for a
+waiter that matches the name and pass that through.</p>
+<p>If <cite>deferrable</cite> is True, the waiter will be an AIOWaiter, generated from the
+client that is passed as a parameter. If <cite>deferrable</cite> is True, <cite>client</cite> must be
+provided.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>waiter_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – The name of the waiter. The name should exactly match the
+name of the key in the waiter model file (typically this is CamelCase).</p></li>
+<li><p><strong>parameters</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a><em>[</em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em>, </em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em>] </em><em>| </em><em>None</em>) – w [...]
+corresponding value. If a custom waiter has such keys to be expanded, they need to be provided
+here.</p></li>
+<li><p><strong>deferrable</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a>) – If True, the waiter is going to be an async custom waiter.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.list_waiters">
+<span class="sig-name descname"><span class="pre">list_waiters</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsGenericHook.list_waiters"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.list_waiters" title="Permalink to this definitio [...]
+<dd><p>Returns a list containing the names of all waiters for the service, official and custom.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.</span></span><span class="sig-name descname"><span class="pre">AwsBaseHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">aws_conn_id</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">default_conn_ [...]
+<dd><p>Bases: <a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">AwsGenericHook</span></code></a>[<code class="xref py py-obj docutils literal notranslate"><span class="pre">Union</span></code>[<a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/bo [...]
+<p>Base class for interact with AWS.
+This class provide a thin wrapper around the boto3 python library.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>aws_conn_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – The Airflow connection used for AWS credentials.
+If this is None or empty then the default boto3 behaviour is used. If
+running Airflow in a distributed manner and aws_conn_id is None or
+empty, then default boto3 configuration would be used (and must be
+maintained on each worker node).</p></li>
+<li><p><strong>verify</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – Whether or not to verify SSL certificates. See:
+<a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html">https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html</a></p></li>
+<li><p><strong>region_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – AWS region_name. If not specified then the default boto3 behaviour is used.</p></li>
+<li><p><strong>client_type</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – Reference to <a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.client" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">boto3.client</span> <span [...]
+Mutually exclusive with <code class="docutils literal notranslate"><span class="pre">resource_type</span></code>.</p></li>
+<li><p><strong>resource_type</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – Reference to <a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.resource" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">boto3.resource</span> [...]
+Mutually exclusive with <code class="docutils literal notranslate"><span class="pre">client_type</span></code>.</p></li>
+<li><p><strong>config</strong> (<em>Config</em><em> | </em><em>None</em>) – Configuration for botocore client. See:
+<a class="reference external" href="https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html">https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html</a></p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py function">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.resolve_session_factory">
+<span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.</span></span><span class="sig-name descname"><span class="pre">resolve_session_factory</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#resolve_session_factory"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="hea [...]
+<dd><p>Resolves custom SessionFactory class</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py data">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.SessionFactory">
+<span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.</span></span><span class="sig-name descname"><span class="pre">SessionFactory</span></span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#SessionFactory"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.SessionFactory" title= [...]
+<dd></dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseAsyncSessionFactory">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.</span></span><span class="sig-name descname"><span class="pre">BaseAsyncSessionFactory</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">conn</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">region_name</span></span><span class="o" [...]
+<dd><p>Bases: <a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory" title="airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory"><code class="xref py py-obj docutils literal notranslate"><span class="pre">BaseSessionFactory</span></code></a></p>
+<p>Base AWS Session Factory class to handle aiobotocore session creation.</p>
+<p>It currently, handles ENV, AWS secret key and STS client method <code class="docutils literal notranslate"><span class="pre">assume_role</span></code>
+provided in Airflow connection</p>
+<dl class="field-list simple">
+</dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseAsyncSessionFactory.get_role_credentials">
+<em class="property"><span class="pre">async</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">get_role_credentials</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#BaseAsyncSessionFactory.get_role_credentials"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#ai [...]
+<dd><p>Get the role_arn, method credentials from connection details and get the role credentials detail</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.BaseAsyncSessionFactory.create_session">
+<span class="sig-name descname"><span class="pre">create_session</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">deferrable</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#BaseAsyncSessionFactory.create_sessi [...]
+<dd><p>Create aiobotocore Session from connection and config.</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseAsyncHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.</span></span><span class="sig-name descname"><span class="pre">AwsBaseAsyncHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">aws_conn_id</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">default_ [...]
+<dd><p>Bases: <a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">AwsBaseHook</span></code></a></p>
+<p>Interacts with AWS using aiobotocore asynchronously.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>aws_conn_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – The Airflow connection used for AWS credentials.
+If this is None or empty then the default botocore behaviour is used. If
+running Airflow in a distributed manner and aws_conn_id is None or
+empty, then default botocore configuration would be used (and must be
+maintained on each worker node).</p></li>
+<li><p><strong>verify</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – Whether to verify SSL certificates.</p></li>
+<li><p><strong>region_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – AWS region_name. If not specified then the default boto3 behaviour is used.</p></li>
+<li><p><strong>client_type</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – boto3.client client_type. Eg ‘s3’, ‘emr’ etc</p></li>
+<li><p><strong>resource_type</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em> | </em><em>None</em>) – boto3.resource resource_type. Eg ‘dynamodb’ etc</p></li>
+<li><p><strong>config</strong> (<em>Config</em><em> | </em><em>None</em>) – Configuration for botocore client.</p></li>
+</ul>
+</dd>
+</dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseAsyncHook.get_async_session">
+<span class="sig-name descname"><span class="pre">get_async_session</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsBaseAsyncHook.get_async_session"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseAsyncHook.get_async_session" title="Permalin [...]
+<dd><p>Get the underlying aiobotocore.session.AioSession(…).</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseAsyncHook.get_client_async">
+<em class="property"><span class="pre">async</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">get_client_async</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/base_aws.html#AwsBaseAsyncHook.get_client_async"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers [...]
+<dd><p>Get the underlying aiobotocore client using aiobotocore session</p>
+</dd></dl>
+
+</dd></dl>
+
+</div>
+</div>
+</div>
+
+
+ <div class="pager" role="navigation" aria-label="related navigation">
+ <a rel="prev" title="airflow.providers.amazon.aws.hooks.athena" href="../athena/index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="p">Previous</button>
+ </a>
+ <a rel="next" title="airflow.providers.amazon.aws.hooks.batch_client" href="../batch_client/index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="n">Next</button>
+ </a>
+ </div>
+ </div>
+
+ </div>
+ </div>
+ </div>
+ </div>
+ <div class="rating-container">
+ <p class="bodytext__medium--greyish-brown font-weight-500">Was this entry helpful?</p>
+ <div class="rating">
+
+ <div id="rate-star-5" class="rate-star">
+ <svg xmlns="http://www.w3.org/2000/svg" width="21.05" height="20.02" viewBox="0 0 21.05 20.02">
+ <g id="Group_806" data-name="Group 806" transform="translate(-774.404 -13.178)">
+ <path id="Path_715" d="M791.434 33.2l-6.5-3.42-6.5 3.42 1.242-7.243-5.262-5.13 7.273-1.057 3.252-6.59 3.252 6.59 7.273 1.057-5.262 5.13z" fill="#017cee" data-name="Path 715"></path>
+ </g>
+ </svg>
+
+ </div>
+
+ <div id="rate-star-4" class="rate-star">
+ <svg xmlns="http://www.w3.org/2000/svg" width="21.05" height="20.02" viewBox="0 0 21.05 20.02">
+ <g id="Group_806" data-name="Group 806" transform="translate(-774.404 -13.178)">
+ <path id="Path_715" d="M791.434 33.2l-6.5-3.42-6.5 3.42 1.242-7.243-5.262-5.13 7.273-1.057 3.252-6.59 3.252 6.59 7.273 1.057-5.262 5.13z" fill="#017cee" data-name="Path 715"></path>
+ </g>
+ </svg>
+
+ </div>
+
+ <div id="rate-star-3" class="rate-star">
+ <svg xmlns="http://www.w3.org/2000/svg" width="21.05" height="20.02" viewBox="0 0 21.05 20.02">
+ <g id="Group_806" data-name="Group 806" transform="translate(-774.404 -13.178)">
+ <path id="Path_715" d="M791.434 33.2l-6.5-3.42-6.5 3.42 1.242-7.243-5.262-5.13 7.273-1.057 3.252-6.59 3.252 6.59 7.273 1.057-5.262 5.13z" fill="#017cee" data-name="Path 715"></path>
+ </g>
+ </svg>
+
+ </div>
+
+ <div id="rate-star-2" class="rate-star">
+ <svg xmlns="http://www.w3.org/2000/svg" width="21.05" height="20.02" viewBox="0 0 21.05 20.02">
+ <g id="Group_806" data-name="Group 806" transform="translate(-774.404 -13.178)">
+ <path id="Path_715" d="M791.434 33.2l-6.5-3.42-6.5 3.42 1.242-7.243-5.262-5.13 7.273-1.057 3.252-6.59 3.252 6.59 7.273 1.057-5.262 5.13z" fill="#017cee" data-name="Path 715"></path>
+ </g>
+ </svg>
+
+ </div>
+
+ <div id="rate-star-1" class="rate-star">
+ <svg xmlns="http://www.w3.org/2000/svg" width="21.05" height="20.02" viewBox="0 0 21.05 20.02">
+ <g id="Group_806" data-name="Group 806" transform="translate(-774.404 -13.178)">
+ <path id="Path_715" d="M791.434 33.2l-6.5-3.42-6.5 3.42 1.242-7.243-5.262-5.13 7.273-1.057 3.252-6.59 3.252 6.59 7.273 1.057-5.262 5.13z" fill="#017cee" data-name="Path 715"></path>
+ </g>
+ </svg>
+
+ </div>
+
+ </div>
+ </div>
+
+ </main>
+
+
+
+
+ <nav class="wy-nav-side-toc">
+ <div class="wy-menu-vertical">
+ <ul>
+<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws</span></code></a><ul>
+<li><a class="reference internal" href="#module-contents">Module Contents</a><ul>
+<li><a class="reference internal" href="#classes">Classes</a></li>
+<li><a class="reference internal" href="#functions">Functions</a></li>
+<li><a class="reference internal" href="#attributes">Attributes</a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseAwsConnection"><code class="docutils literal notranslate"><span class="pre">BaseAwsConnection</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory"><code class="docutils literal notranslate"><span class="pre">BaseSessionFactory</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.extra_config"><code class="docutils literal notranslate"><span class="pre">BaseSessionFactory.extra_config</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.region_name"><code class="docutils literal notranslate"><span class="pre">BaseSessionFactory.region_name</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.config"><code class="docutils literal notranslate"><span class="pre">BaseSessionFactory.config</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.role_arn"><code class="docutils literal notranslate"><span class="pre">BaseSessionFactory.role_arn</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.conn"><code class="docutils literal notranslate"><span class="pre">BaseSessionFactory.conn()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.basic_session"><code class="docutils literal notranslate"><span class="pre">BaseSessionFactory.basic_session()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.get_async_session"><code class="docutils literal notranslate"><span class="pre">BaseSessionFactory.get_async_session()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory.create_session"><code class="docutils literal notranslate"><span class="pre">BaseSessionFactory.create_session()</span></code></a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.service_config"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.service_config</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.region_name"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.region_name</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.config"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.config</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.verify"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.verify</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.async_conn"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.async_conn</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_region_name"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.conn_region_name</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_partition"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.conn_partition</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_name_attr"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.conn_name_attr</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.default_conn_name"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.default_conn_name</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_type"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.conn_type</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.hook_name"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.hook_name</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_config"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.conn_config()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_session"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.get_session()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_client_type"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.get_client_type()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_resource_type"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.get_resource_type()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.conn()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.conn_client_meta"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.conn_client_meta()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_conn"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.get_conn()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_credentials"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.get_credentials()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.expand_role"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.expand_role()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.retry"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.retry()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_ui_field_behaviour"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.get_ui_field_behaviour()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.test_connection"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.test_connection()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.waiter_path"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.waiter_path()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.get_waiter"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.get_waiter()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook.list_waiters"><code class="docutils literal notranslate"><span class="pre">AwsGenericHook.list_waiters()</span></code></a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="docutils literal notranslate"><span class="pre">AwsBaseHook</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.resolve_session_factory"><code class="docutils literal notranslate"><span class="pre">resolve_session_factory()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.SessionFactory"><code class="docutils literal notranslate"><span class="pre">SessionFactory</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseAsyncSessionFactory"><code class="docutils literal notranslate"><span class="pre">BaseAsyncSessionFactory</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseAsyncSessionFactory.get_role_credentials"><code class="docutils literal notranslate"><span class="pre">BaseAsyncSessionFactory.get_role_credentials()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.BaseAsyncSessionFactory.create_session"><code class="docutils literal notranslate"><span class="pre">BaseAsyncSessionFactory.create_session()</span></code></a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseAsyncHook"><code class="docutils literal notranslate"><span class="pre">AwsBaseAsyncHook</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseAsyncHook.get_async_session"><code class="docutils literal notranslate"><span class="pre">AwsBaseAsyncHook.get_async_session()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseAsyncHook.get_client_async"><code class="docutils literal notranslate"><span class="pre">AwsBaseAsyncHook.get_client_async()</span></code></a></li>
+</ul>
+</li>
+</ul>
+</li>
+</ul>
+</li>
+</ul>
+</li>
+</ul>
+
+ </div>
+ </nav>
+
+ </div>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ <div class="base-layout--button">
+ <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-amazon/_api/airflow/providers/amazon/aws/hooks/base_aws/index.rst" rel="nofollow">
+
+ <button class="btn-hollow btn-brown btn-with-icon with-box-shadow button-fixed">
+ <svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
+ <path d="M15.379 0a15.381 15.381 0 00-4.86 29.974c.769.141 1.05-.334 1.05-.741 0-.365-.013-1.332-.021-2.616-4.278.929-5.181-2.062-5.181-2.062a4.071 4.071 0 00-1.708-2.25c-1.4-.954.106-.935.106-.935a3.231 3.231 0 012.356 1.585 3.274 3.274 0 004.476 1.278 3.287 3.287 0 01.976-2.056c-3.415-.388-7.005-1.707-7.005-7.6a5.947 5.947 0 011.583-4.127 5.53 5.53 0 01.151-4.07s1.291-.413 4.229 1.577a14.577 14.577 0 017.7 0c2.936-1.99 4.225-1.577 4.225-1.577a5.523 5.523 0 01.15 [...]
+ </svg>
+
+ <span class="bodytext__medium--brownish-grey">Suggest a change on this page</span>
+ </button>
+ </a>
+ </div>
+
+
+ </div>
+
+
+
+
+<footer>
+ <div class="footer-section footer-section__media-section">
+ <div class="d-flex align-items-center">
+
+ <a class="footer-section__media-section--link" target="_blank" href="https://github.com/apache/airflow">
+ <svg xmlns="http://www.w3.org/2000/svg" width="46.14" height="45" viewBox="0 0 46.14 45">
+ <path id="Path_207" d="M228.962 1078.578a23.072 23.072 0 0 0-7.29 44.96c1.154.212 1.574-.5 1.574-1.112 0-.548-.02-2-.031-3.924-6.417 1.394-7.771-3.093-7.771-3.093a6.109 6.109 0 0 0-2.562-3.375c-2.095-1.431.159-1.4.159-1.4a4.846 4.846 0 0 1 3.533 2.377c2.058 3.525 5.4 2.507 6.714 1.917a4.926 4.926 0 0 1 1.464-3.084c-5.123-.582-10.508-2.562-10.508-11.4a8.919 8.919 0 0 1 2.374-6.191 8.3 8.3 0 0 1 .226-6.105s1.937-.62 6.344 2.365a21.857 21.857 0 0 1 11.551 0c4.4-2.985 6.3 [...]
+ </svg>
+
+ </a>
+
+ <a class="footer-section__media-section--link" target="_blank" href="https://github.com/apache/airflow/issues">
+ <svg xmlns="http://www.w3.org/2000/svg" width="45" height="45" viewBox="0 0 45 45">
+ <g id="Group_210" data-name="Group 210" transform="translate(-339.789 -1315.282)">
+ <path id="Path_218" d="M394.82 1315.282h-21.671a9.784 9.784 0 0 0 9.784 9.778h3.986v3.857a9.784 9.784 0 0 0 9.784 9.771v-21.523a1.884 1.884 0 0 0-1.883-1.883z" fill="#fff" data-name="Path 218" transform="translate(-11.914)"></path>
+ <path id="Path_219" d="M378.14 1332.072h-21.671a9.778 9.778 0 0 0 9.778 9.778h4.018v3.857a9.784 9.784 0 0 0 9.752 9.778v-21.536a1.877 1.877 0 0 0-1.877-1.877z" fill="#fff" data-name="Path 219" transform="translate(-5.957 -5.996)"></path>
+ <path id="Path_220" d="M361.46 1348.862h-21.671a9.778 9.778 0 0 0 9.778 9.778h3.992v3.857a9.778 9.778 0 0 0 9.778 9.778v-21.529a1.883 1.883 0 0 0-1.877-1.884z" fill="#fff" data-name="Path 220" transform="translate(0 -11.993)"></path>
+ </g>
+ </svg>
+
+ </a>
+
+ <a class="footer-section__media-section--link" target="_blank" href="https://s.apache.org/airflow-slack">
+ <svg xmlns="http://www.w3.org/2000/svg" width="45.073" height="45.073" viewBox="0 0 45.073 45.073">
+ <g id="Group_208" data-name="Group 208" transform="translate(-661.145 -806.287)">
+ <g id="Group_204" data-name="Group 204" transform="translate(661.145 830.01)">
+ <path id="Path_208" d="M670.634 856.859a4.744 4.744 0 1 1-4.744-4.744h4.744z" fill="#fff" data-name="Path 208" transform="translate(-661.145 -852.115)"></path>
+ <path id="Path_209" d="M684.059 856.859a4.744 4.744 0 0 1 9.489 0v11.861a4.744 4.744 0 1 1-9.489 0z" fill="#fff" data-name="Path 209" transform="translate(-672.198 -852.115)"></path>
+ </g>
+ <g id="Group_205" data-name="Group 205" transform="translate(661.145 806.287)">
+ <path id="Path_210" d="M688.8 815.776a4.744 4.744 0 1 1 4.744-4.745v4.745z" fill="#fff" data-name="Path 210" transform="translate(-672.198 -806.287)"></path>
+ <path id="Path_211" d="M677.751 829.2a4.744 4.744 0 0 1 0 9.489H665.89a4.744 4.744 0 1 1 0-9.489z" fill="#fff" data-name="Path 211" transform="translate(-661.145 -817.34)"></path>
+ </g>
+ <g id="Group_206" data-name="Group 206" transform="translate(684.868 806.287)">
+ <path id="Path_212" d="M729.887 833.945a4.744 4.744 0 1 1 4.745 4.745h-4.745z" fill="#fff" data-name="Path 212" transform="translate(-718.026 -817.34)"></path>
+ <path id="Path_213" d="M716.462 822.893a4.744 4.744 0 1 1-9.489 0v-11.862a4.744 4.744 0 0 1 9.489 0z" fill="#fff" data-name="Path 213" transform="translate(-706.973 -806.287)"></path>
+ </g>
+ <g id="Group_207" data-name="Group 207" transform="translate(684.868 830.01)">
+ <path id="Path_214" d="M711.718 875.029a4.744 4.744 0 1 1-4.745 4.744v-4.744z" fill="#fff" data-name="Path 214" transform="translate(-706.973 -863.168)"></path>
+ <path id="Path_215" d="M711.718 861.6a4.744 4.744 0 1 1 0-9.489h11.861a4.744 4.744 0 0 1 0 9.489z" fill="#fff" data-name="Path 215" transform="translate(-706.973 -852.115)"></path>
+ </g>
+ </g>
+ </svg>
+
+ </a>
+
+ <a class="footer-section__media-section--link" target="_blank" href="https://stackoverflow.com/questions/tagged/airflow">
+ <svg xmlns="http://www.w3.org/2000/svg" width="37.647" height="44.6" viewBox="0 0 37.647 44.6">
+ <g id="Group_209" data-name="Group 209" transform="translate(-645.2 -975.455)">
+ <path id="Path_216" d="M677.028 1043.1v-11.948h3.966v15.914H645.2v-15.914h3.966v11.948z" fill="#fff" data-name="Path 216" transform="translate(0 -27.014)"></path>
+ <path id="Path_217" d="M661.012 1003.008l19.467 4.069.824-3.914-19.467-4.069zm2.575-9.27l18.025 8.395 1.648-3.605-18.025-8.446zm5-8.858l15.3 12.721 2.524-3.039-15.3-12.721zm9.888-9.425l-3.193 2.369 11.845 15.965 3.193-2.369zm-17.875 36.617h19.879v-3.966H660.6z" fill="#fff" data-name="Path 217" transform="translate(-7.469)"></path>
+ </g>
+ </svg>
+
+ </a>
+
+ <a class="footer-section__media-section--link" target="_blank" href="https://twitter.com/ApacheAirflow">
+ <svg xmlns="http://www.w3.org/2000/svg" width="44.355" height="44.355" viewBox="0 0 44.355 44.355">
+ <g id="Group_211" data-name="Group 211" transform="translate(-503 -1382.2)">
+ <path id="Path_221" d="M541.811 1382.2h-33.267a5.546 5.546 0 0 0-5.544 5.544v33.266a5.547 5.547 0 0 0 5.544 5.545h33.267a5.546 5.546 0 0 0 5.544-5.545v-33.266a5.546 5.546 0 0 0-5.544-5.544zm-5.611 16.533c.011.244.011.488.011.732 0 7.507-5.722 16.178-16.179 16.178a16.173 16.173 0 0 1-8.7-2.539 10.92 10.92 0 0 0 1.353.078 11.4 11.4 0 0 0 7.064-2.44 5.685 5.685 0 0 1-5.311-3.947 5.725 5.725 0 0 0 2.561-.1 5.7 5.7 0 0 1-4.557-5.578v-.078a5.752 5.752 0 0 0 2.572.71 5.7 [...]
+ </g>
+ </svg>
+
+ </a>
+
+ <a class="footer-section__media-section--link" target="_blank" href="https://www.youtube.com/channel/UCSXwxpWZQ7XZ1WL3wqevChA">
+ <svg xmlns="http://www.w3.org/2000/svg" width="49.594" height="34.941" viewBox="0 0 49.594 34.941">
+ <path id="Path_223" d="M1124.557 1230a6.232 6.232 0 0 0-4.385-4.413c-3.867-1.043-19.376-1.043-19.376-1.043s-15.508 0-19.376 1.043a6.232 6.232 0 0 0-4.385 4.413c-1.036 3.893-1.036 12.014-1.036 12.014s0 8.122 1.036 12.015a6.232 6.232 0 0 0 4.385 4.413c3.867 1.043 19.376 1.043 19.376 1.043s15.509 0 19.376-1.043a6.232 6.232 0 0 0 4.385-4.413c1.036-3.893 1.036-12.015 1.036-12.015s.001-8.123-1.036-12.014zm-28.833 19.388v-14.748l12.962 7.374z" fill="#fff" data-name="Path 223 [...]
+ </svg>
+
+ </a>
+
+ </div>
+
+
+ <div class="footer-section__media-section--button-with-text">
+ <span class="footer-section__media-section--text">Want to be a part of Apache Airflow?</span>
+ <a href="/community">
+
+ <button id="" class="btn-filled bodytext__medium--white ">Join community</button>
+
+ </a>
+ </div>
+
+
+ </div>
+ <div class="footer-section footer-section__policies-section">
+ <div class="footer-section">
+ <span>© The Apache Software Foundation <script>document.write(new Date().getFullYear())</script></span>
+ <div class="footer-section__policies-section--policies">
+
+ <a href="https://www.apache.org/licenses/" class="footer-section__policies-section--policy-item">
+ <span>License</span>
+ </a>
+
+ <a href="https://www.apache.org/foundation/sponsorship.html" class="footer-section__policies-section--policy-item">
+ <span>Donate</span>
+ </a>
+
+ <a href="https://www.apache.org/foundation/thanks.html" class="footer-section__policies-section--policy-item">
+ <span>Thanks</span>
+ </a>
+
+ <a href="https://www.apache.org/security/" class="footer-section__policies-section--policy-item">
+ <span>Security</span>
+ </a>
+
+
+ </div>
+ </div>
+ <span class="footer-section__policies-section--disclaimer">
+ Apache Airflow, Apache, Airflow, the Airflow logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.
+ All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.
+ </span>
+ </div>
+</footer>
+ <script type="text/javascript" src="../../../../../../../_static/_gen/js/docs.js"></script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../../../../" src="../../../../../../../_static/documentation_options.js"></script>
+ <script src="https://code.jquery.com/jquery-3.4.1.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
+ <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js" integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy" crossorigin="anonymous"></script>
+ <script data-url_root="../../../../../../../" id="documentation_options" src="../../../../../../../_static/documentation_options.js"></script>
+ <script src="../../../../../../../_static/jquery.js"></script>
+ <script src="../../../../../../../_static/underscore.js"></script>
+ <script src="../../../../../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
+ <script src="../../../../../../../_static/doctools.js"></script>
+ <script src="../../../../../../../_static/sphinx_highlight.js"></script>
+ <script src="../../../../../../../_static/clipboard.min.js"></script>
+ <script src="../../../../../../../_static/copybutton.js"></script>
+ <script src="../../../../../../../_static/js/globaltoc.js"></script>
+</body>
+</html>
\ No newline at end of file
diff --git a/docs-archive/apache-airflow-providers/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/batch_client/index.html
similarity index 52%
copy from docs-archive/apache-airflow-providers/index.html
copy to docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/batch_client/index.html
index 70eef65adf..ee203882ae 100644
--- a/docs-archive/apache-airflow-providers/index.html
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/batch_client/index.html
@@ -9,18 +9,20 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Provider packages — apache-airflow-providers Documentation</title>
- <link rel="stylesheet" href="_static/_gen/css/main.min.css" type="text/css" />
- <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main.min.css" />
- <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main-custom.min.css" />
- <link rel="shortcut icon" href="_static/pin_32.png"/>
- <link rel="index" title="Index" href="genindex.html" />
- <link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Providers packages reference" href="packages-ref.html" />
-
- <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers/stable/index.html" />
+ <title>airflow.providers.amazon.aws.hooks.batch_client — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../../search.html" />
+ <link rel="next" title="airflow.providers.amazon.aws.hooks.batch_waiters" href="../batch_waiters/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.hooks.base_aws" href="../base_aws/index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/hooks/batch_client/index.html" />
<!-- Matomo -->
<script>
@@ -282,8 +284,21 @@
<div class="content-drawer">
<div class="td-sidebar">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -313,14 +328,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -401,8 +434,21 @@
<div class="td-sidebar desktop-only d-print-none">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -432,14 +478,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -528,9 +592,15 @@
<ul class="breadcrumb">
- <li class="breadcrumb-item"><a href="#" class="icon icon-home"> Home</a></li>
+ <li class="breadcrumb-item"><a href="../../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
- <li class="breadcrumb-item"><a href="index.html"> Provider packages</a></li>
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.batch_client</span></code></a></li>
</ul>
</div>
@@ -541,440 +611,543 @@
<div class="bodywrapper">
<div class="body" role="main">
- <blockquote>
-<div></div></blockquote>
-<div class="section" id="provider-packages">
-<h1>Provider packages<a class="headerlink" href="#provider-packages" title="Permalink to this heading">¶</a></h1>
-<p>Apache Airflow 2 is built in modular way. The “Core” of Apache Airflow provides core scheduler
-functionality which allow you to write some basic tasks, but the capabilities of Apache Airflow can
-be extended by installing additional packages, called <code class="docutils literal notranslate"><span class="pre">providers</span></code>.</p>
-<p>Providers can contain operators, hooks, sensor, and transfer operators to communicate with a
-multitude of external systems, but they can also extend Airflow core with new capabilities.</p>
-<p>You can install those provider packages separately in order to interface with a given service. The providers
-for <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> are designed in the way that you can write your own providers easily. The
-<code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span> <span class="pre">Community</span></code> develops and maintain more than 60 provider packages, but you are free to
-develop your own providers - the providers you build have exactly the same capability as the providers
-written by the community, so you can release and share those providers with others.</p>
-<p>The full list of community managed providers is available at
-<a class="reference external" href="https://airflow.apache.org/docs/#providers-packages-docs-apache-airflow-providers-index-html">Providers Index</a>.</p>
-<p>You can also see index of all community provider’s operators and hooks in
-<a class="reference internal" href="operators-and-hooks-ref/index.html"><span class="doc">Operators and Hooks Reference</span></a></p>
-</div>
-<div class="section" id="extending-airflow-core-functionality">
-<h1>Extending Airflow core functionality<a class="headerlink" href="#extending-airflow-core-functionality" title="Permalink to this heading">¶</a></h1>
-<p>Providers give you the capability of extending core Airflow with extra capabilities. The Core airflow
-provides basic and solid functionality of scheduling, the providers extend its capabilities. Here we
-describe all the custom capabilities.</p>
-<p>Airflow automatically discovers which providers add those additional capabilities and, once you install
-provider package and re-start Airflow, those become automatically available to Airflow Users.</p>
-<p>The summary of the core functionalities that can be extended are available in
-<a class="reference internal" href="core-extensions/index.html"><span class="doc">Core Extensions</span></a>.</p>
-<div class="section" id="auth-backends">
-<h2>Auth backends<a class="headerlink" href="#auth-backends" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom authentication backends, that allow you to configure the way how your
-web server authenticates your users, integrating it with public or private authentication services.</p>
-<p>You can see all the authentication backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/auth-backends.html"><span class="doc">Auth backends</span></a></p>
-</div>
-<div class="section" id="custom-connections">
-<h2>Custom connections<a class="headerlink" href="#custom-connections" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom connection types, extending connection form and handling custom form field
-behaviour for the connections defined by the provider.</p>
-<p>You can see all custom connections available via community-managed providers in
-<a class="reference internal" href="core-extensions/connections.html"><span class="doc">Connections</span></a>.</p>
-</div>
-<div class="section" id="extra-links">
-<h2>Extra links<a class="headerlink" href="#extra-links" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add extra custom links to operators delivered by the provider. Those will be visible in
-task details view of the task.</p>
-<p>You can see all the extra links available via community-managed providers in
-<a class="reference internal" href="core-extensions/extra-links.html"><span class="doc">Extra Links</span></a>.</p>
-</div>
-<div class="section" id="logging">
-<h2>Logging<a class="headerlink" href="#logging" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add additional task logging capabilities. By default <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> saves logs for
-tasks locally and make them available to Airflow UI via internal http server. However, providers
-can add extra logging capabilities, where Airflow Logs can be written to a remote service and
-retrieved from those services.</p>
-<p>You can see all task loggers available via community-managed providers in
-<a class="reference internal" href="core-extensions/logging.html"><span class="doc">Writing logs</span></a>.</p>
-</div>
-<div class="section" id="secret-backends">
-<h2>Secret backends<a class="headerlink" href="#secret-backends" title="Permalink to this heading">¶</a></h2>
-<p>Airflow has the capability of reading connections, variables and configuration from Secret Backends rather
-than from its own Database.</p>
-<p>You can see all secret backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/secrets-backends.html"><span class="doc">Secret backends</span></a>.</p>
-</div>
-<div class="section" id="notifications">
-<h2>Notifications<a class="headerlink" href="#notifications" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom notifications, that allow you to configure the way how you would like to receive
-notifications about the status of your tasks/DAGs.</p>
-<p>You can see all the notifications available via community-managed providers in
-<a class="reference internal" href="core-extensions/notifications.html"><span class="doc">Notifications</span></a>.</p>
-</div>
+ <div class="section" id="module-airflow.providers.amazon.aws.hooks.batch_client">
+<span id="airflow-providers-amazon-aws-hooks-batch-client"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.hooks.batch_client" title="airflow.providers.amazon.aws.hooks.batch_client"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.batch_client</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.hooks.batch_client" title="Permalink to this heading">¶</a></h1>
+<p>A client for AWS Batch services</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html">https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html</a></p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/batch.html">https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/batch.html</a></p></li>
+<li><p><a class="reference external" href="https://docs.aws.amazon.com/batch/latest/APIReference/Welcome.html">https://docs.aws.amazon.com/batch/latest/APIReference/Welcome.html</a></p></li>
+</ul>
</div>
-<div class="section" id="installing-and-upgrading-providers">
-<h1>Installing and upgrading providers<a class="headerlink" href="#installing-and-upgrading-providers" title="Permalink to this heading">¶</a></h1>
-<p>Separate provider packages give the possibilities that were not available in 1.10:</p>
-<ol class="arabic simple">
-<li><p>You can upgrade to latest version of particular providers without the need of Apache Airflow core upgrade.</p></li>
-<li><p>You can downgrade to previous version of particular provider in case the new version introduces
-some problems, without impacting the main Apache Airflow core package.</p></li>
-<li><p>You can release and upgrade/downgrade provider packages incrementally, independent from each other. This
-means that you can incrementally validate each of the provider package update in your environment,
-following the usual tests you have in your environment.</p></li>
-</ol>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<div class="section" id="classes">
+<h3>Classes<a class="headerlink" href="#classes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol" title="airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol"><code class="xref py py-obj docutils literal notranslate"><span class="pre">BatchProtocol</span></code></a></p></td>
+<td><p>A structured Protocol for <code class="docutils literal notranslate"><span class="pre">boto3.client('batch')</span> <span class="pre">-></span> <span class="pre">botocore.client.Batch</span></code>.</p></td>
+</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook" title="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">BatchClientHook</span></code></a></p></td>
+<td><p>Interact with AWS Batch.</p></td>
+</tr>
+</tbody>
+</table>
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.batch_client.</span></span><span class="sig-name descname"><span class="pre">BatchProtocol</span></span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchProtocol"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class= [...]
+<dd><p>Bases: <code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.typing_compat.Protocol</span></code></p>
+<p>A structured Protocol for <code class="docutils literal notranslate"><span class="pre">boto3.client('batch')</span> <span class="pre">-></span> <span class="pre">botocore.client.Batch</span></code>.
+This is used for type hints on <code class="xref py py-meth docutils literal notranslate"><span class="pre">BatchClient.client()</span></code>; it covers
+only the subset of client methods required.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://mypy.readthedocs.io/en/latest/protocols.html">https://mypy.readthedocs.io/en/latest/protocols.html</a></p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/batch.html">https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/batch.html</a></p></li>
+</ul>
</div>
-<div class="section" id="types-of-providers">
-<h1>Types of providers<a class="headerlink" href="#types-of-providers" title="Permalink to this heading">¶</a></h1>
-<p>Providers have the same capacity - no matter if they are provided by the community or if they are
-third-party providers. This chapter explains how community managed providers are versioned and released
-and how you can create your own providers.</p>
-<div class="section" id="community-maintained-providers">
-<h2>Community maintained providers<a class="headerlink" href="#community-maintained-providers" title="Permalink to this heading">¶</a></h2>
-<p>From the point of view of the community, Airflow is delivered in multiple, separate packages.
-The core of Airflow scheduling system is delivered as <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package and there are more than
-60 provider packages which can be installed separately as so called <code class="docutils literal notranslate"><span class="pre">Airflow</span> <span class="pre">Provider</span> <span class="pre">packages</span></code>.
-Those packages are available as <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers</span></code> packages - for example there is an
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> or <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> package).</p>
-<p>Community maintained providers are released and versioned separately from the Airflow releases. We are
-following the <a class="reference external" href="https://semver.org/">Semver</a> versioning scheme for the packages. Some versions of the
-provider packages might depend on particular versions of Airflow, but the general approach we have is that
-unless there is a good reason, new version of providers should work with recent versions of Airflow 2.x.
-Details will vary per-provider and if there is a limitation for particular version of particular provider,
-constraining the Airflow version used, it will be included as limitation of dependencies in the provider
-package.</p>
-<p>Each community provider has corresponding extra which can be used when installing airflow to install the
-provider together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> - for example you can install airflow with those extras:
-<code class="docutils literal notranslate"><span class="pre">apache-airflow[google,amazon]</span></code> (with correct constraints -see <a class="reference external" href="/docs/apache-airflow/stable/installation/index.html" title="(in apache-airflow v2.6.0.dev0)"><span>Installation</span></a>) and you
-will install the appropriate versions of the <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> and
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> packages together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>.</p>
-<p>Some of the community providers have cross-provider dependencies as well. Those are not required
-dependencies, they might simply enable certain features (for example transfer operators often create
-dependency between different providers. Again, the general approach here is that the providers are backwards
-compatible, including cross-dependencies. Any kind of breaking changes and requirements on particular versions of other
-provider packages are automatically documented in the release notes of every provider.</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol.describe_jobs">
+<span class="sig-name descname"><span class="pre">describe_jobs</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">jobs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchProtocol.describe_jobs"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazo [...]
+<dd><p>Get job descriptions from AWS Batch</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>jobs</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#list" title="(in Python v3.11)"><em>list</em></a><em>[</em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em>]</em>) – a list of JobId to describe</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>an API response to describe jobs</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol.get_waiter">
+<span class="sig-name descname"><span class="pre">get_waiter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">waiterName</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchProtocol.get_waiter"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazo [...]
+<dd><p>Get an AWS Batch service waiter</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>waiterName</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – The name of the waiter. The name should match
+the name (including the casing) of the key name in the waiter
+model file (typically this is CamelCasing).</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>a waiter object for the named AWS Batch service</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p>botocore.waiter.Waiter</p>
+</dd>
+</dl>
<div class="admonition note">
<p class="admonition-title">Note</p>
-<p>For Airflow 1.10 We also provided <code class="docutils literal notranslate"><span class="pre">apache-airflow-backport-providers</span></code> packages that could be installed
-with those versions Those were the same providers as for 2.0 but automatically back-ported to work for
-Airflow 1.10. The last release of backport providers was done on March 17, 2021 and the backport
-providers will no longer be released, since Airflow 1.10 has reached End-Of-Life as of June 17, 2021.</p>
-</div>
-<p>If you want to contribute to <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>, you can see how to build and extend community
-managed providers in <a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a>.</p>
-</div>
-<div class="section" id="custom-provider-packages">
-<span id="providers-community-maintained-providers"></span><h2>Custom provider packages<a class="headerlink" href="#custom-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p>You can develop and release your own providers. Your custom operators, hooks, sensors, transfer operators
-can be packaged together in a standard airflow package and installed using the same mechanisms.
-Moreover they can also use the same mechanisms to extend the Airflow Core with auth backends,
-custom connections, logging, secret backends and extra operator links as described in the previous chapter.</p>
+<p>AWS Batch might not have any waiters (until botocore PR-1307 is released).</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">boto3</span>
+
+<span class="n">boto3</span><span class="o">.</span><span class="n">client</span><span class="p">(</span><span class="s2">"batch"</span><span class="p">)</span><span class="o">.</span><span class="n">waiter_names</span> <span class="o">==</span> <span class="p">[]</span>
+</pre></div>
</div>
</div>
-<div class="section" id="how-to-create-your-own-provider">
-<h1>How to create your own provider<a class="headerlink" href="#how-to-create-your-own-provider" title="Permalink to this heading">¶</a></h1>
-<p>As mentioned in the <a class="reference external" href="http://airflow.apache.org/docs/apache-airflow-providers/index.html">Providers</a>
-documentation, custom providers can extend Airflow core - they can add extra links to operators as well
-as custom connections. You can use build your own providers and install them as packages if you would like
-to use the mechanism for your own, custom providers.</p>
-<p>Adding a provider to Airflow is just a matter of building a Python package and adding the right meta-data to
-the package. We are using standard mechanism of python to define
-<a class="reference external" href="https://docs.python.org/3/library/importlib.metadata.html#entry-points">entry points</a> . Your package
-needs to define appropriate entry-point <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> which has to point to a callable
-implemented by your package and return a dictionary containing the list of discoverable capabilities
-of your package. The dictionary has to follow the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json">json-schema specification</a>.</p>
-<p>Most of the schema provides extension point for the documentation (which you might want to also use for
-your own purpose) but the important fields from the extensibility point of view are those:</p>
-<p>Displaying package information in CLI/API:</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">package-name</span></code> - Name of the package for the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code> - Human-friendly name of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">description</span></code> - Additional description of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">version</span></code> - List of versions of the package (in reverse-chronological order). The first version in the
-list is the current package version. It is taken from the version of package installed, not from the
-provider_info information.</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/clients.html#waiters">https://boto3.amazonaws.com/v1/documentation/api/latest/guide/clients.html#waiters</a></p></li>
+<li><p><a class="reference external" href="https://github.com/boto/botocore/pull/1307">https://github.com/boto/botocore/pull/1307</a></p></li>
</ul>
-<p>Exposing customized functionality to the Airflow’s core:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">extra-links</span></code> - this field should contain the list of all operator class names that are adding extra links
-capability. See <a class="reference external" href="/docs/apache-airflow/stable/howto/define-extra-link.html" title="(in apache-airflow v2.6.0.dev0)"><span>Define an operator extra link</span></a> for description of how to add extra link
-capability to the operators of yours.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">connection-types</span></code> - this field should contain the list of all connection types together with hook
-class names implementing those custom connection types (providing custom extra fields and
-custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
-<code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> (deprecated) - this field should contain the list of all hook class names that provide
-custom connection types with custom extra fields and field behaviour. The <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array
-is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
-providers are targeting Airflow 2.2.0+ you do not have to include the <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array, if
-you want to also target earlier versions of Airflow 2, you should include both <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> and
-<code class="docutils literal notranslate"><span class="pre">connection-types</span></code> arrays. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details.</p></li>
-</ul>
-<p>When your providers are installed you can query the installed providers and their capabilities with the
-<code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">providers</span></code> command. This way you can verify if your providers are properly recognized and whether
-they define the extensions properly. See <a class="reference external" href="/docs/apache-airflow/stable/cli-and-env-variables-ref.html" title="(in apache-airflow v2.6.0.dev0)"><span>Command Line Interface and Environment Variables Reference</span></a> for details of available CLI
-sub-commands.</p>
-<p>When you write your own provider, consider following the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#naming-conventions-for-provider-packages">Naming conventions for provider packages</a></p>
</div>
-<div class="section" id="faq-for-airflow-and-providers">
-<h1>FAQ for Airflow and Providers<a class="headerlink" href="#faq-for-airflow-and-providers" title="Permalink to this heading">¶</a></h1>
-<div class="section" id="upgrading-airflow-2-0-and-providers">
-<h2>Upgrading Airflow 2.0 and Providers<a class="headerlink" href="#upgrading-airflow-2-0-and-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When upgrading to a new Airflow version such as 2.0, but possibly 2.0.1 and beyond, is the best practice
-to also upgrade provider packages at the same time?</strong></p>
-<p>It depends on your use case. If you have automated or semi-automated verification of your installation,
-that you can run a new version of Airflow including all provider packages, then definitely go for it.
-If you rely more on manual testing, it is advised that you upgrade in stages. Depending on your choice
-you can either upgrade all used provider packages first, and then upgrade Airflow Core or the other way
-round. The first approach - when you first upgrade all providers is probably safer, as you can do it
-incrementally, step-by-step replacing provider by provider in your environment.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol.submit_job">
+<span class="sig-name descname"><span class="pre">submit_job</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">jobName</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">jobQueue</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">jobDefinition</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">arrayProperties</span></span></em>, <em class="sig-param"><span class=" [...]
+<dd><p>Submit a Batch job</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>jobName</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – the name for the AWS Batch job</p></li>
+<li><p><strong>jobQueue</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – the queue name on AWS Batch</p></li>
+<li><p><strong>jobDefinition</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – the job definition name on AWS Batch</p></li>
+<li><p><strong>arrayProperties</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a>) – the same parameter that boto3 will receive</p></li>
+<li><p><strong>parameters</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a>) – the same parameter that boto3 will receive</p></li>
+<li><p><strong>containerOverrides</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a>) – the same parameter that boto3 will receive</p></li>
+<li><p><strong>tags</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a>) – the same parameter that boto3 will receive</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>an API response</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol.terminate_job">
+<span class="sig-name descname"><span class="pre">terminate_job</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">jobId</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">reason</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchProtocol.terminate_job"><span class="viewcode-link"><span class= [...]
+<dd><p>Terminate a Batch job</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>jobId</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a job ID to terminate</p></li>
+<li><p><strong>reason</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a reason to terminate job ID</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>an API response</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.batch_client.</span></span><span class="sig-name descname"><span class="pre">BatchClientHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="n"><span class=" [...]
+<dd><p>Bases: <a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p>
+<p>Interact with AWS Batch.
+Provide thick wrapper around <a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/batch.html#Batch.Client" title="(in Boto3 v1.26.120)"><code class="xref py py-class docutils literal notranslate"><span class="pre">boto3.client("batch")</span></code></a>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>max_retries</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><em>None</em>) – exponential back-off retries, 4200 = 48 hours;
+polling is only used when waiters is None</p></li>
+<li><p><strong>status_retries</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><em>None</em>) – number of HTTP retries to get job status, 10;
+polling is only used when waiters is None</p></li>
+</ul>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Several methods use a default random delay to check or poll for job status, i.e.
+<code class="docutils literal notranslate"><span class="pre">random.uniform(DEFAULT_DELAY_MIN,</span> <span class="pre">DEFAULT_DELAY_MAX)</span></code>
+Using a random interval helps to avoid AWS API throttle limits
+when many concurrent tasks request job-descriptions.</p>
+<p>To modify the global defaults for the range of jitter allowed when a
+random delay is used to check Batch job status, modify these defaults, e.g.:
+.. code-block:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">BatchClient</span><span class="o">.</span><span class="n">DEFAULT_DELAY_MIN</span> <span class="o">=</span> <span class="mi">0</span>
+<span class="n">BatchClient</span><span class="o">.</span><span class="n">DEFAULT_DELAY_MAX</span> <span class="o">=</span> <span class="mi">5</span>
+</pre></div>
</div>
-<div class="section" id="customizing-provider-packages">
-<h2>Customizing Provider Packages<a class="headerlink" href="#customizing-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an older version of my provider package which we have lightly customized and is working
-fine with my MSSQL installation. I am upgrading my Airflow version. Do I need to upgrade my provider,
-or can I keep it as it is?</strong></p>
-<p>It depends on the scope of customization. There is no need to upgrade the provider packages to later
-versions unless you want to upgrade to Airflow version that introduces backwards-incompatible changes.
-Generally speaking, with Airflow 2 we are following the <a class="reference external" href="https://semver.org/">Semver</a> approach where
-we will introduce backwards-incompatible changes in Major releases, so all your modifications (as long
-as you have not used internal Airflow classes) should work for All Airflow 2.* versions.</p>
+<p>When explicit delay values are used, a 1 second random jitter is applied to the
+delay (e.g. a delay of 0 sec will be a <code class="docutils literal notranslate"><span class="pre">random.uniform(0,</span> <span class="pre">1)</span></code> delay. It is
+generally recommended that random jitter is added to API requests. A
+convenience method is provided for this, e.g. to get a random delay of
+10 sec +/- 5 sec: <code class="docutils literal notranslate"><span class="pre">delay</span> <span class="pre">=</span> <span class="pre">BatchClient.add_jitter(10,</span> <span class="pre">width=5,</span> <span class="pre">minima=0)</span></code></p>
</div>
-<div class="section" id="creating-your-own-providers">
-<h2>Creating your own providers<a class="headerlink" href="#creating-your-own-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When I write my own provider, do I need to do anything special to make it available to others?</strong></p>
-<p>You do not need to do anything special besides creating the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point
-returning properly formatted meta-data - dictionary with <code class="docutils literal notranslate"><span class="pre">extra-links</span></code> and <code class="docutils literal notranslate"><span class="pre">connection-types</span></code> fields
-(and deprecated <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> field if you are also targeting versions of Airflow before 2.2.0).</p>
-<p>Anyone who runs airflow in an environment that has your Python package installed will be able to use the
-package as a provider package.</p>
-<p><strong>What do I need to do to turn a package into a provider?</strong></p>
-<p>You need to do the following to turn an existing Python package into a provider (see below for examples):</p>
+<p>Additional arguments (such as <code class="docutils literal notranslate"><span class="pre">aws_conn_id</span></code>) may be specified and
+are passed down to the underlying AwsBaseHook.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Add the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point in the <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> - this tells airflow where to get
-the required provider metadata</p></li>
-<li><p>Create the function that you refer to in the first step as part of your package: this functions returns a
-dictionary that contains all meta-data about your provider package</p></li>
-<li><p>If you want Airflow to link to documentation of your Provider in the providers page, make sure
-to add “project-url/documentation” <a class="reference external" href="https://peps.python.org/pep-0621/#example">metadata</a> to your package.
-This will also add link to your documentation in PyPI.</p></li>
-<li><p>note that the dictionary should be compliant with <code class="docutils literal notranslate"><span class="pre">airflow/provider_info.schema.json</span></code> JSON-schema
-specification. The community-managed providers have more fields there that are used to build
-documentation, but the requirement for runtime information only contains several fields which are defined
-in the schema:</p></li>
+<li><p><a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-class docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p></li>
+<li><p><a class="reference external" href="https://docs.aws.amazon.com/general/latest/gr/api-retries.html">https://docs.aws.amazon.com/general/latest/gr/api-retries.html</a></p></li>
+<li><p><a class="reference external" href="https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/">https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/</a></p></li>
</ul>
-<div class="example-block-wrapper compound">
-<p class="example-header compound-first"><span class="example-title">airflow/provider_info.schema.json</span></p>
-<div class="compound-last highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"$schema"</span><span class="p">:</span><span class="w"> </span><span class="s2">"http://json-schema.org/draft-07/schema#"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"package-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Package name available under which the package is available in the PyPI repository."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Provider name"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Information about the package in RST format"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-names"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class names that provide connection types to core (deprecated by connection-types)"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"deprecated"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"The hook-class-names property has been deprecated in favour of connection-types which is more performant version allowing to only import individual Hooks rather than all hooks at once"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"deprecatedVersion"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.2.0"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"connection-types"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Map of connection types mapped to hook class names."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"connection-type"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Type of connection defined by the provider"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class name that implements the connection type"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"connection-type"</span><span class="p">,</span><span class="w"> </span><span class="s2">"hook-class-name"</span><span class="p">]</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"extra-links"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Operator class names that provide extra link functionality"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"secrets-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Secrets Backend class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"logging"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Logging Task Handlers class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"auth-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"API Auth Backend module names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"notifications"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Notification class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"task-decorators"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Apply custom decorators to the TaskFlow API. Can be accessed by users via '@task.<name>'"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"path"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
-<span class="w"> </span><span class="s2">"name"</span><span class="p">,</span>
-<span class="w"> </span><span class="s2">"description"</span>
-<span class="w"> </span><span class="p">]</span>
-<span class="p">}</span>
-</pre></div>
</div>
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.client">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">client</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol" title="airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol"><span class="pre">BatchProtocol</span></a><span class="w"> </span><span cla [...]
+<dd><p>An AWS API client for Batch services.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Returns</dt>
+<dd class="field-odd"><p>a boto3 ‘batch’ client for the <code class="docutils literal notranslate"><span class="pre">.region_name</span></code></p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol" title="airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol">BatchProtocol</a> | botocore.client.BaseClient</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.MAX_RETRIES">
+<span class="sig-name descname"><span class="pre">MAX_RETRIES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">4200</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.MAX_RETRIES"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.provider [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.STATUS_RETRIES">
+<span class="sig-name descname"><span class="pre">STATUS_RETRIES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">10</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.STATUS_RETRIES"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.prov [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.DEFAULT_DELAY_MIN">
+<span class="sig-name descname"><span class="pre">DEFAULT_DELAY_MIN</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.DEFAULT_DELAY_MIN"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.DEFAULT_DELAY_MAX">
+<span class="sig-name descname"><span class="pre">DEFAULT_DELAY_MAX</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">10</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.DEFAULT_DELAY_MAX"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflo [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.FAILURE_STATE">
+<span class="sig-name descname"><span class="pre">FAILURE_STATE</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'FAILED'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.FAILURE_STATE"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow. [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.SUCCESS_STATE">
+<span class="sig-name descname"><span class="pre">SUCCESS_STATE</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'SUCCEEDED'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.SUCCESS_STATE"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airfl [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.RUNNING_STATE">
+<span class="sig-name descname"><span class="pre">RUNNING_STATE</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'RUNNING'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.RUNNING_STATE"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.INTERMEDIATE_STATES">
+<span class="sig-name descname"><span class="pre">INTERMEDIATE_STATES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('SUBMITTED',</span> <span class="pre">'PENDING',</span> <span class="pre">'RUNNABLE',</span> <span class="pre">'STARTING')</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.INT [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.COMPUTE_ENVIRONMENT_TERMINAL_STATUS">
+<span class="sig-name descname"><span class="pre">COMPUTE_ENVIRONMENT_TERMINAL_STATUS</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('VALID',</span> <span class="pre">'DELETED')</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.COMPUTE_ENVIRONMENT_TERMINAL_STATUS"><span class="viewcode-link"> [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.COMPUTE_ENVIRONMENT_INTERMEDIATE_STATUS">
+<span class="sig-name descname"><span class="pre">COMPUTE_ENVIRONMENT_INTERMEDIATE_STATUS</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('CREATING',</span> <span class="pre">'UPDATING',</span> <span class="pre">'DELETING')</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.COMPUTE_ENVIRONMENT_ [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.JOB_QUEUE_TERMINAL_STATUS">
+<span class="sig-name descname"><span class="pre">JOB_QUEUE_TERMINAL_STATUS</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('VALID',</span> <span class="pre">'DELETED')</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.JOB_QUEUE_TERMINAL_STATUS"><span class="viewcode-link"><span class="pre">[s [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.JOB_QUEUE_INTERMEDIATE_STATUS">
+<span class="sig-name descname"><span class="pre">JOB_QUEUE_INTERMEDIATE_STATUS</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('CREATING',</span> <span class="pre">'UPDATING',</span> <span class="pre">'DELETING')</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.JOB_QUEUE_INTERMEDIATE_STATUS" [...]
+<dd></dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.terminate_job">
+<span class="sig-name descname"><span class="pre">terminate_job</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">reason</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.terminate_job"><span class="viewcode-link"><span cla [...]
+<dd><p>Terminate a Batch job</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a job ID to terminate</p></li>
+<li><p><strong>reason</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a reason to terminate job ID</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>an API response</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.check_job_success">
+<span class="sig-name descname"><span class="pre">check_job_success</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.check_job_success"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.pro [...]
+<dd><p>Check the final status of the Batch job; return True if the job
+‘SUCCEEDED’, else raise an AirflowException</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a Batch job ID</p>
+</dd>
+<dt class="field-even">Raises</dt>
+<dd class="field-even"><p>AirflowException</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.wait_for_job">
+<span class="sig-name descname"><span class="pre">wait_for_job</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">delay</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/pr [...]
+<dd><p>Wait for Batch job to complete</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a Batch job ID</p></li>
+<li><p><strong>delay</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.11)"><em>float</em></a><em> | </em><em>None</em>) – a delay before polling for job status</p></li>
+</ul>
+</dd>
+<dt class="field-even">Raises</dt>
+<dd class="field-even"><p>AirflowException</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.poll_for_job_running">
+<span class="sig-name descname"><span class="pre">poll_for_job_running</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">delay</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/ai [...]
+<dd><p>Poll for job running. The status that indicates a job is running or
+already complete are: ‘RUNNING’|’SUCCEEDED’|’FAILED’.</p>
+<p>So the status options that this will wait for are the transitions from:
+‘SUBMITTED’>’PENDING’>’RUNNABLE’>’STARTING’>’RUNNING’|’SUCCEEDED’|’FAILED’</p>
+<p>The completed status options are included for cases where the status
+changes too quickly for polling to detect a RUNNING status that moves
+quickly from STARTING to RUNNING to completed (often a failure).</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a Batch job ID</p></li>
+<li><p><strong>delay</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.11)"><em>float</em></a><em> | </em><em>None</em>) – a delay before polling for job status</p></li>
+</ul>
+</dd>
+<dt class="field-even">Raises</dt>
+<dd class="field-even"><p>AirflowException</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.poll_for_job_complete">
+<span class="sig-name descname"><span class="pre">poll_for_job_complete</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">delay</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/a [...]
+<dd><p>Poll for job completion. The status that indicates job completion
+are: ‘SUCCEEDED’|’FAILED’.</p>
+<p>So the status options that this will wait for are the transitions from:
+‘SUBMITTED’>’PENDING’>’RUNNABLE’>’STARTING’>’RUNNING’>’SUCCEEDED’|’FAILED’</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a Batch job ID</p></li>
+<li><p><strong>delay</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.11)"><em>float</em></a><em> | </em><em>None</em>) – a delay before polling for job status</p></li>
+</ul>
+</dd>
+<dt class="field-even">Raises</dt>
+<dd class="field-even"><p>AirflowException</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.poll_job_status">
+<span class="sig-name descname"><span class="pre">poll_job_status</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">match_status</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.poll_job_status"><span class="viewcode-link" [...]
+<dd><p>Poll for job status using an exponential back-off strategy (with max_retries).</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a Batch job ID</p></li>
+<li><p><strong>match_status</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#list" title="(in Python v3.11)"><em>list</em></a><em>[</em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em>]</em>) – a list of job status to match; the Batch job status are:
+‘SUBMITTED’|’PENDING’|’RUNNABLE’|’STARTING’|’RUNNING’|’SUCCEEDED’|’FAILED’</p></li>
+</ul>
+</dd>
+<dt class="field-even">Raises</dt>
+<dd class="field-even"><p>AirflowException</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.get_job_description">
+<span class="sig-name descname"><span class="pre">get_job_description</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.get_job_description"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow [...]
+<dd><p>Get job description (using status_retries).</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a Batch job ID</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>an API response for describe jobs</p>
+</dd>
+<dt class="field-odd">Raises</dt>
+<dd class="field-odd"><p>AirflowException</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.parse_job_description">
+<em class="property"><span class="pre">static</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">parse_job_description</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">response</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/h [...]
+<dd><p>Parse job description to extract description for job_id</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a Batch job ID</p></li>
+<li><p><strong>response</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a>) – an API response for describe jobs</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>an API response to describe job_id</p>
+</dd>
+<dt class="field-odd">Raises</dt>
+<dd class="field-odd"><p>AirflowException</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.get_job_awslogs_info">
+<span class="sig-name descname"><span class="pre">get_job_awslogs_info</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.get_job_awslogs_info"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airfl [...]
+<dd><dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.get_job_all_awslogs_info">
+<span class="sig-name descname"><span class="pre">get_job_all_awslogs_info</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.get_job_all_awslogs_info"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href [...]
+<dd><p>Parse job description to extract AWS CloudWatch information.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – AWS Batch Job ID</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.add_jitter">
+<em class="property"><span class="pre">static</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">add_jitter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">delay</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">width</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span cla [...]
+<dd><p>Use delay +/- width for random jitter</p>
+<p>Adding jitter to status polling can help to avoid
+AWS Batch API limits for monitoring Batch jobs with
+a high concurrency in Airflow tasks.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>delay</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.11)"><em>float</em></a>) – number of seconds to pause;
+delay is assumed to be a positive number</p></li>
+<li><p><strong>width</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.11)"><em>float</em></a>) – delay +/- width for random jitter;
+width is assumed to be a positive number</p></li>
+<li><p><strong>minima</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.11)"><em>float</em></a>) – minimum delay allowed;
+minima is assumed to be a non-negative number</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>uniform(delay - width, delay + width) jitter
+and it is a non-negative number</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.11)">float</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.delay">
+<em class="property"><span class="pre">static</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">delay</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">delay</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/a [...]
+<dd><p>Pause execution for <code class="docutils literal notranslate"><span class="pre">delay</span></code> seconds.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>delay</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.11)"><em>float</em></a><em> | </em><em>None</em>) – a delay to pause execution using <code class="docutils literal notranslate"><span class="pre">time.sleep(delay)</span></code>;
+a small 1 second jitter is applied to the delay.</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>This method uses a default random delay, i.e.
+<code class="docutils literal notranslate"><span class="pre">random.uniform(DEFAULT_DELAY_MIN,</span> <span class="pre">DEFAULT_DELAY_MAX)</span></code>;
+using a random interval helps to avoid AWS API throttle limits
+when many concurrent tasks request job-descriptions.</p>
</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>:</p>
-<div class="highlight-cfg notranslate"><div class="highlight"><pre><span></span><span class="k">[options.entry_points]</span>
-<span class="c1"># the function get_provider_info is defined in myproviderpackage.somemodule</span>
-<span class="na">apache_airflow_provider</span><span class="o">=</span>
-<span class="w"> </span><span class="na">provider_info</span><span class="o">=</span><span class="s">myproviderpackage.somemodule:get_provider_info</span>
-</pre></div>
-</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">myproviderpackage/somemodule.py</span></code>:</p>
-<div class="highlight-Python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_provider_info</span><span class="p">():</span>
- <span class="k">return</span> <span class="p">{</span>
- <span class="s2">"package-name"</span><span class="p">:</span> <span class="s2">"my-package-name"</span><span class="p">,</span>
- <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"name"</span><span class="p">,</span>
- <span class="s2">"description"</span><span class="p">:</span> <span class="s2">"a description"</span><span class="p">,</span>
- <span class="s2">"hook-class-names"</span><span class="p">:</span> <span class="p">[</span>
- <span class="s2">"myproviderpackage.hooks.source.SourceHook"</span><span class="p">,</span>
- <span class="p">],</span>
- <span class="p">}</span>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.exponential_delay">
+<em class="property"><span class="pre">static</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">exponential_delay</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">tries</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_client.html#BatchClientHook.exponential_delay"><span class="viewcode-link"><spa [...]
+<dd><p>An exponential back-off delay, with random jitter. There is a maximum
+interval of 10 minutes (with random jitter between 3 and 10 minutes).
+This is used in the <code class="xref py py-meth docutils literal notranslate"><span class="pre">poll_for_job_status()</span></code> method.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>tries</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a>) – Number of tries</p>
+</dd>
+</dl>
+<p>Examples of behavior:</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">exp</span><span class="p">(</span><span class="n">tries</span><span class="p">):</span>
+ <span class="n">max_interval</span> <span class="o">=</span> <span class="mf">600.0</span> <span class="c1"># 10 minutes in seconds</span>
+ <span class="n">delay</span> <span class="o">=</span> <span class="mi">1</span> <span class="o">+</span> <span class="nb">pow</span><span class="p">(</span><span class="n">tries</span> <span class="o">*</span> <span class="mf">0.6</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
+ <span class="n">delay</span> <span class="o">=</span> <span class="nb">min</span><span class="p">(</span><span class="n">max_interval</span><span class="p">,</span> <span class="n">delay</span><span class="p">)</span>
+ <span class="nb">print</span><span class="p">(</span><span class="n">delay</span> <span class="o">/</span> <span class="mi">3</span><span class="p">,</span> <span class="n">delay</span><span class="p">)</span>
+
+
+<span class="k">for</span> <span class="n">tries</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
+ <span class="n">exp</span><span class="p">(</span><span class="n">tries</span><span class="p">)</span>
+
+<span class="c1"># 0.33 1.0</span>
+<span class="c1"># 0.45 1.35</span>
+<span class="c1"># 0.81 2.44</span>
+<span class="c1"># 1.41 4.23</span>
+<span class="c1"># 2.25 6.76</span>
+<span class="c1"># 3.33 10.00</span>
+<span class="c1"># 4.65 13.95</span>
+<span class="c1"># 6.21 18.64</span>
+<span class="c1"># 8.01 24.04</span>
+<span class="c1"># 10.05 30.15</span>
</pre></div>
</div>
-<p><strong>How do provider packages work under the hood?</strong></p>
-<p>When running Airflow with your provider package, there will be (at least) three components to your airflow installation:</p>
-<ul class="simple">
-<li><p>The installation itself (for example, a <code class="docutils literal notranslate"><span class="pre">venv</span></code> where you installed airflow with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">apache-airflow</span></code>)
-together with the related files (e.g. <code class="docutils literal notranslate"><span class="pre">dags</span></code> folder)</p></li>
-<li><p>The <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package</p></li>
-<li><p>Your own <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package that is independent of <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> or your airflow installation, which
-can be a local Python package (that you install via <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span> <span class="pre">/path/to/my-package</span></code>), a normal pip package
-(<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">myproviderpackage</span></code>), or any other type of Python package</p></li>
-</ul>
-<p>In the <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package you need to add the entry point and provide the appropriate metadata as described above.
-If you have done that, airflow does the following at runtime:</p>
-<ul class="simple">
-<li><p>Loop through ALL packages installed in your environment / <code class="docutils literal notranslate"><span class="pre">venv</span></code></p></li>
-<li><p>For each package, if the package’s <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> has a section <code class="docutils literal notranslate"><span class="pre">[options.entry_points]</span></code>, and if that section has a value
-for <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code>, then get the value for <code class="docutils literal notranslate"><span class="pre">provider_info</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code></p></li>
-<li><p>That value works like an import statement: <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code> translates to something like
-<code class="docutils literal notranslate"><span class="pre">from</span> <span class="pre">myproviderpackage.somemodule</span> <span class="pre">import</span> <span class="pre">get_provider_info</span></code>, and the <code class="docutils literal notranslate"><span class="pre">get_provider_info</span></code> that is being imported should be a
-callable, i.e. a function</p></li>
-<li><p>This function should return a dictionary with metadata</p></li>
-<li><p>If you have custom connection types as part of your package, that metadata will including a field called <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>
-which should be a list of strings of your custom hooks - those strings should also be in an import-like format, e.g.
-<code class="docutils literal notranslate"><span class="pre">myproviderpackage.hooks.source.SourceHook</span></code> means that there is a class <code class="docutils literal notranslate"><span class="pre">SourceHook</span></code> in <code class="docutils literal notranslate"><span class="pre">myproviderpackage/hooks/source.py</span></code>
-- airflow then imports these hooks and looks for the functions <code class="docutils literal notranslate"><span class="pre">get_ui_field_behaviour</span></code> and <code class="docutils literal notranslate"><span class="pre">get_connection_form_widgets</span></code>
-(both optional) as well as the attributes <code class="docutils literal notranslate"><span class="pre">conn_type</span></code> and <code class="docutils literal notranslate"><span class="pre">hook_name</span></code> to create the custom connection type in the airflow UI</p></li>
-</ul>
-<p><strong>Should I name my provider specifically or should it be created in ``airflow.providers`` package?</strong></p>
-<p>We have quite a number (>60) of providers managed by the community and we are going to maintain them
-together with Apache Airflow. All those providers have well-defined structured and follow the
-naming conventions we defined and they are all in <code class="docutils literal notranslate"><span class="pre">airflow.providers</span></code> package. If your intention is
-to contribute your provider, then you should follow those conventions and make a PR to Apache Airflow
-to contribute to it. But you are free to use any package name as long as there are no conflicts with other
-names, so preferably choose package that is in your “domain”.</p>
-<p><strong>Is there a convention for a connection id and type?</strong></p>
-<p>Very good question. Glad that you asked. We usually follow the convention <code class="docutils literal notranslate"><span class="pre"><NAME>_default</span></code> for connection
-id and just <code class="docutils literal notranslate"><span class="pre"><NAME></span></code> for connection type. Few examples:</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">google_cloud_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">google_cloud_platform</span></code> type</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">aws_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">aws</span></code> type</p></li>
+<li><p><a class="reference external" href="https://docs.aws.amazon.com/general/latest/gr/api-retries.html">https://docs.aws.amazon.com/general/latest/gr/api-retries.html</a></p></li>
+<li><p><a class="reference external" href="https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/">https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/</a></p></li>
</ul>
-<p>You should follow this convention. It is important, to use unique names for connection type,
-so it should be unique for your provider. If two providers try to add connection with the same type
-only one of them will succeed.</p>
-<p><strong>Can I contribute my own provider to Apache Airflow?</strong></p>
-<p>Of course, but it’s better to check at developer’s mailing list whether such contribution will be accepted by
-the Community, before investing time to make the provider compliant with community requirements.
-The Community only accepts providers that are generic enough, are well documented, fully covered by tests
-and with capabilities of being tested by people in the community. So we might not always be in the
-position to accept such contributions.</p>
-<p>After you think that your provider matches the expected values above, you can read
-<a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a> to check all prerequisites for a new
-community Provider and discuss it at the <a class="reference external" href="http://airflow.apache.org/community/">Devlist</a>.</p>
-<p>However, in case you have your own, specific provider, which you can maintain on your own or by your
-team, you are free to publish the providers in whatever form you find appropriate. The custom and
-community-managed providers have exactly the same capabilities.</p>
-<p><strong>Can I advertise my own provider to Apache Airflow users and share it with others as package in PyPI?</strong></p>
-<p>Absolutely! We have an <a class="reference external" href="https://airflow.apache.org/ecosystem/">Ecosystem</a> area on our website where
-we share non-community managed extensions and work for Airflow. Feel free to make a PR to the page and
-add we will evaluate and merge it when we see that such provider can be useful for the community of
-Airflow users.</p>
-<p><strong>Can I charge for the use of my provider?</strong></p>
-<p>This is something that is outside of our control and domain. As an Apache project, we are
-commercial-friendly and there are many businesses built around Apache Airflow and many other
-Apache projects. As a community, we provide all the software for free and this will never
-change. What 3rd-party developers are doing is not under control of Apache Airflow community.</p>
</div>
-<div class="section" id="using-backport-providers-in-airflow-1-10">
-<h2>Using Backport Providers in Airflow 1.10<a class="headerlink" href="#using-backport-providers-in-airflow-1-10" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an Airflow version (1.10.12) running and it is stable. However, because of a Cloud provider change,
-I would like to upgrade the provider package. If I don’t need to upgrade the Airflow version anymore,
-how do I know that this provider version is compatible with my Airflow version?</strong></p>
-<p>We have Backport Providers are compatible with 1.10 but they stopped being released on
-March 17, 2021. Since then, no new changes to providers for Airflow 2.0 are going to be
-released as backport packages. It’s the highest time to upgrade to Airflow 2.0.</p>
-<p>When it comes to compatibility of providers with different Airflow 2 versions, each
-provider package will keep its own dependencies, and while we expect those providers to be generally
-backwards-compatible, particular versions of particular providers might introduce dependencies on
-specific Airflow versions.</p>
-<div class="toctree-wrapper compound">
+</dd></dl>
+
+</dd></dl>
+
</div>
</div>
</div>
- <div class="pager" role="navigation" aria-label="related navigation"><button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Previous</button>
- <a rel="next" title="Providers packages reference" href="packages-ref.html" >
+ <div class="pager" role="navigation" aria-label="related navigation">
+ <a rel="prev" title="airflow.providers.amazon.aws.hooks.base_aws" href="../base_aws/index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="p">Previous</button>
+ </a>
+ <a rel="next" title="airflow.providers.amazon.aws.hooks.batch_waiters" href="../batch_waiters/index.html" >
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="n">Next</button>
</a>
</div>
@@ -1044,28 +1217,49 @@ specific Airflow versions.</p>
<nav class="wy-nav-side-toc">
<div class="wy-menu-vertical">
<ul>
-<li><a class="reference internal" href="#">Provider packages</a></li>
-<li><a class="reference internal" href="#extending-airflow-core-functionality">Extending Airflow core functionality</a><ul>
-<li><a class="reference internal" href="#auth-backends">Auth backends</a></li>
-<li><a class="reference internal" href="#custom-connections">Custom connections</a></li>
-<li><a class="reference internal" href="#extra-links">Extra links</a></li>
-<li><a class="reference internal" href="#logging">Logging</a></li>
-<li><a class="reference internal" href="#secret-backends">Secret backends</a></li>
-<li><a class="reference internal" href="#notifications">Notifications</a></li>
+<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.batch_client</span></code></a><ul>
+<li><a class="reference internal" href="#module-contents">Module Contents</a><ul>
+<li><a class="reference internal" href="#classes">Classes</a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol"><code class="docutils literal notranslate"><span class="pre">BatchProtocol</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol.describe_jobs"><code class="docutils literal notranslate"><span class="pre">BatchProtocol.describe_jobs()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol.get_waiter"><code class="docutils literal notranslate"><span class="pre">BatchProtocol.get_waiter()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol.submit_job"><code class="docutils literal notranslate"><span class="pre">BatchProtocol.submit_job()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchProtocol.terminate_job"><code class="docutils literal notranslate"><span class="pre">BatchProtocol.terminate_job()</span></code></a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook"><code class="docutils literal notranslate"><span class="pre">BatchClientHook</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.client"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.client</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.MAX_RETRIES"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.MAX_RETRIES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.STATUS_RETRIES"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.STATUS_RETRIES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.DEFAULT_DELAY_MIN"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.DEFAULT_DELAY_MIN</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.DEFAULT_DELAY_MAX"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.DEFAULT_DELAY_MAX</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.FAILURE_STATE"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.FAILURE_STATE</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.SUCCESS_STATE"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.SUCCESS_STATE</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.RUNNING_STATE"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.RUNNING_STATE</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.INTERMEDIATE_STATES"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.INTERMEDIATE_STATES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.COMPUTE_ENVIRONMENT_TERMINAL_STATUS"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.COMPUTE_ENVIRONMENT_TERMINAL_STATUS</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.COMPUTE_ENVIRONMENT_INTERMEDIATE_STATUS"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.COMPUTE_ENVIRONMENT_INTERMEDIATE_STATUS</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.JOB_QUEUE_TERMINAL_STATUS"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.JOB_QUEUE_TERMINAL_STATUS</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.JOB_QUEUE_INTERMEDIATE_STATUS"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.JOB_QUEUE_INTERMEDIATE_STATUS</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.terminate_job"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.terminate_job()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.check_job_success"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.check_job_success()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.wait_for_job"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.wait_for_job()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.poll_for_job_running"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.poll_for_job_running()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.poll_for_job_complete"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.poll_for_job_complete()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.poll_job_status"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.poll_job_status()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.get_job_description"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.get_job_description()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.parse_job_description"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.parse_job_description()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.get_job_awslogs_info"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.get_job_awslogs_info()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.get_job_all_awslogs_info"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.get_job_all_awslogs_info()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.add_jitter"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.add_jitter()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.delay"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.delay()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook.exponential_delay"><code class="docutils literal notranslate"><span class="pre">BatchClientHook.exponential_delay()</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
-<li><a class="reference internal" href="#installing-and-upgrading-providers">Installing and upgrading providers</a></li>
-<li><a class="reference internal" href="#types-of-providers">Types of providers</a><ul>
-<li><a class="reference internal" href="#community-maintained-providers">Community maintained providers</a></li>
-<li><a class="reference internal" href="#custom-provider-packages">Custom provider packages</a></li>
</ul>
</li>
-<li><a class="reference internal" href="#how-to-create-your-own-provider">How to create your own provider</a></li>
-<li><a class="reference internal" href="#faq-for-airflow-and-providers">FAQ for Airflow and Providers</a><ul>
-<li><a class="reference internal" href="#upgrading-airflow-2-0-and-providers">Upgrading Airflow 2.0 and Providers</a></li>
-<li><a class="reference internal" href="#customizing-provider-packages">Customizing Provider Packages</a></li>
-<li><a class="reference internal" href="#creating-your-own-providers">Creating your own providers</a></li>
-<li><a class="reference internal" href="#using-backport-providers-in-airflow-1-10">Using Backport Providers in Airflow 1.10</a></li>
</ul>
</li>
</ul>
@@ -1090,7 +1284,7 @@ specific Airflow versions.</p>
<div class="base-layout--button">
- <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers/index.rst" rel="nofollow">
+ <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-amazon/_api/airflow/providers/amazon/aws/hooks/batch_client/index.rst" rel="nofollow">
<button class="btn-hollow btn-brown btn-with-icon with-box-shadow button-fixed">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
@@ -1224,19 +1418,19 @@ specific Airflow versions.</p>
</span>
</div>
</footer>
- <script type="text/javascript" src="_static/_gen/js/docs.js"></script>
- <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
+ <script type="text/javascript" src="../../../../../../../_static/_gen/js/docs.js"></script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../../../../" src="../../../../../../../_static/documentation_options.js"></script>
<script src="https://code.jquery.com/jquery-3.4.1.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js" integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy" crossorigin="anonymous"></script>
- <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
- <script src="_static/jquery.js"></script>
- <script src="_static/underscore.js"></script>
- <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
- <script src="_static/doctools.js"></script>
- <script src="_static/sphinx_highlight.js"></script>
- <script src="_static/clipboard.min.js"></script>
- <script src="_static/copybutton.js"></script>
- <script src="_static/js/globaltoc.js"></script>
+ <script data-url_root="../../../../../../../" id="documentation_options" src="../../../../../../../_static/documentation_options.js"></script>
+ <script src="../../../../../../../_static/jquery.js"></script>
+ <script src="../../../../../../../_static/underscore.js"></script>
+ <script src="../../../../../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
+ <script src="../../../../../../../_static/doctools.js"></script>
+ <script src="../../../../../../../_static/sphinx_highlight.js"></script>
+ <script src="../../../../../../../_static/clipboard.min.js"></script>
+ <script src="../../../../../../../_static/copybutton.js"></script>
+ <script src="../../../../../../../_static/js/globaltoc.js"></script>
</body>
</html>
\ No newline at end of file
diff --git a/docs-archive/apache-airflow-providers/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/batch_waiters/index.html
similarity index 60%
copy from docs-archive/apache-airflow-providers/index.html
copy to docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/batch_waiters/index.html
index 70eef65adf..f730300a33 100644
--- a/docs-archive/apache-airflow-providers/index.html
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/batch_waiters/index.html
@@ -9,18 +9,20 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Provider packages — apache-airflow-providers Documentation</title>
- <link rel="stylesheet" href="_static/_gen/css/main.min.css" type="text/css" />
- <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main.min.css" />
- <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main-custom.min.css" />
- <link rel="shortcut icon" href="_static/pin_32.png"/>
- <link rel="index" title="Index" href="genindex.html" />
- <link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Providers packages reference" href="packages-ref.html" />
-
- <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers/stable/index.html" />
+ <title>airflow.providers.amazon.aws.hooks.batch_waiters — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../../search.html" />
+ <link rel="next" title="airflow.providers.amazon.aws.hooks.cloud_formation" href="../cloud_formation/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.hooks.batch_client" href="../batch_client/index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/hooks/batch_waiters/index.html" />
<!-- Matomo -->
<script>
@@ -282,8 +284,21 @@
<div class="content-drawer">
<div class="td-sidebar">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -313,14 +328,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -401,8 +434,21 @@
<div class="td-sidebar desktop-only d-print-none">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -432,14 +478,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -528,9 +592,15 @@
<ul class="breadcrumb">
- <li class="breadcrumb-item"><a href="#" class="icon icon-home"> Home</a></li>
+ <li class="breadcrumb-item"><a href="../../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks</span></code></a></li>
- <li class="breadcrumb-item"><a href="index.html"> Provider packages</a></li>
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.batch_waiters</span></code></a></li>
</ul>
</div>
@@ -541,440 +611,231 @@
<div class="bodywrapper">
<div class="body" role="main">
- <blockquote>
-<div></div></blockquote>
-<div class="section" id="provider-packages">
-<h1>Provider packages<a class="headerlink" href="#provider-packages" title="Permalink to this heading">¶</a></h1>
-<p>Apache Airflow 2 is built in modular way. The “Core” of Apache Airflow provides core scheduler
-functionality which allow you to write some basic tasks, but the capabilities of Apache Airflow can
-be extended by installing additional packages, called <code class="docutils literal notranslate"><span class="pre">providers</span></code>.</p>
-<p>Providers can contain operators, hooks, sensor, and transfer operators to communicate with a
-multitude of external systems, but they can also extend Airflow core with new capabilities.</p>
-<p>You can install those provider packages separately in order to interface with a given service. The providers
-for <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> are designed in the way that you can write your own providers easily. The
-<code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span> <span class="pre">Community</span></code> develops and maintain more than 60 provider packages, but you are free to
-develop your own providers - the providers you build have exactly the same capability as the providers
-written by the community, so you can release and share those providers with others.</p>
-<p>The full list of community managed providers is available at
-<a class="reference external" href="https://airflow.apache.org/docs/#providers-packages-docs-apache-airflow-providers-index-html">Providers Index</a>.</p>
-<p>You can also see index of all community provider’s operators and hooks in
-<a class="reference internal" href="operators-and-hooks-ref/index.html"><span class="doc">Operators and Hooks Reference</span></a></p>
-</div>
-<div class="section" id="extending-airflow-core-functionality">
-<h1>Extending Airflow core functionality<a class="headerlink" href="#extending-airflow-core-functionality" title="Permalink to this heading">¶</a></h1>
-<p>Providers give you the capability of extending core Airflow with extra capabilities. The Core airflow
-provides basic and solid functionality of scheduling, the providers extend its capabilities. Here we
-describe all the custom capabilities.</p>
-<p>Airflow automatically discovers which providers add those additional capabilities and, once you install
-provider package and re-start Airflow, those become automatically available to Airflow Users.</p>
-<p>The summary of the core functionalities that can be extended are available in
-<a class="reference internal" href="core-extensions/index.html"><span class="doc">Core Extensions</span></a>.</p>
-<div class="section" id="auth-backends">
-<h2>Auth backends<a class="headerlink" href="#auth-backends" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom authentication backends, that allow you to configure the way how your
-web server authenticates your users, integrating it with public or private authentication services.</p>
-<p>You can see all the authentication backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/auth-backends.html"><span class="doc">Auth backends</span></a></p>
-</div>
-<div class="section" id="custom-connections">
-<h2>Custom connections<a class="headerlink" href="#custom-connections" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom connection types, extending connection form and handling custom form field
-behaviour for the connections defined by the provider.</p>
-<p>You can see all custom connections available via community-managed providers in
-<a class="reference internal" href="core-extensions/connections.html"><span class="doc">Connections</span></a>.</p>
-</div>
-<div class="section" id="extra-links">
-<h2>Extra links<a class="headerlink" href="#extra-links" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add extra custom links to operators delivered by the provider. Those will be visible in
-task details view of the task.</p>
-<p>You can see all the extra links available via community-managed providers in
-<a class="reference internal" href="core-extensions/extra-links.html"><span class="doc">Extra Links</span></a>.</p>
-</div>
-<div class="section" id="logging">
-<h2>Logging<a class="headerlink" href="#logging" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add additional task logging capabilities. By default <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> saves logs for
-tasks locally and make them available to Airflow UI via internal http server. However, providers
-can add extra logging capabilities, where Airflow Logs can be written to a remote service and
-retrieved from those services.</p>
-<p>You can see all task loggers available via community-managed providers in
-<a class="reference internal" href="core-extensions/logging.html"><span class="doc">Writing logs</span></a>.</p>
-</div>
-<div class="section" id="secret-backends">
-<h2>Secret backends<a class="headerlink" href="#secret-backends" title="Permalink to this heading">¶</a></h2>
-<p>Airflow has the capability of reading connections, variables and configuration from Secret Backends rather
-than from its own Database.</p>
-<p>You can see all secret backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/secrets-backends.html"><span class="doc">Secret backends</span></a>.</p>
-</div>
-<div class="section" id="notifications">
-<h2>Notifications<a class="headerlink" href="#notifications" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom notifications, that allow you to configure the way how you would like to receive
-notifications about the status of your tasks/DAGs.</p>
-<p>You can see all the notifications available via community-managed providers in
-<a class="reference internal" href="core-extensions/notifications.html"><span class="doc">Notifications</span></a>.</p>
-</div>
-</div>
-<div class="section" id="installing-and-upgrading-providers">
-<h1>Installing and upgrading providers<a class="headerlink" href="#installing-and-upgrading-providers" title="Permalink to this heading">¶</a></h1>
-<p>Separate provider packages give the possibilities that were not available in 1.10:</p>
-<ol class="arabic simple">
-<li><p>You can upgrade to latest version of particular providers without the need of Apache Airflow core upgrade.</p></li>
-<li><p>You can downgrade to previous version of particular provider in case the new version introduces
-some problems, without impacting the main Apache Airflow core package.</p></li>
-<li><p>You can release and upgrade/downgrade provider packages incrementally, independent from each other. This
-means that you can incrementally validate each of the provider package update in your environment,
-following the usual tests you have in your environment.</p></li>
-</ol>
-</div>
-<div class="section" id="types-of-providers">
-<h1>Types of providers<a class="headerlink" href="#types-of-providers" title="Permalink to this heading">¶</a></h1>
-<p>Providers have the same capacity - no matter if they are provided by the community or if they are
-third-party providers. This chapter explains how community managed providers are versioned and released
-and how you can create your own providers.</p>
-<div class="section" id="community-maintained-providers">
-<h2>Community maintained providers<a class="headerlink" href="#community-maintained-providers" title="Permalink to this heading">¶</a></h2>
-<p>From the point of view of the community, Airflow is delivered in multiple, separate packages.
-The core of Airflow scheduling system is delivered as <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package and there are more than
-60 provider packages which can be installed separately as so called <code class="docutils literal notranslate"><span class="pre">Airflow</span> <span class="pre">Provider</span> <span class="pre">packages</span></code>.
-Those packages are available as <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers</span></code> packages - for example there is an
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> or <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> package).</p>
-<p>Community maintained providers are released and versioned separately from the Airflow releases. We are
-following the <a class="reference external" href="https://semver.org/">Semver</a> versioning scheme for the packages. Some versions of the
-provider packages might depend on particular versions of Airflow, but the general approach we have is that
-unless there is a good reason, new version of providers should work with recent versions of Airflow 2.x.
-Details will vary per-provider and if there is a limitation for particular version of particular provider,
-constraining the Airflow version used, it will be included as limitation of dependencies in the provider
-package.</p>
-<p>Each community provider has corresponding extra which can be used when installing airflow to install the
-provider together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> - for example you can install airflow with those extras:
-<code class="docutils literal notranslate"><span class="pre">apache-airflow[google,amazon]</span></code> (with correct constraints -see <a class="reference external" href="/docs/apache-airflow/stable/installation/index.html" title="(in apache-airflow v2.6.0.dev0)"><span>Installation</span></a>) and you
-will install the appropriate versions of the <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> and
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> packages together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>.</p>
-<p>Some of the community providers have cross-provider dependencies as well. Those are not required
-dependencies, they might simply enable certain features (for example transfer operators often create
-dependency between different providers. Again, the general approach here is that the providers are backwards
-compatible, including cross-dependencies. Any kind of breaking changes and requirements on particular versions of other
-provider packages are automatically documented in the release notes of every provider.</p>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>For Airflow 1.10 We also provided <code class="docutils literal notranslate"><span class="pre">apache-airflow-backport-providers</span></code> packages that could be installed
-with those versions Those were the same providers as for 2.0 but automatically back-ported to work for
-Airflow 1.10. The last release of backport providers was done on March 17, 2021 and the backport
-providers will no longer be released, since Airflow 1.10 has reached End-Of-Life as of June 17, 2021.</p>
-</div>
-<p>If you want to contribute to <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>, you can see how to build and extend community
-managed providers in <a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a>.</p>
-</div>
-<div class="section" id="custom-provider-packages">
-<span id="providers-community-maintained-providers"></span><h2>Custom provider packages<a class="headerlink" href="#custom-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p>You can develop and release your own providers. Your custom operators, hooks, sensors, transfer operators
-can be packaged together in a standard airflow package and installed using the same mechanisms.
-Moreover they can also use the same mechanisms to extend the Airflow Core with auth backends,
-custom connections, logging, secret backends and extra operator links as described in the previous chapter.</p>
-</div>
-</div>
-<div class="section" id="how-to-create-your-own-provider">
-<h1>How to create your own provider<a class="headerlink" href="#how-to-create-your-own-provider" title="Permalink to this heading">¶</a></h1>
-<p>As mentioned in the <a class="reference external" href="http://airflow.apache.org/docs/apache-airflow-providers/index.html">Providers</a>
-documentation, custom providers can extend Airflow core - they can add extra links to operators as well
-as custom connections. You can use build your own providers and install them as packages if you would like
-to use the mechanism for your own, custom providers.</p>
-<p>Adding a provider to Airflow is just a matter of building a Python package and adding the right meta-data to
-the package. We are using standard mechanism of python to define
-<a class="reference external" href="https://docs.python.org/3/library/importlib.metadata.html#entry-points">entry points</a> . Your package
-needs to define appropriate entry-point <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> which has to point to a callable
-implemented by your package and return a dictionary containing the list of discoverable capabilities
-of your package. The dictionary has to follow the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json">json-schema specification</a>.</p>
-<p>Most of the schema provides extension point for the documentation (which you might want to also use for
-your own purpose) but the important fields from the extensibility point of view are those:</p>
-<p>Displaying package information in CLI/API:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">package-name</span></code> - Name of the package for the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code> - Human-friendly name of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">description</span></code> - Additional description of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">version</span></code> - List of versions of the package (in reverse-chronological order). The first version in the
-list is the current package version. It is taken from the version of package installed, not from the
-provider_info information.</p></li>
-</ul>
-<p>Exposing customized functionality to the Airflow’s core:</p>
+ <div class="section" id="module-airflow.providers.amazon.aws.hooks.batch_waiters">
+<span id="airflow-providers-amazon-aws-hooks-batch-waiters"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.hooks.batch_waiters" title="airflow.providers.amazon.aws.hooks.batch_waiters"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.batch_waiters</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.hooks.batch_waiters" title="Permalink to this heading">¶</a></h1>
+<p>AWS Batch service waiters</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">extra-links</span></code> - this field should contain the list of all operator class names that are adding extra links
-capability. See <a class="reference external" href="/docs/apache-airflow/stable/howto/define-extra-link.html" title="(in apache-airflow v2.6.0.dev0)"><span>Define an operator extra link</span></a> for description of how to add extra link
-capability to the operators of yours.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">connection-types</span></code> - this field should contain the list of all connection types together with hook
-class names implementing those custom connection types (providing custom extra fields and
-custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
-<code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> (deprecated) - this field should contain the list of all hook class names that provide
-custom connection types with custom extra fields and field behaviour. The <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array
-is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
-providers are targeting Airflow 2.2.0+ you do not have to include the <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array, if
-you want to also target earlier versions of Airflow 2, you should include both <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> and
-<code class="docutils literal notranslate"><span class="pre">connection-types</span></code> arrays. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details.</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/clients.html#waiters">https://boto3.amazonaws.com/v1/documentation/api/latest/guide/clients.html#waiters</a></p></li>
+<li><p><a class="reference external" href="https://github.com/boto/botocore/blob/develop/botocore/waiter.py">https://github.com/boto/botocore/blob/develop/botocore/waiter.py</a></p></li>
</ul>
-<p>When your providers are installed you can query the installed providers and their capabilities with the
-<code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">providers</span></code> command. This way you can verify if your providers are properly recognized and whether
-they define the extensions properly. See <a class="reference external" href="/docs/apache-airflow/stable/cli-and-env-variables-ref.html" title="(in apache-airflow v2.6.0.dev0)"><span>Command Line Interface and Environment Variables Reference</span></a> for details of available CLI
-sub-commands.</p>
-<p>When you write your own provider, consider following the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#naming-conventions-for-provider-packages">Naming conventions for provider packages</a></p>
-</div>
-<div class="section" id="faq-for-airflow-and-providers">
-<h1>FAQ for Airflow and Providers<a class="headerlink" href="#faq-for-airflow-and-providers" title="Permalink to this heading">¶</a></h1>
-<div class="section" id="upgrading-airflow-2-0-and-providers">
-<h2>Upgrading Airflow 2.0 and Providers<a class="headerlink" href="#upgrading-airflow-2-0-and-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When upgrading to a new Airflow version such as 2.0, but possibly 2.0.1 and beyond, is the best practice
-to also upgrade provider packages at the same time?</strong></p>
-<p>It depends on your use case. If you have automated or semi-automated verification of your installation,
-that you can run a new version of Airflow including all provider packages, then definitely go for it.
-If you rely more on manual testing, it is advised that you upgrade in stages. Depending on your choice
-you can either upgrade all used provider packages first, and then upgrade Airflow Core or the other way
-round. The first approach - when you first upgrade all providers is probably safer, as you can do it
-incrementally, step-by-step replacing provider by provider in your environment.</p>
</div>
-<div class="section" id="customizing-provider-packages">
-<h2>Customizing Provider Packages<a class="headerlink" href="#customizing-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an older version of my provider package which we have lightly customized and is working
-fine with my MSSQL installation. I am upgrading my Airflow version. Do I need to upgrade my provider,
-or can I keep it as it is?</strong></p>
-<p>It depends on the scope of customization. There is no need to upgrade the provider packages to later
-versions unless you want to upgrade to Airflow version that introduces backwards-incompatible changes.
-Generally speaking, with Airflow 2 we are following the <a class="reference external" href="https://semver.org/">Semver</a> approach where
-we will introduce backwards-incompatible changes in Major releases, so all your modifications (as long
-as you have not used internal Airflow classes) should work for All Airflow 2.* versions.</p>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<div class="section" id="classes">
+<h3>Classes<a class="headerlink" href="#classes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook" title="airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">BatchWaitersHook</span></code></a></p></td>
+<td><p>A utility to manage waiters for AWS Batch services.</p></td>
+</tr>
+</tbody>
+</table>
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.batch_waiters.</span></span><span class="sig-name descname"><span class="pre">BatchWaitersHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="n"><span class [...]
+<dd><p>Bases: <a class="reference internal" href="../batch_client/index.html#airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook" title="airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.batch_client.BatchClientHook</span></code></a></p>
+<p>A utility to manage waiters for AWS Batch services.</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">random</span>
+<span class="kn">from</span> <span class="nn">airflow.providers.amazon.aws.operators.batch_waiters</span> <span class="kn">import</span> <span class="n">BatchWaiters</span>
+
+<span class="c1"># to inspect default waiters</span>
+<span class="n">waiters</span> <span class="o">=</span> <span class="n">BatchWaiters</span><span class="p">()</span>
+<span class="n">config</span> <span class="o">=</span> <span class="n">waiters</span><span class="o">.</span><span class="n">default_config</span> <span class="c1"># type: Dict</span>
+<span class="n">waiter_names</span> <span class="o">=</span> <span class="n">waiters</span><span class="o">.</span><span class="n">list_waiters</span><span class="p">()</span> <span class="c1"># -> ["JobComplete", "JobExists", "JobRunning"]</span>
+
+<span class="c1"># The default_config is a useful stepping stone to creating custom waiters, e.g.</span>
+<span class="n">custom_config</span> <span class="o">=</span> <span class="n">waiters</span><span class="o">.</span><span class="n">default_config</span> <span class="c1"># this is a deepcopy</span>
+<span class="c1"># modify custom_config['waiters'] as necessary and get a new instance:</span>
+<span class="n">waiters</span> <span class="o">=</span> <span class="n">BatchWaiters</span><span class="p">(</span><span class="n">waiter_config</span><span class="o">=</span><span class="n">custom_config</span><span class="p">)</span>
+<span class="n">waiters</span><span class="o">.</span><span class="n">waiter_config</span> <span class="c1"># check the custom configuration (this is a deepcopy)</span>
+<span class="n">waiters</span><span class="o">.</span><span class="n">list_waiters</span><span class="p">()</span> <span class="c1"># names of custom waiters</span>
+
+<span class="c1"># During the init for BatchWaiters, the waiter_config is used to build a waiter_model;</span>
+<span class="c1"># and note that this only occurs during the class init, to avoid any accidental mutations</span>
+<span class="c1"># of waiter_config leaking into the waiter_model.</span>
+<span class="n">waiters</span><span class="o">.</span><span class="n">waiter_model</span> <span class="c1"># -> botocore.waiter.WaiterModel object</span>
+
+<span class="c1"># The waiter_model is combined with the waiters.client to get a specific waiter</span>
+<span class="c1"># and the details of the config on that waiter can be further modified without any</span>
+<span class="c1"># accidental impact on the generation of new waiters from the defined waiter_model, e.g.</span>
+<span class="n">waiters</span><span class="o">.</span><span class="n">get_waiter</span><span class="p">(</span><span class="s2">"JobExists"</span><span class="p">)</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">delay</span> <span class="c1"># -> 5</span>
+<span class="n">waiter</span> <span class="o">=</span> <span class="n">waiters</span><span class="o">.</span><span class="n">get_waiter</span><span class="p">(</span><span class="s2">"JobExists"</span><span class="p">)</span> <span class="c1"># -> botocore.waiter.Batch.Waiter.JobExists object</span>
+<span class="n">waiter</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">delay</span> <span class="o">=</span> <span class="mi">10</span>
+<span class="n">waiters</span><span class="o">.</span><span class="n">get_waiter</span><span class="p">(</span><span class="s2">"JobExists"</span><span class="p">)</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">delay</span> <span class="c1"># -> 5 as defined by waiter_model</span>
+
+<span class="c1"># To use a specific waiter, update the config and call the `wait()` method for jobId, e.g.</span>
+<span class="n">waiter</span> <span class="o">=</span> <span class="n">waiters</span><span class="o">.</span><span class="n">get_waiter</span><span class="p">(</span><span class="s2">"JobExists"</span><span class="p">)</span> <span class="c1"># -> botocore.waiter.Batch.Waiter.JobExists object</span>
+<span class="n">waiter</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">delay</span> <span class="o">=</span> <span class="n">random</span><span class="o">.</span><span class="n">uniform</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="c1"># seconds</span>
+<span class="n">waiter</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">max_attempts</span> <span class="o">=</span> <span class="mi">10</span>
+<span class="n">waiter</span><span class="o">.</span><span class="n">wait</span><span class="p">(</span><span class="n">jobs</span><span class="o">=</span><span class="p">[</span><span class="n">jobId</span><span class="p">])</span>
+</pre></div>
</div>
-<div class="section" id="creating-your-own-providers">
-<h2>Creating your own providers<a class="headerlink" href="#creating-your-own-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When I write my own provider, do I need to do anything special to make it available to others?</strong></p>
-<p>You do not need to do anything special besides creating the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point
-returning properly formatted meta-data - dictionary with <code class="docutils literal notranslate"><span class="pre">extra-links</span></code> and <code class="docutils literal notranslate"><span class="pre">connection-types</span></code> fields
-(and deprecated <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> field if you are also targeting versions of Airflow before 2.2.0).</p>
-<p>Anyone who runs airflow in an environment that has your Python package installed will be able to use the
-package as a provider package.</p>
-<p><strong>What do I need to do to turn a package into a provider?</strong></p>
-<p>You need to do the following to turn an existing Python package into a provider (see below for examples):</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Add the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point in the <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> - this tells airflow where to get
-the required provider metadata</p></li>
-<li><p>Create the function that you refer to in the first step as part of your package: this functions returns a
-dictionary that contains all meta-data about your provider package</p></li>
-<li><p>If you want Airflow to link to documentation of your Provider in the providers page, make sure
-to add “project-url/documentation” <a class="reference external" href="https://peps.python.org/pep-0621/#example">metadata</a> to your package.
-This will also add link to your documentation in PyPI.</p></li>
-<li><p>note that the dictionary should be compliant with <code class="docutils literal notranslate"><span class="pre">airflow/provider_info.schema.json</span></code> JSON-schema
-specification. The community-managed providers have more fields there that are used to build
-documentation, but the requirement for runtime information only contains several fields which are defined
-in the schema:</p></li>
+<li><p><a class="reference external" href="https://www.2ndwatch.com/blog/use-waiters-boto3-write/">https://www.2ndwatch.com/blog/use-waiters-boto3-write/</a></p></li>
+<li><p><a class="reference external" href="https://github.com/boto/botocore/blob/develop/botocore/waiter.py">https://github.com/boto/botocore/blob/develop/botocore/waiter.py</a></p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#waiters">https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#waiters</a></p></li>
+<li><p><a class="reference external" href="https://github.com/boto/botocore/tree/develop/botocore/data/ec2/2016-11-15">https://github.com/boto/botocore/tree/develop/botocore/data/ec2/2016-11-15</a></p></li>
+<li><p><a class="reference external" href="https://github.com/boto/botocore/issues/1915">https://github.com/boto/botocore/issues/1915</a></p></li>
</ul>
-<div class="example-block-wrapper compound">
-<p class="example-header compound-first"><span class="example-title">airflow/provider_info.schema.json</span></p>
-<div class="compound-last highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"$schema"</span><span class="p">:</span><span class="w"> </span><span class="s2">"http://json-schema.org/draft-07/schema#"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"package-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Package name available under which the package is available in the PyPI repository."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Provider name"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Information about the package in RST format"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-names"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class names that provide connection types to core (deprecated by connection-types)"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"deprecated"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"The hook-class-names property has been deprecated in favour of connection-types which is more performant version allowing to only import individual Hooks rather than all hooks at once"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"deprecatedVersion"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.2.0"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"connection-types"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Map of connection types mapped to hook class names."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"connection-type"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Type of connection defined by the provider"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class name that implements the connection type"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"connection-type"</span><span class="p">,</span><span class="w"> </span><span class="s2">"hook-class-name"</span><span class="p">]</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"extra-links"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Operator class names that provide extra link functionality"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"secrets-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Secrets Backend class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"logging"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Logging Task Handlers class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"auth-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"API Auth Backend module names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"notifications"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Notification class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"task-decorators"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Apply custom decorators to the TaskFlow API. Can be accessed by users via '@task.<name>'"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"path"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
-<span class="w"> </span><span class="s2">"name"</span><span class="p">,</span>
-<span class="w"> </span><span class="s2">"description"</span>
-<span class="w"> </span><span class="p">]</span>
-<span class="p">}</span>
-</pre></div>
-</div>
</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>:</p>
-<div class="highlight-cfg notranslate"><div class="highlight"><pre><span></span><span class="k">[options.entry_points]</span>
-<span class="c1"># the function get_provider_info is defined in myproviderpackage.somemodule</span>
-<span class="na">apache_airflow_provider</span><span class="o">=</span>
-<span class="w"> </span><span class="na">provider_info</span><span class="o">=</span><span class="s">myproviderpackage.somemodule:get_provider_info</span>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>waiter_config</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a><em> | </em><em>None</em>) – a custom waiter configuration for AWS Batch services</p></li>
+<li><p><strong>aws_conn_id</strong> – connection id of AWS credentials / region name. If None,
+credential boto3 strategy will be used
+(<a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html">https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html</a>).</p></li>
+<li><p><strong>region_name</strong> – region name to use in AWS client.
+Override the AWS region in connection (if provided)</p></li>
+</ul>
+</dd>
+</dl>
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.default_config">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">default_config</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><span class="pre">dict</span></a></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers [...]
+<dd><p>An immutable default waiter configuration</p>
+<dl class="field-list simple">
+<dt class="field-odd">Returns</dt>
+<dd class="field-odd"><p>a waiter configuration for AWS Batch services</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.waiter_config">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">waiter_config</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><span class="pre">dict</span></a></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/ [...]
+<dd><p>An immutable waiter configuration for this instance; a <code class="docutils literal notranslate"><span class="pre">deepcopy</span></code> is returned by this
+property. During the init for BatchWaiters, the waiter_config is used to build a
+waiter_model and this only occurs during the class init, to avoid any accidental
+mutations of waiter_config leaking into the waiter_model.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Returns</dt>
+<dd class="field-odd"><p>a waiter configuration for AWS Batch services</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.waiter_model">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">waiter_model</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">botocore.waiter.WaiterModel</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_waiters.html#BatchWaitersHook.waiter_model"><span class="viewcode-link"><sp [...]
+<dd><p>A configured waiter model used to generate waiters on AWS Batch services.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Returns</dt>
+<dd class="field-odd"><p>a waiter model for AWS Batch services</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p>botocore.waiter.WaiterModel</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.get_waiter">
+<span class="sig-name descname"><span class="pre">get_waiter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">waiter_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">_</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">deferrable</span></span><span class="o"><span class [...]
+<dd><p>Get an AWS Batch service waiter, using the configured <code class="docutils literal notranslate"><span class="pre">.waiter_model</span></code>.</p>
+<p>The <code class="docutils literal notranslate"><span class="pre">.waiter_model</span></code> is combined with the <code class="docutils literal notranslate"><span class="pre">.client</span></code> to get a specific waiter and
+the properties of that waiter can be modified without any accidental impact on the
+generation of new waiters from the <code class="docutils literal notranslate"><span class="pre">.waiter_model</span></code>, e.g.</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">waiters</span><span class="o">.</span><span class="n">get_waiter</span><span class="p">(</span><span class="s2">"JobExists"</span><span class="p">)</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">delay</span> <span class="c1"># -> 5</span>
+<span class="n">waiter</span> <span class="o">=</span> <span class="n">waiters</span><span class="o">.</span><span class="n">get_waiter</span><span class="p">(</span><span class="s2">"JobExists"</span><span class="p">)</span> <span class="c1"># a new waiter object</span>
+<span class="n">waiter</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">delay</span> <span class="o">=</span> <span class="mi">10</span>
+<span class="n">waiters</span><span class="o">.</span><span class="n">get_waiter</span><span class="p">(</span><span class="s2">"JobExists"</span><span class="p">)</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">delay</span> <span class="c1"># -> 5 as defined by waiter_model</span>
</pre></div>
</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">myproviderpackage/somemodule.py</span></code>:</p>
-<div class="highlight-Python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_provider_info</span><span class="p">():</span>
- <span class="k">return</span> <span class="p">{</span>
- <span class="s2">"package-name"</span><span class="p">:</span> <span class="s2">"my-package-name"</span><span class="p">,</span>
- <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"name"</span><span class="p">,</span>
- <span class="s2">"description"</span><span class="p">:</span> <span class="s2">"a description"</span><span class="p">,</span>
- <span class="s2">"hook-class-names"</span><span class="p">:</span> <span class="p">[</span>
- <span class="s2">"myproviderpackage.hooks.source.SourceHook"</span><span class="p">,</span>
- <span class="p">],</span>
- <span class="p">}</span>
+<p>To use a specific waiter, update the config and call the <cite>wait()</cite> method for jobId, e.g.</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">random</span>
+
+<span class="n">waiter</span> <span class="o">=</span> <span class="n">waiters</span><span class="o">.</span><span class="n">get_waiter</span><span class="p">(</span><span class="s2">"JobExists"</span><span class="p">)</span> <span class="c1"># a new waiter object</span>
+<span class="n">waiter</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">delay</span> <span class="o">=</span> <span class="n">random</span><span class="o">.</span><span class="n">uniform</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="c1"># seconds</span>
+<span class="n">waiter</span><span class="o">.</span><span class="n">config</span><span class="o">.</span><span class="n">max_attempts</span> <span class="o">=</span> <span class="mi">10</span>
+<span class="n">waiter</span><span class="o">.</span><span class="n">wait</span><span class="p">(</span><span class="n">jobs</span><span class="o">=</span><span class="p">[</span><span class="n">jobId</span><span class="p">])</span>
</pre></div>
</div>
-<p><strong>How do provider packages work under the hood?</strong></p>
-<p>When running Airflow with your provider package, there will be (at least) three components to your airflow installation:</p>
-<ul class="simple">
-<li><p>The installation itself (for example, a <code class="docutils literal notranslate"><span class="pre">venv</span></code> where you installed airflow with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">apache-airflow</span></code>)
-together with the related files (e.g. <code class="docutils literal notranslate"><span class="pre">dags</span></code> folder)</p></li>
-<li><p>The <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package</p></li>
-<li><p>Your own <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package that is independent of <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> or your airflow installation, which
-can be a local Python package (that you install via <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span> <span class="pre">/path/to/my-package</span></code>), a normal pip package
-(<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">myproviderpackage</span></code>), or any other type of Python package</p></li>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>waiter_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – The name of the waiter. The name should match
+the name (including the casing) of the key name in the waiter
+model file (typically this is CamelCasing); see <code class="docutils literal notranslate"><span class="pre">.list_waiters</span></code>.</p></li>
+<li><p><strong>_</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a><em>[</em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em>, </em><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a><em>] </em><em>| </em><em>None</em>) – unused, ju [...]
</ul>
-<p>In the <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package you need to add the entry point and provide the appropriate metadata as described above.
-If you have done that, airflow does the following at runtime:</p>
-<ul class="simple">
-<li><p>Loop through ALL packages installed in your environment / <code class="docutils literal notranslate"><span class="pre">venv</span></code></p></li>
-<li><p>For each package, if the package’s <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> has a section <code class="docutils literal notranslate"><span class="pre">[options.entry_points]</span></code>, and if that section has a value
-for <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code>, then get the value for <code class="docutils literal notranslate"><span class="pre">provider_info</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code></p></li>
-<li><p>That value works like an import statement: <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code> translates to something like
-<code class="docutils literal notranslate"><span class="pre">from</span> <span class="pre">myproviderpackage.somemodule</span> <span class="pre">import</span> <span class="pre">get_provider_info</span></code>, and the <code class="docutils literal notranslate"><span class="pre">get_provider_info</span></code> that is being imported should be a
-callable, i.e. a function</p></li>
-<li><p>This function should return a dictionary with metadata</p></li>
-<li><p>If you have custom connection types as part of your package, that metadata will including a field called <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>
-which should be a list of strings of your custom hooks - those strings should also be in an import-like format, e.g.
-<code class="docutils literal notranslate"><span class="pre">myproviderpackage.hooks.source.SourceHook</span></code> means that there is a class <code class="docutils literal notranslate"><span class="pre">SourceHook</span></code> in <code class="docutils literal notranslate"><span class="pre">myproviderpackage/hooks/source.py</span></code>
-- airflow then imports these hooks and looks for the functions <code class="docutils literal notranslate"><span class="pre">get_ui_field_behaviour</span></code> and <code class="docutils literal notranslate"><span class="pre">get_connection_form_widgets</span></code>
-(both optional) as well as the attributes <code class="docutils literal notranslate"><span class="pre">conn_type</span></code> and <code class="docutils literal notranslate"><span class="pre">hook_name</span></code> to create the custom connection type in the airflow UI</p></li>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>a waiter object for the named AWS Batch service</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p>botocore.waiter.Waiter</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.list_waiters">
+<span class="sig-name descname"><span class="pre">list_waiters</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/batch_waiters.html#BatchWaitersHook.list_waiters"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.list_waiters" title="Permalink to [...]
+<dd><p>List the waiters in a waiter configuration for AWS Batch services.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Returns</dt>
+<dd class="field-odd"><p>waiter names for AWS Batch services</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#list" title="(in Python v3.11)">list</a>[<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a>]</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.wait_for_job">
+<span class="sig-name descname"><span class="pre">wait_for_job</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">job_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">delay</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/pr [...]
+<dd><p>Wait for Batch job to complete. This assumes that the <code class="docutils literal notranslate"><span class="pre">.waiter_model</span></code> is configured
+using some variation of the <code class="docutils literal notranslate"><span class="pre">.default_config</span></code> so that it can generate waiters with the
+following names: “JobExists”, “JobRunning” and “JobComplete”.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>job_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – a Batch job ID</p></li>
+<li><p><strong>delay</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a><em> | </em><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.11)"><em>float</em></a><em> | </em><em>None</em>) – A delay before polling for job status</p></li>
</ul>
-<p><strong>Should I name my provider specifically or should it be created in ``airflow.providers`` package?</strong></p>
-<p>We have quite a number (>60) of providers managed by the community and we are going to maintain them
-together with Apache Airflow. All those providers have well-defined structured and follow the
-naming conventions we defined and they are all in <code class="docutils literal notranslate"><span class="pre">airflow.providers</span></code> package. If your intention is
-to contribute your provider, then you should follow those conventions and make a PR to Apache Airflow
-to contribute to it. But you are free to use any package name as long as there are no conflicts with other
-names, so preferably choose package that is in your “domain”.</p>
-<p><strong>Is there a convention for a connection id and type?</strong></p>
-<p>Very good question. Glad that you asked. We usually follow the convention <code class="docutils literal notranslate"><span class="pre"><NAME>_default</span></code> for connection
-id and just <code class="docutils literal notranslate"><span class="pre"><NAME></span></code> for connection type. Few examples:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">google_cloud_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">google_cloud_platform</span></code> type</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">aws_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">aws</span></code> type</p></li>
-</ul>
-<p>You should follow this convention. It is important, to use unique names for connection type,
-so it should be unique for your provider. If two providers try to add connection with the same type
-only one of them will succeed.</p>
-<p><strong>Can I contribute my own provider to Apache Airflow?</strong></p>
-<p>Of course, but it’s better to check at developer’s mailing list whether such contribution will be accepted by
-the Community, before investing time to make the provider compliant with community requirements.
-The Community only accepts providers that are generic enough, are well documented, fully covered by tests
-and with capabilities of being tested by people in the community. So we might not always be in the
-position to accept such contributions.</p>
-<p>After you think that your provider matches the expected values above, you can read
-<a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a> to check all prerequisites for a new
-community Provider and discuss it at the <a class="reference external" href="http://airflow.apache.org/community/">Devlist</a>.</p>
-<p>However, in case you have your own, specific provider, which you can maintain on your own or by your
-team, you are free to publish the providers in whatever form you find appropriate. The custom and
-community-managed providers have exactly the same capabilities.</p>
-<p><strong>Can I advertise my own provider to Apache Airflow users and share it with others as package in PyPI?</strong></p>
-<p>Absolutely! We have an <a class="reference external" href="https://airflow.apache.org/ecosystem/">Ecosystem</a> area on our website where
-we share non-community managed extensions and work for Airflow. Feel free to make a PR to the page and
-add we will evaluate and merge it when we see that such provider can be useful for the community of
-Airflow users.</p>
-<p><strong>Can I charge for the use of my provider?</strong></p>
-<p>This is something that is outside of our control and domain. As an Apache project, we are
-commercial-friendly and there are many businesses built around Apache Airflow and many other
-Apache projects. As a community, we provide all the software for free and this will never
-change. What 3rd-party developers are doing is not under control of Apache Airflow community.</p>
+</dd>
+<dt class="field-even">Raises</dt>
+<dd class="field-even"><p>AirflowException</p>
+</dd>
+</dl>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>This method adds a small random jitter to the <code class="docutils literal notranslate"><span class="pre">delay</span></code> (+/- 2 sec, >= 1 sec).
+Using a random interval helps to avoid AWS API throttle limits when many
+concurrent tasks request job-descriptions.</p>
+<p>It also modifies the <code class="docutils literal notranslate"><span class="pre">max_attempts</span></code> to use the <code class="docutils literal notranslate"><span class="pre">sys.maxsize</span></code>,
+which allows Airflow to manage the timeout on waiting.</p>
</div>
-<div class="section" id="using-backport-providers-in-airflow-1-10">
-<h2>Using Backport Providers in Airflow 1.10<a class="headerlink" href="#using-backport-providers-in-airflow-1-10" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an Airflow version (1.10.12) running and it is stable. However, because of a Cloud provider change,
-I would like to upgrade the provider package. If I don’t need to upgrade the Airflow version anymore,
-how do I know that this provider version is compatible with my Airflow version?</strong></p>
-<p>We have Backport Providers are compatible with 1.10 but they stopped being released on
-March 17, 2021. Since then, no new changes to providers for Airflow 2.0 are going to be
-released as backport packages. It’s the highest time to upgrade to Airflow 2.0.</p>
-<p>When it comes to compatibility of providers with different Airflow 2 versions, each
-provider package will keep its own dependencies, and while we expect those providers to be generally
-backwards-compatible, particular versions of particular providers might introduce dependencies on
-specific Airflow versions.</p>
-<div class="toctree-wrapper compound">
+</dd></dl>
+
+</dd></dl>
+
</div>
</div>
</div>
- <div class="pager" role="navigation" aria-label="related navigation"><button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Previous</button>
- <a rel="next" title="Providers packages reference" href="packages-ref.html" >
+ <div class="pager" role="navigation" aria-label="related navigation">
+ <a rel="prev" title="airflow.providers.amazon.aws.hooks.batch_client" href="../batch_client/index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="p">Previous</button>
+ </a>
+ <a rel="next" title="airflow.providers.amazon.aws.hooks.cloud_formation" href="../cloud_formation/index.html" >
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="n">Next</button>
</a>
</div>
@@ -1044,28 +905,22 @@ specific Airflow versions.</p>
<nav class="wy-nav-side-toc">
<div class="wy-menu-vertical">
<ul>
-<li><a class="reference internal" href="#">Provider packages</a></li>
-<li><a class="reference internal" href="#extending-airflow-core-functionality">Extending Airflow core functionality</a><ul>
-<li><a class="reference internal" href="#auth-backends">Auth backends</a></li>
-<li><a class="reference internal" href="#custom-connections">Custom connections</a></li>
-<li><a class="reference internal" href="#extra-links">Extra links</a></li>
-<li><a class="reference internal" href="#logging">Logging</a></li>
-<li><a class="reference internal" href="#secret-backends">Secret backends</a></li>
-<li><a class="reference internal" href="#notifications">Notifications</a></li>
+<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.batch_waiters</span></code></a><ul>
+<li><a class="reference internal" href="#module-contents">Module Contents</a><ul>
+<li><a class="reference internal" href="#classes">Classes</a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook"><code class="docutils literal notranslate"><span class="pre">BatchWaitersHook</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.default_config"><code class="docutils literal notranslate"><span class="pre">BatchWaitersHook.default_config</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.waiter_config"><code class="docutils literal notranslate"><span class="pre">BatchWaitersHook.waiter_config</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.waiter_model"><code class="docutils literal notranslate"><span class="pre">BatchWaitersHook.waiter_model</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.get_waiter"><code class="docutils literal notranslate"><span class="pre">BatchWaitersHook.get_waiter()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.list_waiters"><code class="docutils literal notranslate"><span class="pre">BatchWaitersHook.list_waiters()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.batch_waiters.BatchWaitersHook.wait_for_job"><code class="docutils literal notranslate"><span class="pre">BatchWaitersHook.wait_for_job()</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
-<li><a class="reference internal" href="#installing-and-upgrading-providers">Installing and upgrading providers</a></li>
-<li><a class="reference internal" href="#types-of-providers">Types of providers</a><ul>
-<li><a class="reference internal" href="#community-maintained-providers">Community maintained providers</a></li>
-<li><a class="reference internal" href="#custom-provider-packages">Custom provider packages</a></li>
</ul>
</li>
-<li><a class="reference internal" href="#how-to-create-your-own-provider">How to create your own provider</a></li>
-<li><a class="reference internal" href="#faq-for-airflow-and-providers">FAQ for Airflow and Providers</a><ul>
-<li><a class="reference internal" href="#upgrading-airflow-2-0-and-providers">Upgrading Airflow 2.0 and Providers</a></li>
-<li><a class="reference internal" href="#customizing-provider-packages">Customizing Provider Packages</a></li>
-<li><a class="reference internal" href="#creating-your-own-providers">Creating your own providers</a></li>
-<li><a class="reference internal" href="#using-backport-providers-in-airflow-1-10">Using Backport Providers in Airflow 1.10</a></li>
</ul>
</li>
</ul>
@@ -1090,7 +945,7 @@ specific Airflow versions.</p>
<div class="base-layout--button">
- <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers/index.rst" rel="nofollow">
+ <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-amazon/_api/airflow/providers/amazon/aws/hooks/batch_waiters/index.rst" rel="nofollow">
<button class="btn-hollow btn-brown btn-with-icon with-box-shadow button-fixed">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
@@ -1224,19 +1079,19 @@ specific Airflow versions.</p>
</span>
</div>
</footer>
- <script type="text/javascript" src="_static/_gen/js/docs.js"></script>
- <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
+ <script type="text/javascript" src="../../../../../../../_static/_gen/js/docs.js"></script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../../../../" src="../../../../../../../_static/documentation_options.js"></script>
<script src="https://code.jquery.com/jquery-3.4.1.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js" integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy" crossorigin="anonymous"></script>
- <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
- <script src="_static/jquery.js"></script>
- <script src="_static/underscore.js"></script>
- <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
- <script src="_static/doctools.js"></script>
- <script src="_static/sphinx_highlight.js"></script>
- <script src="_static/clipboard.min.js"></script>
- <script src="_static/copybutton.js"></script>
- <script src="_static/js/globaltoc.js"></script>
+ <script data-url_root="../../../../../../../" id="documentation_options" src="../../../../../../../_static/documentation_options.js"></script>
+ <script src="../../../../../../../_static/jquery.js"></script>
+ <script src="../../../../../../../_static/underscore.js"></script>
+ <script src="../../../../../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
+ <script src="../../../../../../../_static/doctools.js"></script>
+ <script src="../../../../../../../_static/sphinx_highlight.js"></script>
+ <script src="../../../../../../../_static/clipboard.min.js"></script>
+ <script src="../../../../../../../_static/copybutton.js"></script>
+ <script src="../../../../../../../_static/js/globaltoc.js"></script>
</body>
</html>
\ No newline at end of file
diff --git a/docs-archive/apache-airflow-providers/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/cloud_formation/index.html
similarity index 60%
copy from docs-archive/apache-airflow-providers/index.html
copy to docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/cloud_formation/index.html
index 70eef65adf..2eb43eef5b 100644
--- a/docs-archive/apache-airflow-providers/index.html
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/cloud_formation/index.html
@@ -9,18 +9,20 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Provider packages — apache-airflow-providers Documentation</title>
- <link rel="stylesheet" href="_static/_gen/css/main.min.css" type="text/css" />
- <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main.min.css" />
- <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main-custom.min.css" />
- <link rel="shortcut icon" href="_static/pin_32.png"/>
- <link rel="index" title="Index" href="genindex.html" />
- <link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Providers packages reference" href="packages-ref.html" />
-
- <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers/stable/index.html" />
+ <title>airflow.providers.amazon.aws.hooks.cloud_formation — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../../search.html" />
+ <link rel="next" title="airflow.providers.amazon.aws.hooks.datasync" href="../datasync/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.hooks.batch_waiters" href="../batch_waiters/index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/hooks/cloud_formation/index.html" />
<!-- Matomo -->
<script>
@@ -282,8 +284,21 @@
<div class="content-drawer">
<div class="td-sidebar">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -313,14 +328,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -401,8 +434,21 @@
<div class="td-sidebar desktop-only d-print-none">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -432,14 +478,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -528,9 +592,15 @@
<ul class="breadcrumb">
- <li class="breadcrumb-item"><a href="#" class="icon icon-home"> Home</a></li>
+ <li class="breadcrumb-item"><a href="../../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
- <li class="breadcrumb-item"><a href="index.html"> Provider packages</a></li>
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.cloud_formation</span></code></a></li>
</ul>
</div>
@@ -541,440 +611,105 @@
<div class="bodywrapper">
<div class="body" role="main">
- <blockquote>
-<div></div></blockquote>
-<div class="section" id="provider-packages">
-<h1>Provider packages<a class="headerlink" href="#provider-packages" title="Permalink to this heading">¶</a></h1>
-<p>Apache Airflow 2 is built in modular way. The “Core” of Apache Airflow provides core scheduler
-functionality which allow you to write some basic tasks, but the capabilities of Apache Airflow can
-be extended by installing additional packages, called <code class="docutils literal notranslate"><span class="pre">providers</span></code>.</p>
-<p>Providers can contain operators, hooks, sensor, and transfer operators to communicate with a
-multitude of external systems, but they can also extend Airflow core with new capabilities.</p>
-<p>You can install those provider packages separately in order to interface with a given service. The providers
-for <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> are designed in the way that you can write your own providers easily. The
-<code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span> <span class="pre">Community</span></code> develops and maintain more than 60 provider packages, but you are free to
-develop your own providers - the providers you build have exactly the same capability as the providers
-written by the community, so you can release and share those providers with others.</p>
-<p>The full list of community managed providers is available at
-<a class="reference external" href="https://airflow.apache.org/docs/#providers-packages-docs-apache-airflow-providers-index-html">Providers Index</a>.</p>
-<p>You can also see index of all community provider’s operators and hooks in
-<a class="reference internal" href="operators-and-hooks-ref/index.html"><span class="doc">Operators and Hooks Reference</span></a></p>
-</div>
-<div class="section" id="extending-airflow-core-functionality">
-<h1>Extending Airflow core functionality<a class="headerlink" href="#extending-airflow-core-functionality" title="Permalink to this heading">¶</a></h1>
-<p>Providers give you the capability of extending core Airflow with extra capabilities. The Core airflow
-provides basic and solid functionality of scheduling, the providers extend its capabilities. Here we
-describe all the custom capabilities.</p>
-<p>Airflow automatically discovers which providers add those additional capabilities and, once you install
-provider package and re-start Airflow, those become automatically available to Airflow Users.</p>
-<p>The summary of the core functionalities that can be extended are available in
-<a class="reference internal" href="core-extensions/index.html"><span class="doc">Core Extensions</span></a>.</p>
-<div class="section" id="auth-backends">
-<h2>Auth backends<a class="headerlink" href="#auth-backends" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom authentication backends, that allow you to configure the way how your
-web server authenticates your users, integrating it with public or private authentication services.</p>
-<p>You can see all the authentication backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/auth-backends.html"><span class="doc">Auth backends</span></a></p>
-</div>
-<div class="section" id="custom-connections">
-<h2>Custom connections<a class="headerlink" href="#custom-connections" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom connection types, extending connection form and handling custom form field
-behaviour for the connections defined by the provider.</p>
-<p>You can see all custom connections available via community-managed providers in
-<a class="reference internal" href="core-extensions/connections.html"><span class="doc">Connections</span></a>.</p>
-</div>
-<div class="section" id="extra-links">
-<h2>Extra links<a class="headerlink" href="#extra-links" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add extra custom links to operators delivered by the provider. Those will be visible in
-task details view of the task.</p>
-<p>You can see all the extra links available via community-managed providers in
-<a class="reference internal" href="core-extensions/extra-links.html"><span class="doc">Extra Links</span></a>.</p>
-</div>
-<div class="section" id="logging">
-<h2>Logging<a class="headerlink" href="#logging" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add additional task logging capabilities. By default <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> saves logs for
-tasks locally and make them available to Airflow UI via internal http server. However, providers
-can add extra logging capabilities, where Airflow Logs can be written to a remote service and
-retrieved from those services.</p>
-<p>You can see all task loggers available via community-managed providers in
-<a class="reference internal" href="core-extensions/logging.html"><span class="doc">Writing logs</span></a>.</p>
-</div>
-<div class="section" id="secret-backends">
-<h2>Secret backends<a class="headerlink" href="#secret-backends" title="Permalink to this heading">¶</a></h2>
-<p>Airflow has the capability of reading connections, variables and configuration from Secret Backends rather
-than from its own Database.</p>
-<p>You can see all secret backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/secrets-backends.html"><span class="doc">Secret backends</span></a>.</p>
-</div>
-<div class="section" id="notifications">
-<h2>Notifications<a class="headerlink" href="#notifications" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom notifications, that allow you to configure the way how you would like to receive
-notifications about the status of your tasks/DAGs.</p>
-<p>You can see all the notifications available via community-managed providers in
-<a class="reference internal" href="core-extensions/notifications.html"><span class="doc">Notifications</span></a>.</p>
-</div>
-</div>
-<div class="section" id="installing-and-upgrading-providers">
-<h1>Installing and upgrading providers<a class="headerlink" href="#installing-and-upgrading-providers" title="Permalink to this heading">¶</a></h1>
-<p>Separate provider packages give the possibilities that were not available in 1.10:</p>
-<ol class="arabic simple">
-<li><p>You can upgrade to latest version of particular providers without the need of Apache Airflow core upgrade.</p></li>
-<li><p>You can downgrade to previous version of particular provider in case the new version introduces
-some problems, without impacting the main Apache Airflow core package.</p></li>
-<li><p>You can release and upgrade/downgrade provider packages incrementally, independent from each other. This
-means that you can incrementally validate each of the provider package update in your environment,
-following the usual tests you have in your environment.</p></li>
-</ol>
-</div>
-<div class="section" id="types-of-providers">
-<h1>Types of providers<a class="headerlink" href="#types-of-providers" title="Permalink to this heading">¶</a></h1>
-<p>Providers have the same capacity - no matter if they are provided by the community or if they are
-third-party providers. This chapter explains how community managed providers are versioned and released
-and how you can create your own providers.</p>
-<div class="section" id="community-maintained-providers">
-<h2>Community maintained providers<a class="headerlink" href="#community-maintained-providers" title="Permalink to this heading">¶</a></h2>
-<p>From the point of view of the community, Airflow is delivered in multiple, separate packages.
-The core of Airflow scheduling system is delivered as <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package and there are more than
-60 provider packages which can be installed separately as so called <code class="docutils literal notranslate"><span class="pre">Airflow</span> <span class="pre">Provider</span> <span class="pre">packages</span></code>.
-Those packages are available as <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers</span></code> packages - for example there is an
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> or <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> package).</p>
-<p>Community maintained providers are released and versioned separately from the Airflow releases. We are
-following the <a class="reference external" href="https://semver.org/">Semver</a> versioning scheme for the packages. Some versions of the
-provider packages might depend on particular versions of Airflow, but the general approach we have is that
-unless there is a good reason, new version of providers should work with recent versions of Airflow 2.x.
-Details will vary per-provider and if there is a limitation for particular version of particular provider,
-constraining the Airflow version used, it will be included as limitation of dependencies in the provider
-package.</p>
-<p>Each community provider has corresponding extra which can be used when installing airflow to install the
-provider together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> - for example you can install airflow with those extras:
-<code class="docutils literal notranslate"><span class="pre">apache-airflow[google,amazon]</span></code> (with correct constraints -see <a class="reference external" href="/docs/apache-airflow/stable/installation/index.html" title="(in apache-airflow v2.6.0.dev0)"><span>Installation</span></a>) and you
-will install the appropriate versions of the <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> and
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> packages together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>.</p>
-<p>Some of the community providers have cross-provider dependencies as well. Those are not required
-dependencies, they might simply enable certain features (for example transfer operators often create
-dependency between different providers. Again, the general approach here is that the providers are backwards
-compatible, including cross-dependencies. Any kind of breaking changes and requirements on particular versions of other
-provider packages are automatically documented in the release notes of every provider.</p>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>For Airflow 1.10 We also provided <code class="docutils literal notranslate"><span class="pre">apache-airflow-backport-providers</span></code> packages that could be installed
-with those versions Those were the same providers as for 2.0 but automatically back-ported to work for
-Airflow 1.10. The last release of backport providers was done on March 17, 2021 and the backport
-providers will no longer be released, since Airflow 1.10 has reached End-Of-Life as of June 17, 2021.</p>
-</div>
-<p>If you want to contribute to <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>, you can see how to build and extend community
-managed providers in <a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a>.</p>
-</div>
-<div class="section" id="custom-provider-packages">
-<span id="providers-community-maintained-providers"></span><h2>Custom provider packages<a class="headerlink" href="#custom-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p>You can develop and release your own providers. Your custom operators, hooks, sensors, transfer operators
-can be packaged together in a standard airflow package and installed using the same mechanisms.
-Moreover they can also use the same mechanisms to extend the Airflow Core with auth backends,
-custom connections, logging, secret backends and extra operator links as described in the previous chapter.</p>
-</div>
-</div>
-<div class="section" id="how-to-create-your-own-provider">
-<h1>How to create your own provider<a class="headerlink" href="#how-to-create-your-own-provider" title="Permalink to this heading">¶</a></h1>
-<p>As mentioned in the <a class="reference external" href="http://airflow.apache.org/docs/apache-airflow-providers/index.html">Providers</a>
-documentation, custom providers can extend Airflow core - they can add extra links to operators as well
-as custom connections. You can use build your own providers and install them as packages if you would like
-to use the mechanism for your own, custom providers.</p>
-<p>Adding a provider to Airflow is just a matter of building a Python package and adding the right meta-data to
-the package. We are using standard mechanism of python to define
-<a class="reference external" href="https://docs.python.org/3/library/importlib.metadata.html#entry-points">entry points</a> . Your package
-needs to define appropriate entry-point <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> which has to point to a callable
-implemented by your package and return a dictionary containing the list of discoverable capabilities
-of your package. The dictionary has to follow the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json">json-schema specification</a>.</p>
-<p>Most of the schema provides extension point for the documentation (which you might want to also use for
-your own purpose) but the important fields from the extensibility point of view are those:</p>
-<p>Displaying package information in CLI/API:</p>
+ <div class="section" id="module-airflow.providers.amazon.aws.hooks.cloud_formation">
+<span id="airflow-providers-amazon-aws-hooks-cloud-formation"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.hooks.cloud_formation" title="airflow.providers.amazon.aws.hooks.cloud_formation"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.cloud_formation</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.hooks.cloud_formation" title="Permalink to this heading [...]
+<p>This module contains AWS CloudFormation Hook</p>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<div class="section" id="classes">
+<h3>Classes<a class="headerlink" href="#classes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook" title="airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">CloudFormationHook</span></code></a></p></td>
+<td><p>Interact with AWS CloudFormation.</p></td>
+</tr>
+</tbody>
+</table>
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.cloud_formation.</span></span><span class="sig-name descname"><span class="pre">CloudFormationHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span c [...]
+<dd><p>Bases: <a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p>
+<p>Interact with AWS CloudFormation.
+Provide thin wrapper around
+<a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation.html#CloudFormation.Client" title="(in Boto3 v1.26.120)"><code class="xref py py-class docutils literal notranslate"><span class="pre">boto3.client("cloudformation")</span></code></a>.</p>
+<p>Additional arguments (such as <code class="docutils literal notranslate"><span class="pre">aws_conn_id</span></code>) may be specified and
+are passed down to the underlying AwsBaseHook.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">package-name</span></code> - Name of the package for the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code> - Human-friendly name of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">description</span></code> - Additional description of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">version</span></code> - List of versions of the package (in reverse-chronological order). The first version in the
-list is the current package version. It is taken from the version of package installed, not from the
-provider_info information.</p></li>
+<li><p><a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-class docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p></li>
</ul>
-<p>Exposing customized functionality to the Airflow’s core:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">extra-links</span></code> - this field should contain the list of all operator class names that are adding extra links
-capability. See <a class="reference external" href="/docs/apache-airflow/stable/howto/define-extra-link.html" title="(in apache-airflow v2.6.0.dev0)"><span>Define an operator extra link</span></a> for description of how to add extra link
-capability to the operators of yours.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">connection-types</span></code> - this field should contain the list of all connection types together with hook
-class names implementing those custom connection types (providing custom extra fields and
-custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
-<code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> (deprecated) - this field should contain the list of all hook class names that provide
-custom connection types with custom extra fields and field behaviour. The <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array
-is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
-providers are targeting Airflow 2.2.0+ you do not have to include the <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array, if
-you want to also target earlier versions of Airflow 2, you should include both <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> and
-<code class="docutils literal notranslate"><span class="pre">connection-types</span></code> arrays. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details.</p></li>
-</ul>
-<p>When your providers are installed you can query the installed providers and their capabilities with the
-<code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">providers</span></code> command. This way you can verify if your providers are properly recognized and whether
-they define the extensions properly. See <a class="reference external" href="/docs/apache-airflow/stable/cli-and-env-variables-ref.html" title="(in apache-airflow v2.6.0.dev0)"><span>Command Line Interface and Environment Variables Reference</span></a> for details of available CLI
-sub-commands.</p>
-<p>When you write your own provider, consider following the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#naming-conventions-for-provider-packages">Naming conventions for provider packages</a></p>
-</div>
-<div class="section" id="faq-for-airflow-and-providers">
-<h1>FAQ for Airflow and Providers<a class="headerlink" href="#faq-for-airflow-and-providers" title="Permalink to this heading">¶</a></h1>
-<div class="section" id="upgrading-airflow-2-0-and-providers">
-<h2>Upgrading Airflow 2.0 and Providers<a class="headerlink" href="#upgrading-airflow-2-0-and-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When upgrading to a new Airflow version such as 2.0, but possibly 2.0.1 and beyond, is the best practice
-to also upgrade provider packages at the same time?</strong></p>
-<p>It depends on your use case. If you have automated or semi-automated verification of your installation,
-that you can run a new version of Airflow including all provider packages, then definitely go for it.
-If you rely more on manual testing, it is advised that you upgrade in stages. Depending on your choice
-you can either upgrade all used provider packages first, and then upgrade Airflow Core or the other way
-round. The first approach - when you first upgrade all providers is probably safer, as you can do it
-incrementally, step-by-step replacing provider by provider in your environment.</p>
</div>
-<div class="section" id="customizing-provider-packages">
-<h2>Customizing Provider Packages<a class="headerlink" href="#customizing-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an older version of my provider package which we have lightly customized and is working
-fine with my MSSQL installation. I am upgrading my Airflow version. Do I need to upgrade my provider,
-or can I keep it as it is?</strong></p>
-<p>It depends on the scope of customization. There is no need to upgrade the provider packages to later
-versions unless you want to upgrade to Airflow version that introduces backwards-incompatible changes.
-Generally speaking, with Airflow 2 we are following the <a class="reference external" href="https://semver.org/">Semver</a> approach where
-we will introduce backwards-incompatible changes in Major releases, so all your modifications (as long
-as you have not used internal Airflow classes) should work for All Airflow 2.* versions.</p>
-</div>
-<div class="section" id="creating-your-own-providers">
-<h2>Creating your own providers<a class="headerlink" href="#creating-your-own-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When I write my own provider, do I need to do anything special to make it available to others?</strong></p>
-<p>You do not need to do anything special besides creating the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point
-returning properly formatted meta-data - dictionary with <code class="docutils literal notranslate"><span class="pre">extra-links</span></code> and <code class="docutils literal notranslate"><span class="pre">connection-types</span></code> fields
-(and deprecated <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> field if you are also targeting versions of Airflow before 2.2.0).</p>
-<p>Anyone who runs airflow in an environment that has your Python package installed will be able to use the
-package as a provider package.</p>
-<p><strong>What do I need to do to turn a package into a provider?</strong></p>
-<p>You need to do the following to turn an existing Python package into a provider (see below for examples):</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook.get_stack_status">
+<span class="sig-name descname"><span class="pre">get_stack_status</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">stack_name</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/cloud_formation.html#CloudFormationHook.get_stack_status"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#air [...]
+<dd><p>Get stack status from CloudFormation.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Add the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point in the <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> - this tells airflow where to get
-the required provider metadata</p></li>
-<li><p>Create the function that you refer to in the first step as part of your package: this functions returns a
-dictionary that contains all meta-data about your provider package</p></li>
-<li><p>If you want Airflow to link to documentation of your Provider in the providers page, make sure
-to add “project-url/documentation” <a class="reference external" href="https://peps.python.org/pep-0621/#example">metadata</a> to your package.
-This will also add link to your documentation in PyPI.</p></li>
-<li><p>note that the dictionary should be compliant with <code class="docutils literal notranslate"><span class="pre">airflow/provider_info.schema.json</span></code> JSON-schema
-specification. The community-managed providers have more fields there that are used to build
-documentation, but the requirement for runtime information only contains several fields which are defined
-in the schema:</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation/client/describe_stacks.html#CloudFormation.Client.describe_stacks" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">CloudFormation.Client.describe_stacks()</span></code></a></p></li>
</ul>
-<div class="example-block-wrapper compound">
-<p class="example-header compound-first"><span class="example-title">airflow/provider_info.schema.json</span></p>
-<div class="compound-last highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"$schema"</span><span class="p">:</span><span class="w"> </span><span class="s2">"http://json-schema.org/draft-07/schema#"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"package-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Package name available under which the package is available in the PyPI repository."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Provider name"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Information about the package in RST format"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-names"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class names that provide connection types to core (deprecated by connection-types)"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"deprecated"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"The hook-class-names property has been deprecated in favour of connection-types which is more performant version allowing to only import individual Hooks rather than all hooks at once"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"deprecatedVersion"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.2.0"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"connection-types"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Map of connection types mapped to hook class names."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"connection-type"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Type of connection defined by the provider"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class name that implements the connection type"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"connection-type"</span><span class="p">,</span><span class="w"> </span><span class="s2">"hook-class-name"</span><span class="p">]</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"extra-links"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Operator class names that provide extra link functionality"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"secrets-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Secrets Backend class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"logging"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Logging Task Handlers class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"auth-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"API Auth Backend module names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"notifications"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Notification class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"task-decorators"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Apply custom decorators to the TaskFlow API. Can be accessed by users via '@task.<name>'"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"path"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
-<span class="w"> </span><span class="s2">"name"</span><span class="p">,</span>
-<span class="w"> </span><span class="s2">"description"</span>
-<span class="w"> </span><span class="p">]</span>
-<span class="p">}</span>
-</pre></div>
-</div>
-</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>:</p>
-<div class="highlight-cfg notranslate"><div class="highlight"><pre><span></span><span class="k">[options.entry_points]</span>
-<span class="c1"># the function get_provider_info is defined in myproviderpackage.somemodule</span>
-<span class="na">apache_airflow_provider</span><span class="o">=</span>
-<span class="w"> </span><span class="na">provider_info</span><span class="o">=</span><span class="s">myproviderpackage.somemodule:get_provider_info</span>
-</pre></div>
</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">myproviderpackage/somemodule.py</span></code>:</p>
-<div class="highlight-Python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_provider_info</span><span class="p">():</span>
- <span class="k">return</span> <span class="p">{</span>
- <span class="s2">"package-name"</span><span class="p">:</span> <span class="s2">"my-package-name"</span><span class="p">,</span>
- <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"name"</span><span class="p">,</span>
- <span class="s2">"description"</span><span class="p">:</span> <span class="s2">"a description"</span><span class="p">,</span>
- <span class="s2">"hook-class-names"</span><span class="p">:</span> <span class="p">[</span>
- <span class="s2">"myproviderpackage.hooks.source.SourceHook"</span><span class="p">,</span>
- <span class="p">],</span>
- <span class="p">}</span>
-</pre></div>
-</div>
-<p><strong>How do provider packages work under the hood?</strong></p>
-<p>When running Airflow with your provider package, there will be (at least) three components to your airflow installation:</p>
+<dl class="field-list simple">
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook.create_stack">
+<span class="sig-name descname"><span class="pre">create_stack</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">stack_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">cloudformation_parameters</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/cloud_formation.html#CloudFormationHook.create_stack"><span clas [...]
+<dd><p>Create stack in CloudFormation.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>The installation itself (for example, a <code class="docutils literal notranslate"><span class="pre">venv</span></code> where you installed airflow with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">apache-airflow</span></code>)
-together with the related files (e.g. <code class="docutils literal notranslate"><span class="pre">dags</span></code> folder)</p></li>
-<li><p>The <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package</p></li>
-<li><p>Your own <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package that is independent of <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> or your airflow installation, which
-can be a local Python package (that you install via <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span> <span class="pre">/path/to/my-package</span></code>), a normal pip package
-(<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">myproviderpackage</span></code>), or any other type of Python package</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation/client/create_stack.html#CloudFormation.Client.create_stack" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">CloudFormation.Client.create_stack()</span></code></a></p></li>
</ul>
-<p>In the <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package you need to add the entry point and provide the appropriate metadata as described above.
-If you have done that, airflow does the following at runtime:</p>
-<ul class="simple">
-<li><p>Loop through ALL packages installed in your environment / <code class="docutils literal notranslate"><span class="pre">venv</span></code></p></li>
-<li><p>For each package, if the package’s <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> has a section <code class="docutils literal notranslate"><span class="pre">[options.entry_points]</span></code>, and if that section has a value
-for <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code>, then get the value for <code class="docutils literal notranslate"><span class="pre">provider_info</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code></p></li>
-<li><p>That value works like an import statement: <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code> translates to something like
-<code class="docutils literal notranslate"><span class="pre">from</span> <span class="pre">myproviderpackage.somemodule</span> <span class="pre">import</span> <span class="pre">get_provider_info</span></code>, and the <code class="docutils literal notranslate"><span class="pre">get_provider_info</span></code> that is being imported should be a
-callable, i.e. a function</p></li>
-<li><p>This function should return a dictionary with metadata</p></li>
-<li><p>If you have custom connection types as part of your package, that metadata will including a field called <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>
-which should be a list of strings of your custom hooks - those strings should also be in an import-like format, e.g.
-<code class="docutils literal notranslate"><span class="pre">myproviderpackage.hooks.source.SourceHook</span></code> means that there is a class <code class="docutils literal notranslate"><span class="pre">SourceHook</span></code> in <code class="docutils literal notranslate"><span class="pre">myproviderpackage/hooks/source.py</span></code>
-- airflow then imports these hooks and looks for the functions <code class="docutils literal notranslate"><span class="pre">get_ui_field_behaviour</span></code> and <code class="docutils literal notranslate"><span class="pre">get_connection_form_widgets</span></code>
-(both optional) as well as the attributes <code class="docutils literal notranslate"><span class="pre">conn_type</span></code> and <code class="docutils literal notranslate"><span class="pre">hook_name</span></code> to create the custom connection type in the airflow UI</p></li>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>stack_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – stack_name.</p></li>
+<li><p><strong>cloudformation_parameters</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a>) – parameters to be passed to CloudFormation.</p></li>
</ul>
-<p><strong>Should I name my provider specifically or should it be created in ``airflow.providers`` package?</strong></p>
-<p>We have quite a number (>60) of providers managed by the community and we are going to maintain them
-together with Apache Airflow. All those providers have well-defined structured and follow the
-naming conventions we defined and they are all in <code class="docutils literal notranslate"><span class="pre">airflow.providers</span></code> package. If your intention is
-to contribute your provider, then you should follow those conventions and make a PR to Apache Airflow
-to contribute to it. But you are free to use any package name as long as there are no conflicts with other
-names, so preferably choose package that is in your “domain”.</p>
-<p><strong>Is there a convention for a connection id and type?</strong></p>
-<p>Very good question. Glad that you asked. We usually follow the convention <code class="docutils literal notranslate"><span class="pre"><NAME>_default</span></code> for connection
-id and just <code class="docutils literal notranslate"><span class="pre"><NAME></span></code> for connection type. Few examples:</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook.delete_stack">
+<span class="sig-name descname"><span class="pre">delete_stack</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">stack_name</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">cloudformation_parameters</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../. [...]
+<dd><p>Delete stack in CloudFormation.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">google_cloud_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">google_cloud_platform</span></code> type</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">aws_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">aws</span></code> type</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation/client/delete_stack.html#CloudFormation.Client.delete_stack" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">CloudFormation.Client.delete_stack()</span></code></a></p></li>
</ul>
-<p>You should follow this convention. It is important, to use unique names for connection type,
-so it should be unique for your provider. If two providers try to add connection with the same type
-only one of them will succeed.</p>
-<p><strong>Can I contribute my own provider to Apache Airflow?</strong></p>
-<p>Of course, but it’s better to check at developer’s mailing list whether such contribution will be accepted by
-the Community, before investing time to make the provider compliant with community requirements.
-The Community only accepts providers that are generic enough, are well documented, fully covered by tests
-and with capabilities of being tested by people in the community. So we might not always be in the
-position to accept such contributions.</p>
-<p>After you think that your provider matches the expected values above, you can read
-<a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a> to check all prerequisites for a new
-community Provider and discuss it at the <a class="reference external" href="http://airflow.apache.org/community/">Devlist</a>.</p>
-<p>However, in case you have your own, specific provider, which you can maintain on your own or by your
-team, you are free to publish the providers in whatever form you find appropriate. The custom and
-community-managed providers have exactly the same capabilities.</p>
-<p><strong>Can I advertise my own provider to Apache Airflow users and share it with others as package in PyPI?</strong></p>
-<p>Absolutely! We have an <a class="reference external" href="https://airflow.apache.org/ecosystem/">Ecosystem</a> area on our website where
-we share non-community managed extensions and work for Airflow. Feel free to make a PR to the page and
-add we will evaluate and merge it when we see that such provider can be useful for the community of
-Airflow users.</p>
-<p><strong>Can I charge for the use of my provider?</strong></p>
-<p>This is something that is outside of our control and domain. As an Apache project, we are
-commercial-friendly and there are many businesses built around Apache Airflow and many other
-Apache projects. As a community, we provide all the software for free and this will never
-change. What 3rd-party developers are doing is not under control of Apache Airflow community.</p>
</div>
-<div class="section" id="using-backport-providers-in-airflow-1-10">
-<h2>Using Backport Providers in Airflow 1.10<a class="headerlink" href="#using-backport-providers-in-airflow-1-10" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an Airflow version (1.10.12) running and it is stable. However, because of a Cloud provider change,
-I would like to upgrade the provider package. If I don’t need to upgrade the Airflow version anymore,
-how do I know that this provider version is compatible with my Airflow version?</strong></p>
-<p>We have Backport Providers are compatible with 1.10 but they stopped being released on
-March 17, 2021. Since then, no new changes to providers for Airflow 2.0 are going to be
-released as backport packages. It’s the highest time to upgrade to Airflow 2.0.</p>
-<p>When it comes to compatibility of providers with different Airflow 2 versions, each
-provider package will keep its own dependencies, and while we expect those providers to be generally
-backwards-compatible, particular versions of particular providers might introduce dependencies on
-specific Airflow versions.</p>
-<div class="toctree-wrapper compound">
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>stack_name</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – stack_name.</p></li>
+<li><p><strong>cloudformation_parameters</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a><em> | </em><em>None</em>) – parameters to be passed to CloudFormation (optional).</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+</dd></dl>
+
</div>
</div>
</div>
- <div class="pager" role="navigation" aria-label="related navigation"><button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Previous</button>
- <a rel="next" title="Providers packages reference" href="packages-ref.html" >
+ <div class="pager" role="navigation" aria-label="related navigation">
+ <a rel="prev" title="airflow.providers.amazon.aws.hooks.batch_waiters" href="../batch_waiters/index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="p">Previous</button>
+ </a>
+ <a rel="next" title="airflow.providers.amazon.aws.hooks.datasync" href="../datasync/index.html" >
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="n">Next</button>
</a>
</div>
@@ -1044,28 +779,19 @@ specific Airflow versions.</p>
<nav class="wy-nav-side-toc">
<div class="wy-menu-vertical">
<ul>
-<li><a class="reference internal" href="#">Provider packages</a></li>
-<li><a class="reference internal" href="#extending-airflow-core-functionality">Extending Airflow core functionality</a><ul>
-<li><a class="reference internal" href="#auth-backends">Auth backends</a></li>
-<li><a class="reference internal" href="#custom-connections">Custom connections</a></li>
-<li><a class="reference internal" href="#extra-links">Extra links</a></li>
-<li><a class="reference internal" href="#logging">Logging</a></li>
-<li><a class="reference internal" href="#secret-backends">Secret backends</a></li>
-<li><a class="reference internal" href="#notifications">Notifications</a></li>
+<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.cloud_formation</span></code></a><ul>
+<li><a class="reference internal" href="#module-contents">Module Contents</a><ul>
+<li><a class="reference internal" href="#classes">Classes</a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook"><code class="docutils literal notranslate"><span class="pre">CloudFormationHook</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook.get_stack_status"><code class="docutils literal notranslate"><span class="pre">CloudFormationHook.get_stack_status()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook.create_stack"><code class="docutils literal notranslate"><span class="pre">CloudFormationHook.create_stack()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.cloud_formation.CloudFormationHook.delete_stack"><code class="docutils literal notranslate"><span class="pre">CloudFormationHook.delete_stack()</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
-<li><a class="reference internal" href="#installing-and-upgrading-providers">Installing and upgrading providers</a></li>
-<li><a class="reference internal" href="#types-of-providers">Types of providers</a><ul>
-<li><a class="reference internal" href="#community-maintained-providers">Community maintained providers</a></li>
-<li><a class="reference internal" href="#custom-provider-packages">Custom provider packages</a></li>
</ul>
</li>
-<li><a class="reference internal" href="#how-to-create-your-own-provider">How to create your own provider</a></li>
-<li><a class="reference internal" href="#faq-for-airflow-and-providers">FAQ for Airflow and Providers</a><ul>
-<li><a class="reference internal" href="#upgrading-airflow-2-0-and-providers">Upgrading Airflow 2.0 and Providers</a></li>
-<li><a class="reference internal" href="#customizing-provider-packages">Customizing Provider Packages</a></li>
-<li><a class="reference internal" href="#creating-your-own-providers">Creating your own providers</a></li>
-<li><a class="reference internal" href="#using-backport-providers-in-airflow-1-10">Using Backport Providers in Airflow 1.10</a></li>
</ul>
</li>
</ul>
@@ -1090,7 +816,7 @@ specific Airflow versions.</p>
<div class="base-layout--button">
- <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers/index.rst" rel="nofollow">
+ <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-amazon/_api/airflow/providers/amazon/aws/hooks/cloud_formation/index.rst" rel="nofollow">
<button class="btn-hollow btn-brown btn-with-icon with-box-shadow button-fixed">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
@@ -1224,19 +950,19 @@ specific Airflow versions.</p>
</span>
</div>
</footer>
- <script type="text/javascript" src="_static/_gen/js/docs.js"></script>
- <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
+ <script type="text/javascript" src="../../../../../../../_static/_gen/js/docs.js"></script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../../../../" src="../../../../../../../_static/documentation_options.js"></script>
<script src="https://code.jquery.com/jquery-3.4.1.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js" integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy" crossorigin="anonymous"></script>
- <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
- <script src="_static/jquery.js"></script>
- <script src="_static/underscore.js"></script>
- <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
- <script src="_static/doctools.js"></script>
- <script src="_static/sphinx_highlight.js"></script>
- <script src="_static/clipboard.min.js"></script>
- <script src="_static/copybutton.js"></script>
- <script src="_static/js/globaltoc.js"></script>
+ <script data-url_root="../../../../../../../" id="documentation_options" src="../../../../../../../_static/documentation_options.js"></script>
+ <script src="../../../../../../../_static/jquery.js"></script>
+ <script src="../../../../../../../_static/underscore.js"></script>
+ <script src="../../../../../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
+ <script src="../../../../../../../_static/doctools.js"></script>
+ <script src="../../../../../../../_static/sphinx_highlight.js"></script>
+ <script src="../../../../../../../_static/clipboard.min.js"></script>
+ <script src="../../../../../../../_static/copybutton.js"></script>
+ <script src="../../../../../../../_static/js/globaltoc.js"></script>
</body>
</html>
\ No newline at end of file
diff --git a/docs-archive/apache-airflow-providers/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/datasync/index.html
similarity index 60%
copy from docs-archive/apache-airflow-providers/index.html
copy to docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/datasync/index.html
index 70eef65adf..c5c96eb81a 100644
--- a/docs-archive/apache-airflow-providers/index.html
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/datasync/index.html
@@ -9,18 +9,20 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Provider packages — apache-airflow-providers Documentation</title>
- <link rel="stylesheet" href="_static/_gen/css/main.min.css" type="text/css" />
- <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main.min.css" />
- <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main-custom.min.css" />
- <link rel="shortcut icon" href="_static/pin_32.png"/>
- <link rel="index" title="Index" href="genindex.html" />
- <link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Providers packages reference" href="packages-ref.html" />
-
- <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers/stable/index.html" />
+ <title>airflow.providers.amazon.aws.hooks.datasync — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../../search.html" />
+ <link rel="next" title="airflow.providers.amazon.aws.hooks.dms" href="../dms/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.hooks.cloud_formation" href="../cloud_formation/index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/hooks/datasync/index.html" />
<!-- Matomo -->
<script>
@@ -282,8 +284,21 @@
<div class="content-drawer">
<div class="td-sidebar">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -313,14 +328,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -401,8 +434,21 @@
<div class="td-sidebar desktop-only d-print-none">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -432,14 +478,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -528,9 +592,15 @@
<ul class="breadcrumb">
- <li class="breadcrumb-item"><a href="#" class="icon icon-home"> Home</a></li>
+ <li class="breadcrumb-item"><a href="../../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
- <li class="breadcrumb-item"><a href="index.html"> Provider packages</a></li>
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.datasync</span></code></a></li>
</ul>
</div>
@@ -541,440 +611,362 @@
<div class="bodywrapper">
<div class="body" role="main">
- <blockquote>
-<div></div></blockquote>
-<div class="section" id="provider-packages">
-<h1>Provider packages<a class="headerlink" href="#provider-packages" title="Permalink to this heading">¶</a></h1>
-<p>Apache Airflow 2 is built in modular way. The “Core” of Apache Airflow provides core scheduler
-functionality which allow you to write some basic tasks, but the capabilities of Apache Airflow can
-be extended by installing additional packages, called <code class="docutils literal notranslate"><span class="pre">providers</span></code>.</p>
-<p>Providers can contain operators, hooks, sensor, and transfer operators to communicate with a
-multitude of external systems, but they can also extend Airflow core with new capabilities.</p>
-<p>You can install those provider packages separately in order to interface with a given service. The providers
-for <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> are designed in the way that you can write your own providers easily. The
-<code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span> <span class="pre">Community</span></code> develops and maintain more than 60 provider packages, but you are free to
-develop your own providers - the providers you build have exactly the same capability as the providers
-written by the community, so you can release and share those providers with others.</p>
-<p>The full list of community managed providers is available at
-<a class="reference external" href="https://airflow.apache.org/docs/#providers-packages-docs-apache-airflow-providers-index-html">Providers Index</a>.</p>
-<p>You can also see index of all community provider’s operators and hooks in
-<a class="reference internal" href="operators-and-hooks-ref/index.html"><span class="doc">Operators and Hooks Reference</span></a></p>
-</div>
-<div class="section" id="extending-airflow-core-functionality">
-<h1>Extending Airflow core functionality<a class="headerlink" href="#extending-airflow-core-functionality" title="Permalink to this heading">¶</a></h1>
-<p>Providers give you the capability of extending core Airflow with extra capabilities. The Core airflow
-provides basic and solid functionality of scheduling, the providers extend its capabilities. Here we
-describe all the custom capabilities.</p>
-<p>Airflow automatically discovers which providers add those additional capabilities and, once you install
-provider package and re-start Airflow, those become automatically available to Airflow Users.</p>
-<p>The summary of the core functionalities that can be extended are available in
-<a class="reference internal" href="core-extensions/index.html"><span class="doc">Core Extensions</span></a>.</p>
-<div class="section" id="auth-backends">
-<h2>Auth backends<a class="headerlink" href="#auth-backends" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom authentication backends, that allow you to configure the way how your
-web server authenticates your users, integrating it with public or private authentication services.</p>
-<p>You can see all the authentication backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/auth-backends.html"><span class="doc">Auth backends</span></a></p>
-</div>
-<div class="section" id="custom-connections">
-<h2>Custom connections<a class="headerlink" href="#custom-connections" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom connection types, extending connection form and handling custom form field
-behaviour for the connections defined by the provider.</p>
-<p>You can see all custom connections available via community-managed providers in
-<a class="reference internal" href="core-extensions/connections.html"><span class="doc">Connections</span></a>.</p>
-</div>
-<div class="section" id="extra-links">
-<h2>Extra links<a class="headerlink" href="#extra-links" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add extra custom links to operators delivered by the provider. Those will be visible in
-task details view of the task.</p>
-<p>You can see all the extra links available via community-managed providers in
-<a class="reference internal" href="core-extensions/extra-links.html"><span class="doc">Extra Links</span></a>.</p>
-</div>
-<div class="section" id="logging">
-<h2>Logging<a class="headerlink" href="#logging" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add additional task logging capabilities. By default <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> saves logs for
-tasks locally and make them available to Airflow UI via internal http server. However, providers
-can add extra logging capabilities, where Airflow Logs can be written to a remote service and
-retrieved from those services.</p>
-<p>You can see all task loggers available via community-managed providers in
-<a class="reference internal" href="core-extensions/logging.html"><span class="doc">Writing logs</span></a>.</p>
-</div>
-<div class="section" id="secret-backends">
-<h2>Secret backends<a class="headerlink" href="#secret-backends" title="Permalink to this heading">¶</a></h2>
-<p>Airflow has the capability of reading connections, variables and configuration from Secret Backends rather
-than from its own Database.</p>
-<p>You can see all secret backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/secrets-backends.html"><span class="doc">Secret backends</span></a>.</p>
-</div>
-<div class="section" id="notifications">
-<h2>Notifications<a class="headerlink" href="#notifications" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom notifications, that allow you to configure the way how you would like to receive
-notifications about the status of your tasks/DAGs.</p>
-<p>You can see all the notifications available via community-managed providers in
-<a class="reference internal" href="core-extensions/notifications.html"><span class="doc">Notifications</span></a>.</p>
-</div>
-</div>
-<div class="section" id="installing-and-upgrading-providers">
-<h1>Installing and upgrading providers<a class="headerlink" href="#installing-and-upgrading-providers" title="Permalink to this heading">¶</a></h1>
-<p>Separate provider packages give the possibilities that were not available in 1.10:</p>
-<ol class="arabic simple">
-<li><p>You can upgrade to latest version of particular providers without the need of Apache Airflow core upgrade.</p></li>
-<li><p>You can downgrade to previous version of particular provider in case the new version introduces
-some problems, without impacting the main Apache Airflow core package.</p></li>
-<li><p>You can release and upgrade/downgrade provider packages incrementally, independent from each other. This
-means that you can incrementally validate each of the provider package update in your environment,
-following the usual tests you have in your environment.</p></li>
-</ol>
-</div>
-<div class="section" id="types-of-providers">
-<h1>Types of providers<a class="headerlink" href="#types-of-providers" title="Permalink to this heading">¶</a></h1>
-<p>Providers have the same capacity - no matter if they are provided by the community or if they are
-third-party providers. This chapter explains how community managed providers are versioned and released
-and how you can create your own providers.</p>
-<div class="section" id="community-maintained-providers">
-<h2>Community maintained providers<a class="headerlink" href="#community-maintained-providers" title="Permalink to this heading">¶</a></h2>
-<p>From the point of view of the community, Airflow is delivered in multiple, separate packages.
-The core of Airflow scheduling system is delivered as <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package and there are more than
-60 provider packages which can be installed separately as so called <code class="docutils literal notranslate"><span class="pre">Airflow</span> <span class="pre">Provider</span> <span class="pre">packages</span></code>.
-Those packages are available as <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers</span></code> packages - for example there is an
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> or <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> package).</p>
-<p>Community maintained providers are released and versioned separately from the Airflow releases. We are
-following the <a class="reference external" href="https://semver.org/">Semver</a> versioning scheme for the packages. Some versions of the
-provider packages might depend on particular versions of Airflow, but the general approach we have is that
-unless there is a good reason, new version of providers should work with recent versions of Airflow 2.x.
-Details will vary per-provider and if there is a limitation for particular version of particular provider,
-constraining the Airflow version used, it will be included as limitation of dependencies in the provider
-package.</p>
-<p>Each community provider has corresponding extra which can be used when installing airflow to install the
-provider together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> - for example you can install airflow with those extras:
-<code class="docutils literal notranslate"><span class="pre">apache-airflow[google,amazon]</span></code> (with correct constraints -see <a class="reference external" href="/docs/apache-airflow/stable/installation/index.html" title="(in apache-airflow v2.6.0.dev0)"><span>Installation</span></a>) and you
-will install the appropriate versions of the <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> and
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> packages together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>.</p>
-<p>Some of the community providers have cross-provider dependencies as well. Those are not required
-dependencies, they might simply enable certain features (for example transfer operators often create
-dependency between different providers. Again, the general approach here is that the providers are backwards
-compatible, including cross-dependencies. Any kind of breaking changes and requirements on particular versions of other
-provider packages are automatically documented in the release notes of every provider.</p>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>For Airflow 1.10 We also provided <code class="docutils literal notranslate"><span class="pre">apache-airflow-backport-providers</span></code> packages that could be installed
-with those versions Those were the same providers as for 2.0 but automatically back-ported to work for
-Airflow 1.10. The last release of backport providers was done on March 17, 2021 and the backport
-providers will no longer be released, since Airflow 1.10 has reached End-Of-Life as of June 17, 2021.</p>
-</div>
-<p>If you want to contribute to <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>, you can see how to build and extend community
-managed providers in <a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a>.</p>
-</div>
-<div class="section" id="custom-provider-packages">
-<span id="providers-community-maintained-providers"></span><h2>Custom provider packages<a class="headerlink" href="#custom-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p>You can develop and release your own providers. Your custom operators, hooks, sensors, transfer operators
-can be packaged together in a standard airflow package and installed using the same mechanisms.
-Moreover they can also use the same mechanisms to extend the Airflow Core with auth backends,
-custom connections, logging, secret backends and extra operator links as described in the previous chapter.</p>
-</div>
-</div>
-<div class="section" id="how-to-create-your-own-provider">
-<h1>How to create your own provider<a class="headerlink" href="#how-to-create-your-own-provider" title="Permalink to this heading">¶</a></h1>
-<p>As mentioned in the <a class="reference external" href="http://airflow.apache.org/docs/apache-airflow-providers/index.html">Providers</a>
-documentation, custom providers can extend Airflow core - they can add extra links to operators as well
-as custom connections. You can use build your own providers and install them as packages if you would like
-to use the mechanism for your own, custom providers.</p>
-<p>Adding a provider to Airflow is just a matter of building a Python package and adding the right meta-data to
-the package. We are using standard mechanism of python to define
-<a class="reference external" href="https://docs.python.org/3/library/importlib.metadata.html#entry-points">entry points</a> . Your package
-needs to define appropriate entry-point <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> which has to point to a callable
-implemented by your package and return a dictionary containing the list of discoverable capabilities
-of your package. The dictionary has to follow the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json">json-schema specification</a>.</p>
-<p>Most of the schema provides extension point for the documentation (which you might want to also use for
-your own purpose) but the important fields from the extensibility point of view are those:</p>
-<p>Displaying package information in CLI/API:</p>
+ <div class="section" id="module-airflow.providers.amazon.aws.hooks.datasync">
+<span id="airflow-providers-amazon-aws-hooks-datasync"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.hooks.datasync" title="airflow.providers.amazon.aws.hooks.datasync"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.datasync</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.hooks.datasync" title="Permalink to this heading">¶</a></h1>
+<p>Interact with AWS DataSync, using the AWS <code class="docutils literal notranslate"><span class="pre">boto3</span></code> library.</p>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<div class="section" id="classes">
+<h3>Classes<a class="headerlink" href="#classes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook" title="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">DataSyncHook</span></code></a></p></td>
+<td><p>Interact with AWS DataSync.</p></td>
+</tr>
+</tbody>
+</table>
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.datasync.</span></span><span class="sig-name descname"><span class="pre">DataSyncHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">wait_interval_seconds</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">30 [...]
+<dd><p>Bases: <a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p>
+<p>Interact with AWS DataSync.
+Provide thick wrapper around <a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync.html#DataSync.Client" title="(in Boto3 v1.26.120)"><code class="xref py py-class docutils literal notranslate"><span class="pre">boto3.client("datasync")</span></code></a>.</p>
+<p>Additional arguments (such as <code class="docutils literal notranslate"><span class="pre">aws_conn_id</span></code>) may be specified and
+are passed down to the underlying AwsBaseHook.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>wait_interval_seconds</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a>) – Time to wait between two
+consecutive calls to check TaskExecution status. Defaults to 30 seconds.</p>
+</dd>
+<dt class="field-even">Raises</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/exceptions.html#ValueError" title="(in Python v3.11)"><strong>ValueError</strong></a> – If wait_interval_seconds is not between 0 and 15*60 seconds.</p>
+</dd>
+</dl>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">package-name</span></code> - Name of the package for the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code> - Human-friendly name of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">description</span></code> - Additional description of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">version</span></code> - List of versions of the package (in reverse-chronological order). The first version in the
-list is the current package version. It is taken from the version of package installed, not from the
-provider_info information.</p></li>
+<li><p><a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-class docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p></li>
</ul>
-<p>Exposing customized functionality to the Airflow’s core:</p>
+</div>
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.TASK_EXECUTION_INTERMEDIATE_STATES">
+<span class="sig-name descname"><span class="pre">TASK_EXECUTION_INTERMEDIATE_STATES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('INITIALIZING',</span> <span class="pre">'QUEUED',</span> <span class="pre">'LAUNCHING',</span> <span class="pre">'PREPARING',</span> <span class="pre">'TRANSFERRING',</span> <span class="pre">'VERIFYING')</span></em><a class="reference internal" href="../. [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.TASK_EXECUTION_FAILURE_STATES">
+<span class="sig-name descname"><span class="pre">TASK_EXECUTION_FAILURE_STATES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('ERROR',)</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHook.TASK_EXECUTION_FAILURE_STATES"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class=" [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.TASK_EXECUTION_SUCCESS_STATES">
+<span class="sig-name descname"><span class="pre">TASK_EXECUTION_SUCCESS_STATES</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">('SUCCESS',)</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHook.TASK_EXECUTION_SUCCESS_STATES"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class [...]
+<dd></dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.create_location">
+<span class="sig-name descname"><span class="pre">create_location</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">location_uri</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">create_location_kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.htm [...]
+<dd><p>Creates a new location.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">extra-links</span></code> - this field should contain the list of all operator class names that are adding extra links
-capability. See <a class="reference external" href="/docs/apache-airflow/stable/howto/define-extra-link.html" title="(in apache-airflow v2.6.0.dev0)"><span>Define an operator extra link</span></a> for description of how to add extra link
-capability to the operators of yours.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">connection-types</span></code> - this field should contain the list of all connection types together with hook
-class names implementing those custom connection types (providing custom extra fields and
-custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
-<code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> (deprecated) - this field should contain the list of all hook class names that provide
-custom connection types with custom extra fields and field behaviour. The <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array
-is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
-providers are targeting Airflow 2.2.0+ you do not have to include the <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array, if
-you want to also target earlier versions of Airflow 2, you should include both <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> and
-<code class="docutils literal notranslate"><span class="pre">connection-types</span></code> arrays. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details.</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/create_location_s3.html#DataSync.Client.create_location_s3" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.create_location_s3()</span></code></a></p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/create_location_smb.html#DataSync.Client.create_location_smb" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.create_location_smb()</span></code></a></p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/create_location_nfs.html#DataSync.Client.create_location_nfs" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.create_location_nfs()</span></code></a></p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/create_location_efs.html#DataSync.Client.create_location_efs" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.create_location_efs()</span></code></a></p></li>
</ul>
-<p>When your providers are installed you can query the installed providers and their capabilities with the
-<code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">providers</span></code> command. This way you can verify if your providers are properly recognized and whether
-they define the extensions properly. See <a class="reference external" href="/docs/apache-airflow/stable/cli-and-env-variables-ref.html" title="(in apache-airflow v2.6.0.dev0)"><span>Command Line Interface and Environment Variables Reference</span></a> for details of available CLI
-sub-commands.</p>
-<p>When you write your own provider, consider following the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#naming-conventions-for-provider-packages">Naming conventions for provider packages</a></p>
-</div>
-<div class="section" id="faq-for-airflow-and-providers">
-<h1>FAQ for Airflow and Providers<a class="headerlink" href="#faq-for-airflow-and-providers" title="Permalink to this heading">¶</a></h1>
-<div class="section" id="upgrading-airflow-2-0-and-providers">
-<h2>Upgrading Airflow 2.0 and Providers<a class="headerlink" href="#upgrading-airflow-2-0-and-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When upgrading to a new Airflow version such as 2.0, but possibly 2.0.1 and beyond, is the best practice
-to also upgrade provider packages at the same time?</strong></p>
-<p>It depends on your use case. If you have automated or semi-automated verification of your installation,
-that you can run a new version of Airflow including all provider packages, then definitely go for it.
-If you rely more on manual testing, it is advised that you upgrade in stages. Depending on your choice
-you can either upgrade all used provider packages first, and then upgrade Airflow Core or the other way
-round. The first approach - when you first upgrade all providers is probably safer, as you can do it
-incrementally, step-by-step replacing provider by provider in your environment.</p>
-</div>
-<div class="section" id="customizing-provider-packages">
-<h2>Customizing Provider Packages<a class="headerlink" href="#customizing-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an older version of my provider package which we have lightly customized and is working
-fine with my MSSQL installation. I am upgrading my Airflow version. Do I need to upgrade my provider,
-or can I keep it as it is?</strong></p>
-<p>It depends on the scope of customization. There is no need to upgrade the provider packages to later
-versions unless you want to upgrade to Airflow version that introduces backwards-incompatible changes.
-Generally speaking, with Airflow 2 we are following the <a class="reference external" href="https://semver.org/">Semver</a> approach where
-we will introduce backwards-incompatible changes in Major releases, so all your modifications (as long
-as you have not used internal Airflow classes) should work for All Airflow 2.* versions.</p>
</div>
-<div class="section" id="creating-your-own-providers">
-<h2>Creating your own providers<a class="headerlink" href="#creating-your-own-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When I write my own provider, do I need to do anything special to make it available to others?</strong></p>
-<p>You do not need to do anything special besides creating the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point
-returning properly formatted meta-data - dictionary with <code class="docutils literal notranslate"><span class="pre">extra-links</span></code> and <code class="docutils literal notranslate"><span class="pre">connection-types</span></code> fields
-(and deprecated <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> field if you are also targeting versions of Airflow before 2.2.0).</p>
-<p>Anyone who runs airflow in an environment that has your Python package installed will be able to use the
-package as a provider package.</p>
-<p><strong>What do I need to do to turn a package into a provider?</strong></p>
-<p>You need to do the following to turn an existing Python package into a provider (see below for examples):</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>location_uri</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Location URI used to determine the location type (S3, SMB, NFS, EFS).</p></li>
+<li><p><strong>create_location_kwargs</strong> – Passed to <code class="docutils literal notranslate"><span class="pre">DataSync.Client.create_location_*</span></code> methods.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>LocationArn of the created Location.</p>
+</dd>
+<dt class="field-odd">Raises</dt>
+<dd class="field-odd"><p><strong>AirflowException</strong> – If location type (prefix from <code class="docutils literal notranslate"><span class="pre">location_uri</span></code>) is invalid.</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.get_location_arns">
+<span class="sig-name descname"><span class="pre">get_location_arns</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">location_uri</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">case_sensitive</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ignore_trailing_slash</span>< [...]
+<dd><p>Return all LocationArns which match a LocationUri.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>location_uri</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Location URI to search for, eg <code class="docutils literal notranslate"><span class="pre">s3://mybucket/mypath</span></code></p></li>
+<li><p><strong>case_sensitive</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a>) – Do a case sensitive search for location URI.</p></li>
+<li><p><strong>ignore_trailing_slash</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a>) – Ignore / at the end of URI when matching.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>List of LocationArns.</p>
+</dd>
+<dt class="field-odd">Raises</dt>
+<dd class="field-odd"><p><strong>AirflowBadRequest</strong> – if <code class="docutils literal notranslate"><span class="pre">location_uri</span></code> is empty</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#list" title="(in Python v3.11)">list</a>[<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a>]</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.create_task">
+<span class="sig-name descname"><span class="pre">create_task</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">source_location_arn</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">destination_location_arn</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">create_task_kwargs</span></span></em><span class="sig-paren">)</span><a class="ref [...]
+<dd><p>Create a Task between the specified source and destination LocationArns.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Add the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point in the <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> - this tells airflow where to get
-the required provider metadata</p></li>
-<li><p>Create the function that you refer to in the first step as part of your package: this functions returns a
-dictionary that contains all meta-data about your provider package</p></li>
-<li><p>If you want Airflow to link to documentation of your Provider in the providers page, make sure
-to add “project-url/documentation” <a class="reference external" href="https://peps.python.org/pep-0621/#example">metadata</a> to your package.
-This will also add link to your documentation in PyPI.</p></li>
-<li><p>note that the dictionary should be compliant with <code class="docutils literal notranslate"><span class="pre">airflow/provider_info.schema.json</span></code> JSON-schema
-specification. The community-managed providers have more fields there that are used to build
-documentation, but the requirement for runtime information only contains several fields which are defined
-in the schema:</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/create_task.html#DataSync.Client.create_task" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.create_task()</span></code></a></p></li>
</ul>
-<div class="example-block-wrapper compound">
-<p class="example-header compound-first"><span class="example-title">airflow/provider_info.schema.json</span></p>
-<div class="compound-last highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"$schema"</span><span class="p">:</span><span class="w"> </span><span class="s2">"http://json-schema.org/draft-07/schema#"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"package-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Package name available under which the package is available in the PyPI repository."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Provider name"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Information about the package in RST format"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-names"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class names that provide connection types to core (deprecated by connection-types)"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"deprecated"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"The hook-class-names property has been deprecated in favour of connection-types which is more performant version allowing to only import individual Hooks rather than all hooks at once"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"deprecatedVersion"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.2.0"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"connection-types"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Map of connection types mapped to hook class names."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"connection-type"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Type of connection defined by the provider"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class name that implements the connection type"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"connection-type"</span><span class="p">,</span><span class="w"> </span><span class="s2">"hook-class-name"</span><span class="p">]</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"extra-links"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Operator class names that provide extra link functionality"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"secrets-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Secrets Backend class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"logging"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Logging Task Handlers class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"auth-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"API Auth Backend module names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"notifications"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Notification class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"task-decorators"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Apply custom decorators to the TaskFlow API. Can be accessed by users via '@task.<name>'"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"path"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
-<span class="w"> </span><span class="s2">"name"</span><span class="p">,</span>
-<span class="w"> </span><span class="s2">"description"</span>
-<span class="w"> </span><span class="p">]</span>
-<span class="p">}</span>
-</pre></div>
</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>source_location_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Source LocationArn. Must exist already.</p></li>
+<li><p><strong>destination_location_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Destination LocationArn. Must exist already.</p></li>
+<li><p><strong>create_task_kwargs</strong> – Passed to <code class="docutils literal notranslate"><span class="pre">boto.create_task()</span></code>. See AWS boto3 datasync documentation.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>TaskArn of the created Task</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.update_task">
+<span class="sig-name descname"><span class="pre">update_task</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task_arn</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">update_task_kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHo [...]
+<dd><p>Update a Task.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/update_task.html#DataSync.Client.update_task" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.update_task()</span></code></a></p></li>
+</ul>
</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>:</p>
-<div class="highlight-cfg notranslate"><div class="highlight"><pre><span></span><span class="k">[options.entry_points]</span>
-<span class="c1"># the function get_provider_info is defined in myproviderpackage.somemodule</span>
-<span class="na">apache_airflow_provider</span><span class="o">=</span>
-<span class="w"> </span><span class="na">provider_info</span><span class="o">=</span><span class="s">myproviderpackage.somemodule:get_provider_info</span>
-</pre></div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>task_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – The TaskArn to update.</p></li>
+<li><p><strong>update_task_kwargs</strong> – Passed to <code class="docutils literal notranslate"><span class="pre">boto.update_task()</span></code>, See AWS boto3 datasync documentation.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.delete_task">
+<span class="sig-name descname"><span class="pre">delete_task</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task_arn</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHook.delete_task"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws [...]
+<dd><p>Delete a Task.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/delete_task.html#DataSync.Client.delete_task" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.delete_task()</span></code></a></p></li>
+</ul>
</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">myproviderpackage/somemodule.py</span></code>:</p>
-<div class="highlight-Python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_provider_info</span><span class="p">():</span>
- <span class="k">return</span> <span class="p">{</span>
- <span class="s2">"package-name"</span><span class="p">:</span> <span class="s2">"my-package-name"</span><span class="p">,</span>
- <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"name"</span><span class="p">,</span>
- <span class="s2">"description"</span><span class="p">:</span> <span class="s2">"a description"</span><span class="p">,</span>
- <span class="s2">"hook-class-names"</span><span class="p">:</span> <span class="p">[</span>
- <span class="s2">"myproviderpackage.hooks.source.SourceHook"</span><span class="p">,</span>
- <span class="p">],</span>
- <span class="p">}</span>
-</pre></div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>task_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – The TaskArn to delete.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.get_task_arns_for_location_arns">
+<span class="sig-name descname"><span class="pre">get_task_arns_for_location_arns</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">source_location_arns</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">destination_location_arns</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHook.get_task [...]
+<dd><p>Return list of TaskArns for which use any one of the specified
+source LocationArns and any one of the specified destination LocationArns.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>source_location_arns</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#list" title="(in Python v3.11)"><em>list</em></a>) – List of source LocationArns.</p></li>
+<li><p><strong>destination_location_arns</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#list" title="(in Python v3.11)"><em>list</em></a>) – List of destination LocationArns.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Raises</dt>
+<dd class="field-even"><p><strong>AirflowBadRequest</strong> – if <code class="docutils literal notranslate"><span class="pre">source_location_arns</span></code> or <code class="docutils literal notranslate"><span class="pre">destination_location_arns</span></code> are empty.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.start_task_execution">
+<span class="sig-name descname"><span class="pre">start_task_execution</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task_arn</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHook. [...]
+<dd><p>Start a TaskExecution for the specified task_arn.
+Each task can have at most one TaskExecution.
+Additional keyword arguments send to <code class="docutils literal notranslate"><span class="pre">start_task_execution</span></code> boto3 method.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/start_task_execution.html#DataSync.Client.start_task_execution" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.start_task_execution()</span></code></a></p></li>
+</ul>
</div>
-<p><strong>How do provider packages work under the hood?</strong></p>
-<p>When running Airflow with your provider package, there will be (at least) three components to your airflow installation:</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>task_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – TaskArn</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>TaskExecutionArn</p>
+</dd>
+<dt class="field-odd">Raises</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>ClientError</strong> – If a TaskExecution is already busy running for this <code class="docutils literal notranslate"><span class="pre">task_arn</span></code>.</p></li>
+<li><p><strong>AirflowBadRequest</strong> – If <code class="docutils literal notranslate"><span class="pre">task_arn</span></code> is empty.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.cancel_task_execution">
+<span class="sig-name descname"><span class="pre">cancel_task_execution</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task_execution_arn</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHook.cancel_task_execution"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href= [...]
+<dd><p>Cancel a TaskExecution for the specified <code class="docutils literal notranslate"><span class="pre">task_execution_arn</span></code>.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>The installation itself (for example, a <code class="docutils literal notranslate"><span class="pre">venv</span></code> where you installed airflow with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">apache-airflow</span></code>)
-together with the related files (e.g. <code class="docutils literal notranslate"><span class="pre">dags</span></code> folder)</p></li>
-<li><p>The <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package</p></li>
-<li><p>Your own <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package that is independent of <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> or your airflow installation, which
-can be a local Python package (that you install via <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span> <span class="pre">/path/to/my-package</span></code>), a normal pip package
-(<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">myproviderpackage</span></code>), or any other type of Python package</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/cancel_task_execution.html#DataSync.Client.cancel_task_execution" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.cancel_task_execution()</span></code></a></p></li>
</ul>
-<p>In the <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package you need to add the entry point and provide the appropriate metadata as described above.
-If you have done that, airflow does the following at runtime:</p>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>task_execution_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – TaskExecutionArn.</p>
+</dd>
+<dt class="field-even">Raises</dt>
+<dd class="field-even"><p><strong>AirflowBadRequest</strong> – If <code class="docutils literal notranslate"><span class="pre">task_execution_arn</span></code> is empty.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.get_task_description">
+<span class="sig-name descname"><span class="pre">get_task_description</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task_arn</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHook.get_task_description"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.pr [...]
+<dd><p>Get description for the specified <code class="docutils literal notranslate"><span class="pre">task_arn</span></code>.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Loop through ALL packages installed in your environment / <code class="docutils literal notranslate"><span class="pre">venv</span></code></p></li>
-<li><p>For each package, if the package’s <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> has a section <code class="docutils literal notranslate"><span class="pre">[options.entry_points]</span></code>, and if that section has a value
-for <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code>, then get the value for <code class="docutils literal notranslate"><span class="pre">provider_info</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code></p></li>
-<li><p>That value works like an import statement: <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code> translates to something like
-<code class="docutils literal notranslate"><span class="pre">from</span> <span class="pre">myproviderpackage.somemodule</span> <span class="pre">import</span> <span class="pre">get_provider_info</span></code>, and the <code class="docutils literal notranslate"><span class="pre">get_provider_info</span></code> that is being imported should be a
-callable, i.e. a function</p></li>
-<li><p>This function should return a dictionary with metadata</p></li>
-<li><p>If you have custom connection types as part of your package, that metadata will including a field called <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>
-which should be a list of strings of your custom hooks - those strings should also be in an import-like format, e.g.
-<code class="docutils literal notranslate"><span class="pre">myproviderpackage.hooks.source.SourceHook</span></code> means that there is a class <code class="docutils literal notranslate"><span class="pre">SourceHook</span></code> in <code class="docutils literal notranslate"><span class="pre">myproviderpackage/hooks/source.py</span></code>
-- airflow then imports these hooks and looks for the functions <code class="docutils literal notranslate"><span class="pre">get_ui_field_behaviour</span></code> and <code class="docutils literal notranslate"><span class="pre">get_connection_form_widgets</span></code>
-(both optional) as well as the attributes <code class="docutils literal notranslate"><span class="pre">conn_type</span></code> and <code class="docutils literal notranslate"><span class="pre">hook_name</span></code> to create the custom connection type in the airflow UI</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/describe_task.html#DataSync.Client.describe_task" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.describe_task()</span></code></a></p></li>
</ul>
-<p><strong>Should I name my provider specifically or should it be created in ``airflow.providers`` package?</strong></p>
-<p>We have quite a number (>60) of providers managed by the community and we are going to maintain them
-together with Apache Airflow. All those providers have well-defined structured and follow the
-naming conventions we defined and they are all in <code class="docutils literal notranslate"><span class="pre">airflow.providers</span></code> package. If your intention is
-to contribute your provider, then you should follow those conventions and make a PR to Apache Airflow
-to contribute to it. But you are free to use any package name as long as there are no conflicts with other
-names, so preferably choose package that is in your “domain”.</p>
-<p><strong>Is there a convention for a connection id and type?</strong></p>
-<p>Very good question. Glad that you asked. We usually follow the convention <code class="docutils literal notranslate"><span class="pre"><NAME>_default</span></code> for connection
-id and just <code class="docutils literal notranslate"><span class="pre"><NAME></span></code> for connection type. Few examples:</p>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>task_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – TaskArn</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>AWS metadata about a task.</p>
+</dd>
+<dt class="field-odd">Raises</dt>
+<dd class="field-odd"><p><strong>AirflowBadRequest</strong> – If <code class="docutils literal notranslate"><span class="pre">task_arn</span></code> is empty.</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.describe_task_execution">
+<span class="sig-name descname"><span class="pre">describe_task_execution</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task_execution_arn</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHook.describe_task_execution"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" h [...]
+<dd><p>Get description for the specified <code class="docutils literal notranslate"><span class="pre">task_execution_arn</span></code>.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">google_cloud_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">google_cloud_platform</span></code> type</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">aws_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">aws</span></code> type</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/datasync/client/describe_task_execution.html#DataSync.Client.describe_task_execution" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DataSync.Client.describe_task_execution()</span></code></a></p></li>
</ul>
-<p>You should follow this convention. It is important, to use unique names for connection type,
-so it should be unique for your provider. If two providers try to add connection with the same type
-only one of them will succeed.</p>
-<p><strong>Can I contribute my own provider to Apache Airflow?</strong></p>
-<p>Of course, but it’s better to check at developer’s mailing list whether such contribution will be accepted by
-the Community, before investing time to make the provider compliant with community requirements.
-The Community only accepts providers that are generic enough, are well documented, fully covered by tests
-and with capabilities of being tested by people in the community. So we might not always be in the
-position to accept such contributions.</p>
-<p>After you think that your provider matches the expected values above, you can read
-<a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a> to check all prerequisites for a new
-community Provider and discuss it at the <a class="reference external" href="http://airflow.apache.org/community/">Devlist</a>.</p>
-<p>However, in case you have your own, specific provider, which you can maintain on your own or by your
-team, you are free to publish the providers in whatever form you find appropriate. The custom and
-community-managed providers have exactly the same capabilities.</p>
-<p><strong>Can I advertise my own provider to Apache Airflow users and share it with others as package in PyPI?</strong></p>
-<p>Absolutely! We have an <a class="reference external" href="https://airflow.apache.org/ecosystem/">Ecosystem</a> area on our website where
-we share non-community managed extensions and work for Airflow. Feel free to make a PR to the page and
-add we will evaluate and merge it when we see that such provider can be useful for the community of
-Airflow users.</p>
-<p><strong>Can I charge for the use of my provider?</strong></p>
-<p>This is something that is outside of our control and domain. As an Apache project, we are
-commercial-friendly and there are many businesses built around Apache Airflow and many other
-Apache projects. As a community, we provide all the software for free and this will never
-change. What 3rd-party developers are doing is not under control of Apache Airflow community.</p>
</div>
-<div class="section" id="using-backport-providers-in-airflow-1-10">
-<h2>Using Backport Providers in Airflow 1.10<a class="headerlink" href="#using-backport-providers-in-airflow-1-10" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an Airflow version (1.10.12) running and it is stable. However, because of a Cloud provider change,
-I would like to upgrade the provider package. If I don’t need to upgrade the Airflow version anymore,
-how do I know that this provider version is compatible with my Airflow version?</strong></p>
-<p>We have Backport Providers are compatible with 1.10 but they stopped being released on
-March 17, 2021. Since then, no new changes to providers for Airflow 2.0 are going to be
-released as backport packages. It’s the highest time to upgrade to Airflow 2.0.</p>
-<p>When it comes to compatibility of providers with different Airflow 2 versions, each
-provider package will keep its own dependencies, and while we expect those providers to be generally
-backwards-compatible, particular versions of particular providers might introduce dependencies on
-specific Airflow versions.</p>
-<div class="toctree-wrapper compound">
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>task_execution_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – TaskExecutionArn</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>AWS metadata about a task execution.</p>
+</dd>
+<dt class="field-odd">Raises</dt>
+<dd class="field-odd"><p><strong>AirflowBadRequest</strong> – If <code class="docutils literal notranslate"><span class="pre">task_execution_arn</span></code> is empty.</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)">dict</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.get_current_task_execution_arn">
+<span class="sig-name descname"><span class="pre">get_current_task_execution_arn</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task_arn</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/datasync.html#DataSyncHook.get_current_task_execution_arn"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlin [...]
+<dd><p>Get current TaskExecutionArn (if one exists) for the specified <code class="docutils literal notranslate"><span class="pre">task_arn</span></code>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>task_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – TaskArn</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>CurrentTaskExecutionArn for this <code class="docutils literal notranslate"><span class="pre">task_arn</span></code> or None.</p>
+</dd>
+<dt class="field-odd">Raises</dt>
+<dd class="field-odd"><p><strong>AirflowBadRequest</strong> – if <code class="docutils literal notranslate"><span class="pre">task_arn</span></code> is empty.</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a> | None</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.wait_for_task_execution">
+<span class="sig-name descname"><span class="pre">wait_for_task_execution</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">task_execution_arn</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_iterations</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">60</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../. [...]
+<dd><p>Wait for Task Execution status to be complete (SUCCESS/ERROR).
+The <code class="docutils literal notranslate"><span class="pre">task_execution_arn</span></code> must exist, or a boto3 ClientError will be raised.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>task_execution_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – TaskExecutionArn</p></li>
+<li><p><strong>max_iterations</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.11)"><em>int</em></a>) – Maximum number of iterations before timing out.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Result of task execution.</p>
+</dd>
+<dt class="field-odd">Raises</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>AirflowTaskTimeout</strong> – If maximum iterations is exceeded.</p></li>
+<li><p><strong>AirflowBadRequest</strong> – If <code class="docutils literal notranslate"><span class="pre">task_execution_arn</span></code> is empty.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)">bool</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+</dd></dl>
+
</div>
</div>
</div>
- <div class="pager" role="navigation" aria-label="related navigation"><button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Previous</button>
- <a rel="next" title="Providers packages reference" href="packages-ref.html" >
+ <div class="pager" role="navigation" aria-label="related navigation">
+ <a rel="prev" title="airflow.providers.amazon.aws.hooks.cloud_formation" href="../cloud_formation/index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="p">Previous</button>
+ </a>
+ <a rel="next" title="airflow.providers.amazon.aws.hooks.dms" href="../dms/index.html" >
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="n">Next</button>
</a>
</div>
@@ -1044,28 +1036,31 @@ specific Airflow versions.</p>
<nav class="wy-nav-side-toc">
<div class="wy-menu-vertical">
<ul>
-<li><a class="reference internal" href="#">Provider packages</a></li>
-<li><a class="reference internal" href="#extending-airflow-core-functionality">Extending Airflow core functionality</a><ul>
-<li><a class="reference internal" href="#auth-backends">Auth backends</a></li>
-<li><a class="reference internal" href="#custom-connections">Custom connections</a></li>
-<li><a class="reference internal" href="#extra-links">Extra links</a></li>
-<li><a class="reference internal" href="#logging">Logging</a></li>
-<li><a class="reference internal" href="#secret-backends">Secret backends</a></li>
-<li><a class="reference internal" href="#notifications">Notifications</a></li>
+<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.datasync</span></code></a><ul>
+<li><a class="reference internal" href="#module-contents">Module Contents</a><ul>
+<li><a class="reference internal" href="#classes">Classes</a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook"><code class="docutils literal notranslate"><span class="pre">DataSyncHook</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.TASK_EXECUTION_INTERMEDIATE_STATES"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.TASK_EXECUTION_INTERMEDIATE_STATES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.TASK_EXECUTION_FAILURE_STATES"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.TASK_EXECUTION_FAILURE_STATES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.TASK_EXECUTION_SUCCESS_STATES"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.TASK_EXECUTION_SUCCESS_STATES</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.create_location"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.create_location()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.get_location_arns"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.get_location_arns()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.create_task"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.create_task()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.update_task"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.update_task()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.delete_task"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.delete_task()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.get_task_arns_for_location_arns"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.get_task_arns_for_location_arns()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.start_task_execution"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.start_task_execution()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.cancel_task_execution"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.cancel_task_execution()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.get_task_description"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.get_task_description()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.describe_task_execution"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.describe_task_execution()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.get_current_task_execution_arn"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.get_current_task_execution_arn()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.datasync.DataSyncHook.wait_for_task_execution"><code class="docutils literal notranslate"><span class="pre">DataSyncHook.wait_for_task_execution()</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
-<li><a class="reference internal" href="#installing-and-upgrading-providers">Installing and upgrading providers</a></li>
-<li><a class="reference internal" href="#types-of-providers">Types of providers</a><ul>
-<li><a class="reference internal" href="#community-maintained-providers">Community maintained providers</a></li>
-<li><a class="reference internal" href="#custom-provider-packages">Custom provider packages</a></li>
</ul>
</li>
-<li><a class="reference internal" href="#how-to-create-your-own-provider">How to create your own provider</a></li>
-<li><a class="reference internal" href="#faq-for-airflow-and-providers">FAQ for Airflow and Providers</a><ul>
-<li><a class="reference internal" href="#upgrading-airflow-2-0-and-providers">Upgrading Airflow 2.0 and Providers</a></li>
-<li><a class="reference internal" href="#customizing-provider-packages">Customizing Provider Packages</a></li>
-<li><a class="reference internal" href="#creating-your-own-providers">Creating your own providers</a></li>
-<li><a class="reference internal" href="#using-backport-providers-in-airflow-1-10">Using Backport Providers in Airflow 1.10</a></li>
</ul>
</li>
</ul>
@@ -1090,7 +1085,7 @@ specific Airflow versions.</p>
<div class="base-layout--button">
- <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers/index.rst" rel="nofollow">
+ <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-amazon/_api/airflow/providers/amazon/aws/hooks/datasync/index.rst" rel="nofollow">
<button class="btn-hollow btn-brown btn-with-icon with-box-shadow button-fixed">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
@@ -1224,19 +1219,19 @@ specific Airflow versions.</p>
</span>
</div>
</footer>
- <script type="text/javascript" src="_static/_gen/js/docs.js"></script>
- <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
+ <script type="text/javascript" src="../../../../../../../_static/_gen/js/docs.js"></script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../../../../" src="../../../../../../../_static/documentation_options.js"></script>
<script src="https://code.jquery.com/jquery-3.4.1.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js" integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy" crossorigin="anonymous"></script>
- <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
- <script src="_static/jquery.js"></script>
- <script src="_static/underscore.js"></script>
- <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
- <script src="_static/doctools.js"></script>
- <script src="_static/sphinx_highlight.js"></script>
- <script src="_static/clipboard.min.js"></script>
- <script src="_static/copybutton.js"></script>
- <script src="_static/js/globaltoc.js"></script>
+ <script data-url_root="../../../../../../../" id="documentation_options" src="../../../../../../../_static/documentation_options.js"></script>
+ <script src="../../../../../../../_static/jquery.js"></script>
+ <script src="../../../../../../../_static/underscore.js"></script>
+ <script src="../../../../../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
+ <script src="../../../../../../../_static/doctools.js"></script>
+ <script src="../../../../../../../_static/sphinx_highlight.js"></script>
+ <script src="../../../../../../../_static/clipboard.min.js"></script>
+ <script src="../../../../../../../_static/copybutton.js"></script>
+ <script src="../../../../../../../_static/js/globaltoc.js"></script>
</body>
</html>
\ No newline at end of file
diff --git a/docs-archive/apache-airflow-providers/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/dms/index.html
similarity index 60%
copy from docs-archive/apache-airflow-providers/index.html
copy to docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/dms/index.html
index 70eef65adf..19d58e0ad4 100644
--- a/docs-archive/apache-airflow-providers/index.html
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/dms/index.html
@@ -9,18 +9,20 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Provider packages — apache-airflow-providers Documentation</title>
- <link rel="stylesheet" href="_static/_gen/css/main.min.css" type="text/css" />
- <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main.min.css" />
- <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main-custom.min.css" />
- <link rel="shortcut icon" href="_static/pin_32.png"/>
- <link rel="index" title="Index" href="genindex.html" />
- <link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Providers packages reference" href="packages-ref.html" />
-
- <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers/stable/index.html" />
+ <title>airflow.providers.amazon.aws.hooks.dms — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../../search.html" />
+ <link rel="next" title="airflow.providers.amazon.aws.hooks.dynamodb" href="../dynamodb/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.hooks.datasync" href="../datasync/index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/hooks/dms/index.html" />
<!-- Matomo -->
<script>
@@ -282,8 +284,21 @@
<div class="content-drawer">
<div class="td-sidebar">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -313,14 +328,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -401,8 +434,21 @@
<div class="td-sidebar desktop-only d-print-none">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -432,14 +478,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -528,9 +592,15 @@
<ul class="breadcrumb">
- <li class="breadcrumb-item"><a href="#" class="icon icon-home"> Home</a></li>
+ <li class="breadcrumb-item"><a href="../../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks</span></code></a></li>
- <li class="breadcrumb-item"><a href="index.html"> Provider packages</a></li>
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.dms</span></code></a></li>
</ul>
</div>
@@ -541,440 +611,240 @@
<div class="bodywrapper">
<div class="body" role="main">
- <blockquote>
-<div></div></blockquote>
-<div class="section" id="provider-packages">
-<h1>Provider packages<a class="headerlink" href="#provider-packages" title="Permalink to this heading">¶</a></h1>
-<p>Apache Airflow 2 is built in modular way. The “Core” of Apache Airflow provides core scheduler
-functionality which allow you to write some basic tasks, but the capabilities of Apache Airflow can
-be extended by installing additional packages, called <code class="docutils literal notranslate"><span class="pre">providers</span></code>.</p>
-<p>Providers can contain operators, hooks, sensor, and transfer operators to communicate with a
-multitude of external systems, but they can also extend Airflow core with new capabilities.</p>
-<p>You can install those provider packages separately in order to interface with a given service. The providers
-for <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> are designed in the way that you can write your own providers easily. The
-<code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span> <span class="pre">Community</span></code> develops and maintain more than 60 provider packages, but you are free to
-develop your own providers - the providers you build have exactly the same capability as the providers
-written by the community, so you can release and share those providers with others.</p>
-<p>The full list of community managed providers is available at
-<a class="reference external" href="https://airflow.apache.org/docs/#providers-packages-docs-apache-airflow-providers-index-html">Providers Index</a>.</p>
-<p>You can also see index of all community provider’s operators and hooks in
-<a class="reference internal" href="operators-and-hooks-ref/index.html"><span class="doc">Operators and Hooks Reference</span></a></p>
-</div>
-<div class="section" id="extending-airflow-core-functionality">
-<h1>Extending Airflow core functionality<a class="headerlink" href="#extending-airflow-core-functionality" title="Permalink to this heading">¶</a></h1>
-<p>Providers give you the capability of extending core Airflow with extra capabilities. The Core airflow
-provides basic and solid functionality of scheduling, the providers extend its capabilities. Here we
-describe all the custom capabilities.</p>
-<p>Airflow automatically discovers which providers add those additional capabilities and, once you install
-provider package and re-start Airflow, those become automatically available to Airflow Users.</p>
-<p>The summary of the core functionalities that can be extended are available in
-<a class="reference internal" href="core-extensions/index.html"><span class="doc">Core Extensions</span></a>.</p>
-<div class="section" id="auth-backends">
-<h2>Auth backends<a class="headerlink" href="#auth-backends" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom authentication backends, that allow you to configure the way how your
-web server authenticates your users, integrating it with public or private authentication services.</p>
-<p>You can see all the authentication backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/auth-backends.html"><span class="doc">Auth backends</span></a></p>
-</div>
-<div class="section" id="custom-connections">
-<h2>Custom connections<a class="headerlink" href="#custom-connections" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom connection types, extending connection form and handling custom form field
-behaviour for the connections defined by the provider.</p>
-<p>You can see all custom connections available via community-managed providers in
-<a class="reference internal" href="core-extensions/connections.html"><span class="doc">Connections</span></a>.</p>
-</div>
-<div class="section" id="extra-links">
-<h2>Extra links<a class="headerlink" href="#extra-links" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add extra custom links to operators delivered by the provider. Those will be visible in
-task details view of the task.</p>
-<p>You can see all the extra links available via community-managed providers in
-<a class="reference internal" href="core-extensions/extra-links.html"><span class="doc">Extra Links</span></a>.</p>
-</div>
-<div class="section" id="logging">
-<h2>Logging<a class="headerlink" href="#logging" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add additional task logging capabilities. By default <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> saves logs for
-tasks locally and make them available to Airflow UI via internal http server. However, providers
-can add extra logging capabilities, where Airflow Logs can be written to a remote service and
-retrieved from those services.</p>
-<p>You can see all task loggers available via community-managed providers in
-<a class="reference internal" href="core-extensions/logging.html"><span class="doc">Writing logs</span></a>.</p>
-</div>
-<div class="section" id="secret-backends">
-<h2>Secret backends<a class="headerlink" href="#secret-backends" title="Permalink to this heading">¶</a></h2>
-<p>Airflow has the capability of reading connections, variables and configuration from Secret Backends rather
-than from its own Database.</p>
-<p>You can see all secret backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/secrets-backends.html"><span class="doc">Secret backends</span></a>.</p>
-</div>
-<div class="section" id="notifications">
-<h2>Notifications<a class="headerlink" href="#notifications" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom notifications, that allow you to configure the way how you would like to receive
-notifications about the status of your tasks/DAGs.</p>
-<p>You can see all the notifications available via community-managed providers in
-<a class="reference internal" href="core-extensions/notifications.html"><span class="doc">Notifications</span></a>.</p>
-</div>
-</div>
-<div class="section" id="installing-and-upgrading-providers">
-<h1>Installing and upgrading providers<a class="headerlink" href="#installing-and-upgrading-providers" title="Permalink to this heading">¶</a></h1>
-<p>Separate provider packages give the possibilities that were not available in 1.10:</p>
-<ol class="arabic simple">
-<li><p>You can upgrade to latest version of particular providers without the need of Apache Airflow core upgrade.</p></li>
-<li><p>You can downgrade to previous version of particular provider in case the new version introduces
-some problems, without impacting the main Apache Airflow core package.</p></li>
-<li><p>You can release and upgrade/downgrade provider packages incrementally, independent from each other. This
-means that you can incrementally validate each of the provider package update in your environment,
-following the usual tests you have in your environment.</p></li>
-</ol>
-</div>
-<div class="section" id="types-of-providers">
-<h1>Types of providers<a class="headerlink" href="#types-of-providers" title="Permalink to this heading">¶</a></h1>
-<p>Providers have the same capacity - no matter if they are provided by the community or if they are
-third-party providers. This chapter explains how community managed providers are versioned and released
-and how you can create your own providers.</p>
-<div class="section" id="community-maintained-providers">
-<h2>Community maintained providers<a class="headerlink" href="#community-maintained-providers" title="Permalink to this heading">¶</a></h2>
-<p>From the point of view of the community, Airflow is delivered in multiple, separate packages.
-The core of Airflow scheduling system is delivered as <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package and there are more than
-60 provider packages which can be installed separately as so called <code class="docutils literal notranslate"><span class="pre">Airflow</span> <span class="pre">Provider</span> <span class="pre">packages</span></code>.
-Those packages are available as <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers</span></code> packages - for example there is an
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> or <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> package).</p>
-<p>Community maintained providers are released and versioned separately from the Airflow releases. We are
-following the <a class="reference external" href="https://semver.org/">Semver</a> versioning scheme for the packages. Some versions of the
-provider packages might depend on particular versions of Airflow, but the general approach we have is that
-unless there is a good reason, new version of providers should work with recent versions of Airflow 2.x.
-Details will vary per-provider and if there is a limitation for particular version of particular provider,
-constraining the Airflow version used, it will be included as limitation of dependencies in the provider
-package.</p>
-<p>Each community provider has corresponding extra which can be used when installing airflow to install the
-provider together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> - for example you can install airflow with those extras:
-<code class="docutils literal notranslate"><span class="pre">apache-airflow[google,amazon]</span></code> (with correct constraints -see <a class="reference external" href="/docs/apache-airflow/stable/installation/index.html" title="(in apache-airflow v2.6.0.dev0)"><span>Installation</span></a>) and you
-will install the appropriate versions of the <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> and
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> packages together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>.</p>
-<p>Some of the community providers have cross-provider dependencies as well. Those are not required
-dependencies, they might simply enable certain features (for example transfer operators often create
-dependency between different providers. Again, the general approach here is that the providers are backwards
-compatible, including cross-dependencies. Any kind of breaking changes and requirements on particular versions of other
-provider packages are automatically documented in the release notes of every provider.</p>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>For Airflow 1.10 We also provided <code class="docutils literal notranslate"><span class="pre">apache-airflow-backport-providers</span></code> packages that could be installed
-with those versions Those were the same providers as for 2.0 but automatically back-ported to work for
-Airflow 1.10. The last release of backport providers was done on March 17, 2021 and the backport
-providers will no longer be released, since Airflow 1.10 has reached End-Of-Life as of June 17, 2021.</p>
-</div>
-<p>If you want to contribute to <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>, you can see how to build and extend community
-managed providers in <a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a>.</p>
-</div>
-<div class="section" id="custom-provider-packages">
-<span id="providers-community-maintained-providers"></span><h2>Custom provider packages<a class="headerlink" href="#custom-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p>You can develop and release your own providers. Your custom operators, hooks, sensors, transfer operators
-can be packaged together in a standard airflow package and installed using the same mechanisms.
-Moreover they can also use the same mechanisms to extend the Airflow Core with auth backends,
-custom connections, logging, secret backends and extra operator links as described in the previous chapter.</p>
-</div>
-</div>
-<div class="section" id="how-to-create-your-own-provider">
-<h1>How to create your own provider<a class="headerlink" href="#how-to-create-your-own-provider" title="Permalink to this heading">¶</a></h1>
-<p>As mentioned in the <a class="reference external" href="http://airflow.apache.org/docs/apache-airflow-providers/index.html">Providers</a>
-documentation, custom providers can extend Airflow core - they can add extra links to operators as well
-as custom connections. You can use build your own providers and install them as packages if you would like
-to use the mechanism for your own, custom providers.</p>
-<p>Adding a provider to Airflow is just a matter of building a Python package and adding the right meta-data to
-the package. We are using standard mechanism of python to define
-<a class="reference external" href="https://docs.python.org/3/library/importlib.metadata.html#entry-points">entry points</a> . Your package
-needs to define appropriate entry-point <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> which has to point to a callable
-implemented by your package and return a dictionary containing the list of discoverable capabilities
-of your package. The dictionary has to follow the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json">json-schema specification</a>.</p>
-<p>Most of the schema provides extension point for the documentation (which you might want to also use for
-your own purpose) but the important fields from the extensibility point of view are those:</p>
-<p>Displaying package information in CLI/API:</p>
+ <div class="section" id="module-airflow.providers.amazon.aws.hooks.dms">
+<span id="airflow-providers-amazon-aws-hooks-dms"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.hooks.dms" title="airflow.providers.amazon.aws.hooks.dms"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.dms</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.hooks.dms" title="Permalink to this heading">¶</a></h1>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<div class="section" id="classes">
+<h3>Classes<a class="headerlink" href="#classes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus" title="airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus"><code class="xref py py-obj docutils literal notranslate"><span class="pre">DmsTaskWaiterStatus</span></code></a></p></td>
+<td><p>Available AWS DMS Task Waiter statuses.</p></td>
+</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook" title="airflow.providers.amazon.aws.hooks.dms.DmsHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">DmsHook</span></code></a></p></td>
+<td><p>Interact with AWS Database Migration Service (DMS).</p></td>
+</tr>
+</tbody>
+</table>
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.dms.</span></span><span class="sig-name descname"><span class="pre">DmsTaskWaiterStatus</span></span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsTaskWaiterStatus"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="heade [...]
+<dd><p>Bases: <a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><code class="xref py py-obj docutils literal notranslate"><span class="pre">str</span></code></a>, <a class="reference external" href="https://docs.python.org/3/library/enum.html#enum.Enum" title="(in Python v3.11)"><code class="xref py py-obj docutils literal notranslate"><span class="pre">enum.Enum</span></code></a></p>
+<p>Available AWS DMS Task Waiter statuses.</p>
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus.DELETED">
+<span class="sig-name descname"><span class="pre">DELETED</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'deleted'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsTaskWaiterStatus.DELETED"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus.READY">
+<span class="sig-name descname"><span class="pre">READY</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'ready'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsTaskWaiterStatus.READY"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon.aws.h [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus.RUNNING">
+<span class="sig-name descname"><span class="pre">RUNNING</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'running'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsTaskWaiterStatus.RUNNING"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus.STOPPED">
+<span class="sig-name descname"><span class="pre">STOPPED</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'stopped'</span></em><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsTaskWaiterStatus.STOPPED"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers.amazon [...]
+<dd></dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.dms.</span></span><span class="sig-name descname"><span class="pre">DmsHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></s [...]
+<dd><p>Bases: <a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p>
+<p>Interact with AWS Database Migration Service (DMS).
+Provide thin wrapper around
+<a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dms.html#DatabaseMigrationService.Client" title="(in Boto3 v1.26.120)"><code class="xref py py-class docutils literal notranslate"><span class="pre">boto3.client("dms")</span></code></a>.</p>
+<p>Additional arguments (such as <code class="docutils literal notranslate"><span class="pre">aws_conn_id</span></code>) may be specified and
+are passed down to the underlying AwsBaseHook.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">package-name</span></code> - Name of the package for the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code> - Human-friendly name of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">description</span></code> - Additional description of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">version</span></code> - List of versions of the package (in reverse-chronological order). The first version in the
-list is the current package version. It is taken from the version of package installed, not from the
-provider_info information.</p></li>
+<li><p><a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-class docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p></li>
</ul>
-<p>Exposing customized functionality to the Airflow’s core:</p>
+</div>
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsHook.describe_replication_tasks">
+<span class="sig-name descname"><span class="pre">describe_replication_tasks</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsHook.describe_replication_tasks"><span class="viewcode-link"><span class="pre">[source]</span> [...]
+<dd><p>Describe replication tasks</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">extra-links</span></code> - this field should contain the list of all operator class names that are adding extra links
-capability. See <a class="reference external" href="/docs/apache-airflow/stable/howto/define-extra-link.html" title="(in apache-airflow v2.6.0.dev0)"><span>Define an operator extra link</span></a> for description of how to add extra link
-capability to the operators of yours.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">connection-types</span></code> - this field should contain the list of all connection types together with hook
-class names implementing those custom connection types (providing custom extra fields and
-custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
-<code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> (deprecated) - this field should contain the list of all hook class names that provide
-custom connection types with custom extra fields and field behaviour. The <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array
-is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
-providers are targeting Airflow 2.2.0+ you do not have to include the <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array, if
-you want to also target earlier versions of Airflow 2, you should include both <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> and
-<code class="docutils literal notranslate"><span class="pre">connection-types</span></code> arrays. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details.</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dms/client/describe_replication_tasks.html#DatabaseMigrationService.Client.describe_replication_tasks" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DatabaseMigrationService.Client.describe_replication_tasks()</span></code></a></p></li>
</ul>
-<p>When your providers are installed you can query the installed providers and their capabilities with the
-<code class="docutils literal notranslate"><span class="pre">airflow</span> <span class="pre">providers</span></code> command. This way you can verify if your providers are properly recognized and whether
-they define the extensions properly. See <a class="reference external" href="/docs/apache-airflow/stable/cli-and-env-variables-ref.html" title="(in apache-airflow v2.6.0.dev0)"><span>Command Line Interface and Environment Variables Reference</span></a> for details of available CLI
-sub-commands.</p>
-<p>When you write your own provider, consider following the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#naming-conventions-for-provider-packages">Naming conventions for provider packages</a></p>
-</div>
-<div class="section" id="faq-for-airflow-and-providers">
-<h1>FAQ for Airflow and Providers<a class="headerlink" href="#faq-for-airflow-and-providers" title="Permalink to this heading">¶</a></h1>
-<div class="section" id="upgrading-airflow-2-0-and-providers">
-<h2>Upgrading Airflow 2.0 and Providers<a class="headerlink" href="#upgrading-airflow-2-0-and-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When upgrading to a new Airflow version such as 2.0, but possibly 2.0.1 and beyond, is the best practice
-to also upgrade provider packages at the same time?</strong></p>
-<p>It depends on your use case. If you have automated or semi-automated verification of your installation,
-that you can run a new version of Airflow including all provider packages, then definitely go for it.
-If you rely more on manual testing, it is advised that you upgrade in stages. Depending on your choice
-you can either upgrade all used provider packages first, and then upgrade Airflow Core or the other way
-round. The first approach - when you first upgrade all providers is probably safer, as you can do it
-incrementally, step-by-step replacing provider by provider in your environment.</p>
</div>
-<div class="section" id="customizing-provider-packages">
-<h2>Customizing Provider Packages<a class="headerlink" href="#customizing-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an older version of my provider package which we have lightly customized and is working
-fine with my MSSQL installation. I am upgrading my Airflow version. Do I need to upgrade my provider,
-or can I keep it as it is?</strong></p>
-<p>It depends on the scope of customization. There is no need to upgrade the provider packages to later
-versions unless you want to upgrade to Airflow version that introduces backwards-incompatible changes.
-Generally speaking, with Airflow 2 we are following the <a class="reference external" href="https://semver.org/">Semver</a> approach where
-we will introduce backwards-incompatible changes in Major releases, so all your modifications (as long
-as you have not used internal Airflow classes) should work for All Airflow 2.* versions.</p>
-</div>
-<div class="section" id="creating-your-own-providers">
-<h2>Creating your own providers<a class="headerlink" href="#creating-your-own-providers" title="Permalink to this heading">¶</a></h2>
-<p><strong>When I write my own provider, do I need to do anything special to make it available to others?</strong></p>
-<p>You do not need to do anything special besides creating the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point
-returning properly formatted meta-data - dictionary with <code class="docutils literal notranslate"><span class="pre">extra-links</span></code> and <code class="docutils literal notranslate"><span class="pre">connection-types</span></code> fields
-(and deprecated <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> field if you are also targeting versions of Airflow before 2.2.0).</p>
-<p>Anyone who runs airflow in an environment that has your Python package installed will be able to use the
-package as a provider package.</p>
-<p><strong>What do I need to do to turn a package into a provider?</strong></p>
-<p>You need to do the following to turn an existing Python package into a provider (see below for examples):</p>
+<dl class="field-list simple">
+<dt class="field-odd">Returns</dt>
+<dd class="field-odd"><p>Marker and list of replication tasks</p>
+</dd>
+<dt class="field-even">Return type</dt>
+<dd class="field-even"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#tuple" title="(in Python v3.11)">tuple</a>[<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a> | None, <a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#list" title="(in Python v3.11)">list</a>]</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsHook.find_replication_tasks_by_arn">
+<span class="sig-name descname"><span class="pre">find_replication_tasks_by_arn</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">replication_task_arn</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">without_settings</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em><span class="sig-paren">)</span><a class="reference internal" hre [...]
+<dd><p>Find and describe replication tasks by task ARN</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Add the <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> entry point in the <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> - this tells airflow where to get
-the required provider metadata</p></li>
-<li><p>Create the function that you refer to in the first step as part of your package: this functions returns a
-dictionary that contains all meta-data about your provider package</p></li>
-<li><p>If you want Airflow to link to documentation of your Provider in the providers page, make sure
-to add “project-url/documentation” <a class="reference external" href="https://peps.python.org/pep-0621/#example">metadata</a> to your package.
-This will also add link to your documentation in PyPI.</p></li>
-<li><p>note that the dictionary should be compliant with <code class="docutils literal notranslate"><span class="pre">airflow/provider_info.schema.json</span></code> JSON-schema
-specification. The community-managed providers have more fields there that are used to build
-documentation, but the requirement for runtime information only contains several fields which are defined
-in the schema:</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dms/client/describe_replication_tasks.html#DatabaseMigrationService.Client.describe_replication_tasks" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DatabaseMigrationService.Client.describe_replication_tasks()</span></code></a></p></li>
</ul>
-<div class="example-block-wrapper compound">
-<p class="example-header compound-first"><span class="example-title">airflow/provider_info.schema.json</span></p>
-<div class="compound-last highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"$schema"</span><span class="p">:</span><span class="w"> </span><span class="s2">"http://json-schema.org/draft-07/schema#"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"package-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Package name available under which the package is available in the PyPI repository."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Provider name"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Information about the package in RST format"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-names"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class names that provide connection types to core (deprecated by connection-types)"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"deprecated"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"The hook-class-names property has been deprecated in favour of connection-types which is more performant version allowing to only import individual Hooks rather than all hooks at once"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"deprecatedVersion"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2.2.0"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"connection-types"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Map of connection types mapped to hook class names."</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"object"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"connection-type"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Type of connection defined by the provider"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"hook-class-name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Hook class name that implements the connection type"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"connection-type"</span><span class="p">,</span><span class="w"> </span><span class="s2">"hook-class-name"</span><span class="p">]</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"extra-links"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Operator class names that provide extra link functionality"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"secrets-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Secrets Backend class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"logging"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Logging Task Handlers class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"auth-backends"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"API Auth Backend module names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"notifications"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Notification class names"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"task-decorators"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"array"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"description"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Apply custom decorators to the TaskFlow API. Can be accessed by users via '@task.<name>'"</span><span class="p">,</span>
-<span class="w"> </span><span class="nt">"items"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"path"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span>
-<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"string"</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">}</span>
-<span class="w"> </span><span class="p">},</span>
-<span class="w"> </span><span class="nt">"required"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
-<span class="w"> </span><span class="s2">"name"</span><span class="p">,</span>
-<span class="w"> </span><span class="s2">"description"</span>
-<span class="w"> </span><span class="p">]</span>
-<span class="p">}</span>
-</pre></div>
-</div>
</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code>:</p>
-<div class="highlight-cfg notranslate"><div class="highlight"><pre><span></span><span class="k">[options.entry_points]</span>
-<span class="c1"># the function get_provider_info is defined in myproviderpackage.somemodule</span>
-<span class="na">apache_airflow_provider</span><span class="o">=</span>
-<span class="w"> </span><span class="na">provider_info</span><span class="o">=</span><span class="s">myproviderpackage.somemodule:get_provider_info</span>
-</pre></div>
-</div>
-<p>Example <code class="docutils literal notranslate"><span class="pre">myproviderpackage/somemodule.py</span></code>:</p>
-<div class="highlight-Python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_provider_info</span><span class="p">():</span>
- <span class="k">return</span> <span class="p">{</span>
- <span class="s2">"package-name"</span><span class="p">:</span> <span class="s2">"my-package-name"</span><span class="p">,</span>
- <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"name"</span><span class="p">,</span>
- <span class="s2">"description"</span><span class="p">:</span> <span class="s2">"a description"</span><span class="p">,</span>
- <span class="s2">"hook-class-names"</span><span class="p">:</span> <span class="p">[</span>
- <span class="s2">"myproviderpackage.hooks.source.SourceHook"</span><span class="p">,</span>
- <span class="p">],</span>
- <span class="p">}</span>
-</pre></div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>replication_task_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Replication task arn</p></li>
+<li><p><strong>without_settings</strong> (<a class="reference external" href="https://docs.python.org/3/library/functions.html#bool" title="(in Python v3.11)"><em>bool</em></a><em> | </em><em>None</em>) – Indicates whether to return task information with settings.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>list of replication tasks that match the ARN</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsHook.get_task_status">
+<span class="sig-name descname"><span class="pre">get_task_status</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">replication_task_arn</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsHook.get_task_status"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflow.providers. [...]
+<dd><p>Retrieve task status.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>replication_task_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Replication task ARN</p>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Current task status</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a> | None</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsHook.create_replication_task">
+<span class="sig-name descname"><span class="pre">create_replication_task</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">replication_task_id</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">source_endpoint_arn</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">target_endpoint_arn</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">replication_instance_arn</spa [...]
+<dd><p>Create DMS replication task.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dms/client/create_replication_task.html#DatabaseMigrationService.Client.create_replication_task" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DatabaseMigrationService.Client.create_replication_task()</span></code></a></p></li>
+</ul>
</div>
-<p><strong>How do provider packages work under the hood?</strong></p>
-<p>When running Airflow with your provider package, there will be (at least) three components to your airflow installation:</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>replication_task_id</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Replication task id</p></li>
+<li><p><strong>source_endpoint_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Source endpoint ARN</p></li>
+<li><p><strong>target_endpoint_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Target endpoint ARN</p></li>
+<li><p><strong>replication_instance_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Replication instance ARN</p></li>
+<li><p><strong>table_mappings</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#dict" title="(in Python v3.11)"><em>dict</em></a>) – Table mappings</p></li>
+<li><p><strong>migration_type</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Migration type (‘full-load’|’cdc’|’full-load-and-cdc’), full-load by default.</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns</dt>
+<dd class="field-even"><p>Replication task ARN</p>
+</dd>
+<dt class="field-odd">Return type</dt>
+<dd class="field-odd"><p><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)">str</a></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsHook.start_replication_task">
+<span class="sig-name descname"><span class="pre">start_replication_task</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">replication_task_arn</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">start_replication_task_type</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class=" [...]
+<dd><p>Starts replication task.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>The installation itself (for example, a <code class="docutils literal notranslate"><span class="pre">venv</span></code> where you installed airflow with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">apache-airflow</span></code>)
-together with the related files (e.g. <code class="docutils literal notranslate"><span class="pre">dags</span></code> folder)</p></li>
-<li><p>The <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package</p></li>
-<li><p>Your own <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package that is independent of <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> or your airflow installation, which
-can be a local Python package (that you install via <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span> <span class="pre">/path/to/my-package</span></code>), a normal pip package
-(<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">myproviderpackage</span></code>), or any other type of Python package</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dms/client/start_replication_task.html#DatabaseMigrationService.Client.start_replication_task" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DatabaseMigrationService.Client.start_replication_task()</span></code></a></p></li>
+</ul>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>replication_task_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Replication task ARN</p></li>
+<li><p><strong>start_replication_task_type</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Replication task start type (default=’start-replication’)
+(‘start-replication’|’resume-processing’|’reload-target’)</p></li>
</ul>
-<p>In the <code class="docutils literal notranslate"><span class="pre">myproviderpackage</span></code> package you need to add the entry point and provide the appropriate metadata as described above.
-If you have done that, airflow does the following at runtime:</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsHook.stop_replication_task">
+<span class="sig-name descname"><span class="pre">stop_replication_task</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">replication_task_arn</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsHook.stop_replication_task"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#airflo [...]
+<dd><p>Stops replication task.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p>Loop through ALL packages installed in your environment / <code class="docutils literal notranslate"><span class="pre">venv</span></code></p></li>
-<li><p>For each package, if the package’s <code class="docutils literal notranslate"><span class="pre">setup.cfg</span></code> has a section <code class="docutils literal notranslate"><span class="pre">[options.entry_points]</span></code>, and if that section has a value
-for <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code>, then get the value for <code class="docutils literal notranslate"><span class="pre">provider_info</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code></p></li>
-<li><p>That value works like an import statement: <code class="docutils literal notranslate"><span class="pre">myproviderpackage.somemodule:get_provider_info</span></code> translates to something like
-<code class="docutils literal notranslate"><span class="pre">from</span> <span class="pre">myproviderpackage.somemodule</span> <span class="pre">import</span> <span class="pre">get_provider_info</span></code>, and the <code class="docutils literal notranslate"><span class="pre">get_provider_info</span></code> that is being imported should be a
-callable, i.e. a function</p></li>
-<li><p>This function should return a dictionary with metadata</p></li>
-<li><p>If you have custom connection types as part of your package, that metadata will including a field called <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>
-which should be a list of strings of your custom hooks - those strings should also be in an import-like format, e.g.
-<code class="docutils literal notranslate"><span class="pre">myproviderpackage.hooks.source.SourceHook</span></code> means that there is a class <code class="docutils literal notranslate"><span class="pre">SourceHook</span></code> in <code class="docutils literal notranslate"><span class="pre">myproviderpackage/hooks/source.py</span></code>
-- airflow then imports these hooks and looks for the functions <code class="docutils literal notranslate"><span class="pre">get_ui_field_behaviour</span></code> and <code class="docutils literal notranslate"><span class="pre">get_connection_form_widgets</span></code>
-(both optional) as well as the attributes <code class="docutils literal notranslate"><span class="pre">conn_type</span></code> and <code class="docutils literal notranslate"><span class="pre">hook_name</span></code> to create the custom connection type in the airflow UI</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dms/client/stop_replication_task.html#DatabaseMigrationService.Client.stop_replication_task" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DatabaseMigrationService.Client.stop_replication_task()</span></code></a></p></li>
</ul>
-<p><strong>Should I name my provider specifically or should it be created in ``airflow.providers`` package?</strong></p>
-<p>We have quite a number (>60) of providers managed by the community and we are going to maintain them
-together with Apache Airflow. All those providers have well-defined structured and follow the
-naming conventions we defined and they are all in <code class="docutils literal notranslate"><span class="pre">airflow.providers</span></code> package. If your intention is
-to contribute your provider, then you should follow those conventions and make a PR to Apache Airflow
-to contribute to it. But you are free to use any package name as long as there are no conflicts with other
-names, so preferably choose package that is in your “domain”.</p>
-<p><strong>Is there a convention for a connection id and type?</strong></p>
-<p>Very good question. Glad that you asked. We usually follow the convention <code class="docutils literal notranslate"><span class="pre"><NAME>_default</span></code> for connection
-id and just <code class="docutils literal notranslate"><span class="pre"><NAME></span></code> for connection type. Few examples:</p>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>replication_task_arn</strong> – Replication task ARN</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsHook.delete_replication_task">
+<span class="sig-name descname"><span class="pre">delete_replication_task</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">replication_task_arn</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsHook.delete_replication_task"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#ai [...]
+<dd><p>Starts replication task deletion and waits for it to be deleted.</p>
+<div class="admonition seealso">
+<p class="admonition-title">See also</p>
<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">google_cloud_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">google_cloud_platform</span></code> type</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">aws_default</span></code> id and <code class="docutils literal notranslate"><span class="pre">aws</span></code> type</p></li>
+<li><p><a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dms/client/delete_replication_task.html#DatabaseMigrationService.Client.delete_replication_task" title="(in Boto3 v1.26.120)"><code class="xref py py-meth docutils literal notranslate"><span class="pre">DatabaseMigrationService.Client.delete_replication_task()</span></code></a></p></li>
</ul>
-<p>You should follow this convention. It is important, to use unique names for connection type,
-so it should be unique for your provider. If two providers try to add connection with the same type
-only one of them will succeed.</p>
-<p><strong>Can I contribute my own provider to Apache Airflow?</strong></p>
-<p>Of course, but it’s better to check at developer’s mailing list whether such contribution will be accepted by
-the Community, before investing time to make the provider compliant with community requirements.
-The Community only accepts providers that are generic enough, are well documented, fully covered by tests
-and with capabilities of being tested by people in the community. So we might not always be in the
-position to accept such contributions.</p>
-<p>After you think that your provider matches the expected values above, you can read
-<a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a> to check all prerequisites for a new
-community Provider and discuss it at the <a class="reference external" href="http://airflow.apache.org/community/">Devlist</a>.</p>
-<p>However, in case you have your own, specific provider, which you can maintain on your own or by your
-team, you are free to publish the providers in whatever form you find appropriate. The custom and
-community-managed providers have exactly the same capabilities.</p>
-<p><strong>Can I advertise my own provider to Apache Airflow users and share it with others as package in PyPI?</strong></p>
-<p>Absolutely! We have an <a class="reference external" href="https://airflow.apache.org/ecosystem/">Ecosystem</a> area on our website where
-we share non-community managed extensions and work for Airflow. Feel free to make a PR to the page and
-add we will evaluate and merge it when we see that such provider can be useful for the community of
-Airflow users.</p>
-<p><strong>Can I charge for the use of my provider?</strong></p>
-<p>This is something that is outside of our control and domain. As an Apache project, we are
-commercial-friendly and there are many businesses built around Apache Airflow and many other
-Apache projects. As a community, we provide all the software for free and this will never
-change. What 3rd-party developers are doing is not under control of Apache Airflow community.</p>
</div>
-<div class="section" id="using-backport-providers-in-airflow-1-10">
-<h2>Using Backport Providers in Airflow 1.10<a class="headerlink" href="#using-backport-providers-in-airflow-1-10" title="Permalink to this heading">¶</a></h2>
-<p><strong>I have an Airflow version (1.10.12) running and it is stable. However, because of a Cloud provider change,
-I would like to upgrade the provider package. If I don’t need to upgrade the Airflow version anymore,
-how do I know that this provider version is compatible with my Airflow version?</strong></p>
-<p>We have Backport Providers are compatible with 1.10 but they stopped being released on
-March 17, 2021. Since then, no new changes to providers for Airflow 2.0 are going to be
-released as backport packages. It’s the highest time to upgrade to Airflow 2.0.</p>
-<p>When it comes to compatibility of providers with different Airflow 2 versions, each
-provider package will keep its own dependencies, and while we expect those providers to be generally
-backwards-compatible, particular versions of particular providers might introduce dependencies on
-specific Airflow versions.</p>
-<div class="toctree-wrapper compound">
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><p><strong>replication_task_arn</strong> – Replication task ARN</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dms.DmsHook.wait_for_task_status">
+<span class="sig-name descname"><span class="pre">wait_for_task_status</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">replication_task_arn</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">status</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="../../../../../../../_modules/airflow/providers/amazon/aws/hooks/dms.html#DmsHook.wait_for_task_status"><span class="viewcode-link [...]
+<dd><p>Waits for replication task to reach status.
+Supported statuses: deleted, ready, running, stopped.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters</dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>status</strong> (<a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus" title="airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus"><em>DmsTaskWaiterStatus</em></a>) – Status to wait for</p></li>
+<li><p><strong>replication_task_arn</strong> (<a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.11)"><em>str</em></a>) – Replication task ARN</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+</dd></dl>
+
</div>
</div>
</div>
- <div class="pager" role="navigation" aria-label="related navigation"><button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Previous</button>
- <a rel="next" title="Providers packages reference" href="packages-ref.html" >
+ <div class="pager" role="navigation" aria-label="related navigation">
+ <a rel="prev" title="airflow.providers.amazon.aws.hooks.datasync" href="../datasync/index.html" >
+ <button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="p">Previous</button>
+ </a>
+ <a rel="next" title="airflow.providers.amazon.aws.hooks.dynamodb" href="../dynamodb/index.html" >
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" accesskey="n">Next</button>
</a>
</div>
@@ -1044,28 +914,31 @@ specific Airflow versions.</p>
<nav class="wy-nav-side-toc">
<div class="wy-menu-vertical">
<ul>
-<li><a class="reference internal" href="#">Provider packages</a></li>
-<li><a class="reference internal" href="#extending-airflow-core-functionality">Extending Airflow core functionality</a><ul>
-<li><a class="reference internal" href="#auth-backends">Auth backends</a></li>
-<li><a class="reference internal" href="#custom-connections">Custom connections</a></li>
-<li><a class="reference internal" href="#extra-links">Extra links</a></li>
-<li><a class="reference internal" href="#logging">Logging</a></li>
-<li><a class="reference internal" href="#secret-backends">Secret backends</a></li>
-<li><a class="reference internal" href="#notifications">Notifications</a></li>
+<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.dms</span></code></a><ul>
+<li><a class="reference internal" href="#module-contents">Module Contents</a><ul>
+<li><a class="reference internal" href="#classes">Classes</a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus"><code class="docutils literal notranslate"><span class="pre">DmsTaskWaiterStatus</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus.DELETED"><code class="docutils literal notranslate"><span class="pre">DmsTaskWaiterStatus.DELETED</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus.READY"><code class="docutils literal notranslate"><span class="pre">DmsTaskWaiterStatus.READY</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus.RUNNING"><code class="docutils literal notranslate"><span class="pre">DmsTaskWaiterStatus.RUNNING</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsTaskWaiterStatus.STOPPED"><code class="docutils literal notranslate"><span class="pre">DmsTaskWaiterStatus.STOPPED</span></code></a></li>
+</ul>
+</li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook"><code class="docutils literal notranslate"><span class="pre">DmsHook</span></code></a><ul>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook.describe_replication_tasks"><code class="docutils literal notranslate"><span class="pre">DmsHook.describe_replication_tasks()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook.find_replication_tasks_by_arn"><code class="docutils literal notranslate"><span class="pre">DmsHook.find_replication_tasks_by_arn()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook.get_task_status"><code class="docutils literal notranslate"><span class="pre">DmsHook.get_task_status()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook.create_replication_task"><code class="docutils literal notranslate"><span class="pre">DmsHook.create_replication_task()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook.start_replication_task"><code class="docutils literal notranslate"><span class="pre">DmsHook.start_replication_task()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook.stop_replication_task"><code class="docutils literal notranslate"><span class="pre">DmsHook.stop_replication_task()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook.delete_replication_task"><code class="docutils literal notranslate"><span class="pre">DmsHook.delete_replication_task()</span></code></a></li>
+<li><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dms.DmsHook.wait_for_task_status"><code class="docutils literal notranslate"><span class="pre">DmsHook.wait_for_task_status()</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
-<li><a class="reference internal" href="#installing-and-upgrading-providers">Installing and upgrading providers</a></li>
-<li><a class="reference internal" href="#types-of-providers">Types of providers</a><ul>
-<li><a class="reference internal" href="#community-maintained-providers">Community maintained providers</a></li>
-<li><a class="reference internal" href="#custom-provider-packages">Custom provider packages</a></li>
</ul>
</li>
-<li><a class="reference internal" href="#how-to-create-your-own-provider">How to create your own provider</a></li>
-<li><a class="reference internal" href="#faq-for-airflow-and-providers">FAQ for Airflow and Providers</a><ul>
-<li><a class="reference internal" href="#upgrading-airflow-2-0-and-providers">Upgrading Airflow 2.0 and Providers</a></li>
-<li><a class="reference internal" href="#customizing-provider-packages">Customizing Provider Packages</a></li>
-<li><a class="reference internal" href="#creating-your-own-providers">Creating your own providers</a></li>
-<li><a class="reference internal" href="#using-backport-providers-in-airflow-1-10">Using Backport Providers in Airflow 1.10</a></li>
</ul>
</li>
</ul>
@@ -1090,7 +963,7 @@ specific Airflow versions.</p>
<div class="base-layout--button">
- <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers/index.rst" rel="nofollow">
+ <a href="https://github.com/apache/airflow/edit/main/docs/apache-airflow-providers-amazon/_api/airflow/providers/amazon/aws/hooks/dms/index.rst" rel="nofollow">
<button class="btn-hollow btn-brown btn-with-icon with-box-shadow button-fixed">
<svg xmlns="http://www.w3.org/2000/svg" width="30.76" height="30">
@@ -1224,19 +1097,19 @@ specific Airflow versions.</p>
</span>
</div>
</footer>
- <script type="text/javascript" src="_static/_gen/js/docs.js"></script>
- <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
+ <script type="text/javascript" src="../../../../../../../_static/_gen/js/docs.js"></script>
+ <script type="text/javascript" id="documentation_options" data-url_root="../../../../../../../" src="../../../../../../../_static/documentation_options.js"></script>
<script src="https://code.jquery.com/jquery-3.4.1.min.js" integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js" integrity="sha384-ChfqqxuZUCnJSK3+MXmPNIyE6ZbWh2IMqE241rYiqJxyMiZ6OW/JmZQ5stwEULTy" crossorigin="anonymous"></script>
- <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
- <script src="_static/jquery.js"></script>
- <script src="_static/underscore.js"></script>
- <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
- <script src="_static/doctools.js"></script>
- <script src="_static/sphinx_highlight.js"></script>
- <script src="_static/clipboard.min.js"></script>
- <script src="_static/copybutton.js"></script>
- <script src="_static/js/globaltoc.js"></script>
+ <script data-url_root="../../../../../../../" id="documentation_options" src="../../../../../../../_static/documentation_options.js"></script>
+ <script src="../../../../../../../_static/jquery.js"></script>
+ <script src="../../../../../../../_static/underscore.js"></script>
+ <script src="../../../../../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
+ <script src="../../../../../../../_static/doctools.js"></script>
+ <script src="../../../../../../../_static/sphinx_highlight.js"></script>
+ <script src="../../../../../../../_static/clipboard.min.js"></script>
+ <script src="../../../../../../../_static/copybutton.js"></script>
+ <script src="../../../../../../../_static/js/globaltoc.js"></script>
</body>
</html>
\ No newline at end of file
diff --git a/docs-archive/apache-airflow-providers/index.html b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/dynamodb/index.html
similarity index 60%
copy from docs-archive/apache-airflow-providers/index.html
copy to docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/dynamodb/index.html
index 70eef65adf..011e71ddd9 100644
--- a/docs-archive/apache-airflow-providers/index.html
+++ b/docs-archive/apache-airflow-providers-amazon/8.0.0/_api/airflow/providers/amazon/aws/hooks/dynamodb/index.html
@@ -9,18 +9,20 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Provider packages — apache-airflow-providers Documentation</title>
- <link rel="stylesheet" href="_static/_gen/css/main.min.css" type="text/css" />
- <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main.min.css" />
- <link rel="stylesheet" type="text/css" href="_static/copybutton.css" />
- <link rel="stylesheet" type="text/css" href="_static/_gen/css/main-custom.min.css" />
- <link rel="shortcut icon" href="_static/pin_32.png"/>
- <link rel="index" title="Index" href="genindex.html" />
- <link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Providers packages reference" href="packages-ref.html" />
-
- <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers/stable/index.html" />
+ <title>airflow.providers.amazon.aws.hooks.dynamodb — apache-airflow-providers-amazon Documentation</title>
+ <link rel="stylesheet" href="../../../../../../../_static/_gen/css/main.min.css" type="text/css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/pygments.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/copybutton.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/_gen/css/main-custom.min.css" />
+ <link rel="stylesheet" type="text/css" href="../../../../../../../_static/graphviz.css" />
+ <link rel="shortcut icon" href="../../../../../../../_static/pin_32.png"/>
+ <link rel="index" title="Index" href="../../../../../../../genindex.html" />
+ <link rel="search" title="Search" href="../../../../../../../search.html" />
+ <link rel="next" title="airflow.providers.amazon.aws.hooks.ec2" href="../ec2/index.html" />
+ <link rel="prev" title="airflow.providers.amazon.aws.hooks.dms" href="../dms/index.html" />
+
+ <link rel="canonical" href="https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/_api/airflow/providers/amazon/aws/hooks/dynamodb/index.html" />
<!-- Matomo -->
<script>
@@ -282,8 +284,21 @@
<div class="content-drawer">
<div class="td-sidebar">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -313,14 +328,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -401,8 +434,21 @@
<div class="td-sidebar desktop-only d-print-none">
+<div id="docs-version-selector" class="docs-version-selector sidebar__version-selector">
+ <a class="dropdown-toggle" href="#" id="versionDropdown" role="button" data-toggle="dropdown" aria-haspopup="true"
+ aria-expanded="false">
+ <span class="bodytext__medium--greyish-brown">Version: </span><span class="version">8.0.0</span>
+ </a>
+ <div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
+
+ </div>
+ <script type="application/x-template" id="version-item-template">
+ <a class="dropdown-item"></a>
+ </script>
+</div>
+
<div class="searchb-box">
- <form class="search-form" action="search.html" method="get">
+ <form class="search-form" action="../../../../../../../search.html" method="get">
<input class="search-form__input" type="text" name="q" placeholder="Search docs" size="16">
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
@@ -432,14 +478,32 @@
<div class="toctree" role="navigation" aria-label="main navigation">
- <ul class="current">
-<li class="toctree-l1 current"><a class="current reference internal" href="#">Providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="packages-ref.html">Packages</a></li>
-<li class="toctree-l1"><a class="reference internal" href="operators-and-hooks-ref/index.html">Operators and hooks</a></li>
-<li class="toctree-l1"><a class="reference internal" href="core-extensions/index.html">Core Extensions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="howto/create-update-providers.html">Update community providers</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-sources.html">Installing from sources</a></li>
-<li class="toctree-l1"><a class="reference internal" href="installing-from-pypi.html">Installing from PyPI</a></li>
+ <p class="caption" role="heading"><span class="caption-text">Guides</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../connections/index.html">Connection types</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../operators/index.html">Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../transfer/index.html">Transfers</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../deferrable.html">Deferrable Operators</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../secrets-backends/index.html">Secrets backends</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../logging/index.html">Logging for Tasks</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">References</span></p>
+<ul class="current">
+<li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python API</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">System tests</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference external" href="https://aws-mwaa.github.io/open-source/system-tests/dashboard.html">System Tests Dashboard</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Resources</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../example-dags.html">Example DAGs</a></li>
+<li class="toctree-l1"><a class="reference external" href="https://pypi.org/project/apache-airflow-providers-amazon/">PyPI Repository</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../installing-providers-from-sources.html">Installing from sources</a></li>
+</ul>
+<p class="caption" role="heading"><span class="caption-text">Commits</span></p>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="../../../../../../../commits.html">Detailed list of commits</a></li>
</ul>
</div>
@@ -528,9 +592,15 @@
<ul class="breadcrumb">
- <li class="breadcrumb-item"><a href="#" class="icon icon-home"> Home</a></li>
+ <li class="breadcrumb-item"><a href="../../../../../../../index.html" class="icon icon-home"> Home</a></li>
+
+ <li class="breadcrumb-item"><a href="../../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="../../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws</span></code></a></li>
- <li class="breadcrumb-item"><a href="index.html"> Provider packages</a></li>
+ <li class="breadcrumb-item"><a href="../index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks</span></code></a></li>
+
+ <li class="breadcrumb-item"><a href="index.html"> <code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.dynamodb</span></code></a></li>
</ul>
</div>
@@ -541,440 +611,78 @@
<div class="bodywrapper">
<div class="body" role="main">
- <blockquote>
-<div></div></blockquote>
-<div class="section" id="provider-packages">
-<h1>Provider packages<a class="headerlink" href="#provider-packages" title="Permalink to this heading">¶</a></h1>
-<p>Apache Airflow 2 is built in modular way. The “Core” of Apache Airflow provides core scheduler
-functionality which allow you to write some basic tasks, but the capabilities of Apache Airflow can
-be extended by installing additional packages, called <code class="docutils literal notranslate"><span class="pre">providers</span></code>.</p>
-<p>Providers can contain operators, hooks, sensor, and transfer operators to communicate with a
-multitude of external systems, but they can also extend Airflow core with new capabilities.</p>
-<p>You can install those provider packages separately in order to interface with a given service. The providers
-for <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> are designed in the way that you can write your own providers easily. The
-<code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span> <span class="pre">Community</span></code> develops and maintain more than 60 provider packages, but you are free to
-develop your own providers - the providers you build have exactly the same capability as the providers
-written by the community, so you can release and share those providers with others.</p>
-<p>The full list of community managed providers is available at
-<a class="reference external" href="https://airflow.apache.org/docs/#providers-packages-docs-apache-airflow-providers-index-html">Providers Index</a>.</p>
-<p>You can also see index of all community provider’s operators and hooks in
-<a class="reference internal" href="operators-and-hooks-ref/index.html"><span class="doc">Operators and Hooks Reference</span></a></p>
-</div>
-<div class="section" id="extending-airflow-core-functionality">
-<h1>Extending Airflow core functionality<a class="headerlink" href="#extending-airflow-core-functionality" title="Permalink to this heading">¶</a></h1>
-<p>Providers give you the capability of extending core Airflow with extra capabilities. The Core airflow
-provides basic and solid functionality of scheduling, the providers extend its capabilities. Here we
-describe all the custom capabilities.</p>
-<p>Airflow automatically discovers which providers add those additional capabilities and, once you install
-provider package and re-start Airflow, those become automatically available to Airflow Users.</p>
-<p>The summary of the core functionalities that can be extended are available in
-<a class="reference internal" href="core-extensions/index.html"><span class="doc">Core Extensions</span></a>.</p>
-<div class="section" id="auth-backends">
-<h2>Auth backends<a class="headerlink" href="#auth-backends" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom authentication backends, that allow you to configure the way how your
-web server authenticates your users, integrating it with public or private authentication services.</p>
-<p>You can see all the authentication backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/auth-backends.html"><span class="doc">Auth backends</span></a></p>
-</div>
-<div class="section" id="custom-connections">
-<h2>Custom connections<a class="headerlink" href="#custom-connections" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom connection types, extending connection form and handling custom form field
-behaviour for the connections defined by the provider.</p>
-<p>You can see all custom connections available via community-managed providers in
-<a class="reference internal" href="core-extensions/connections.html"><span class="doc">Connections</span></a>.</p>
-</div>
-<div class="section" id="extra-links">
-<h2>Extra links<a class="headerlink" href="#extra-links" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add extra custom links to operators delivered by the provider. Those will be visible in
-task details view of the task.</p>
-<p>You can see all the extra links available via community-managed providers in
-<a class="reference internal" href="core-extensions/extra-links.html"><span class="doc">Extra Links</span></a>.</p>
-</div>
-<div class="section" id="logging">
-<h2>Logging<a class="headerlink" href="#logging" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add additional task logging capabilities. By default <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> saves logs for
-tasks locally and make them available to Airflow UI via internal http server. However, providers
-can add extra logging capabilities, where Airflow Logs can be written to a remote service and
-retrieved from those services.</p>
-<p>You can see all task loggers available via community-managed providers in
-<a class="reference internal" href="core-extensions/logging.html"><span class="doc">Writing logs</span></a>.</p>
-</div>
-<div class="section" id="secret-backends">
-<h2>Secret backends<a class="headerlink" href="#secret-backends" title="Permalink to this heading">¶</a></h2>
-<p>Airflow has the capability of reading connections, variables and configuration from Secret Backends rather
-than from its own Database.</p>
-<p>You can see all secret backends available via community-managed providers in
-<a class="reference internal" href="core-extensions/secrets-backends.html"><span class="doc">Secret backends</span></a>.</p>
-</div>
-<div class="section" id="notifications">
-<h2>Notifications<a class="headerlink" href="#notifications" title="Permalink to this heading">¶</a></h2>
-<p>The providers can add custom notifications, that allow you to configure the way how you would like to receive
-notifications about the status of your tasks/DAGs.</p>
-<p>You can see all the notifications available via community-managed providers in
-<a class="reference internal" href="core-extensions/notifications.html"><span class="doc">Notifications</span></a>.</p>
-</div>
-</div>
-<div class="section" id="installing-and-upgrading-providers">
-<h1>Installing and upgrading providers<a class="headerlink" href="#installing-and-upgrading-providers" title="Permalink to this heading">¶</a></h1>
-<p>Separate provider packages give the possibilities that were not available in 1.10:</p>
-<ol class="arabic simple">
-<li><p>You can upgrade to latest version of particular providers without the need of Apache Airflow core upgrade.</p></li>
-<li><p>You can downgrade to previous version of particular provider in case the new version introduces
-some problems, without impacting the main Apache Airflow core package.</p></li>
-<li><p>You can release and upgrade/downgrade provider packages incrementally, independent from each other. This
-means that you can incrementally validate each of the provider package update in your environment,
-following the usual tests you have in your environment.</p></li>
-</ol>
-</div>
-<div class="section" id="types-of-providers">
-<h1>Types of providers<a class="headerlink" href="#types-of-providers" title="Permalink to this heading">¶</a></h1>
-<p>Providers have the same capacity - no matter if they are provided by the community or if they are
-third-party providers. This chapter explains how community managed providers are versioned and released
-and how you can create your own providers.</p>
-<div class="section" id="community-maintained-providers">
-<h2>Community maintained providers<a class="headerlink" href="#community-maintained-providers" title="Permalink to this heading">¶</a></h2>
-<p>From the point of view of the community, Airflow is delivered in multiple, separate packages.
-The core of Airflow scheduling system is delivered as <code class="docutils literal notranslate"><span class="pre">apache-airflow</span></code> package and there are more than
-60 provider packages which can be installed separately as so called <code class="docutils literal notranslate"><span class="pre">Airflow</span> <span class="pre">Provider</span> <span class="pre">packages</span></code>.
-Those packages are available as <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers</span></code> packages - for example there is an
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> or <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> package).</p>
-<p>Community maintained providers are released and versioned separately from the Airflow releases. We are
-following the <a class="reference external" href="https://semver.org/">Semver</a> versioning scheme for the packages. Some versions of the
-provider packages might depend on particular versions of Airflow, but the general approach we have is that
-unless there is a good reason, new version of providers should work with recent versions of Airflow 2.x.
-Details will vary per-provider and if there is a limitation for particular version of particular provider,
-constraining the Airflow version used, it will be included as limitation of dependencies in the provider
-package.</p>
-<p>Each community provider has corresponding extra which can be used when installing airflow to install the
-provider together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code> - for example you can install airflow with those extras:
-<code class="docutils literal notranslate"><span class="pre">apache-airflow[google,amazon]</span></code> (with correct constraints -see <a class="reference external" href="/docs/apache-airflow/stable/installation/index.html" title="(in apache-airflow v2.6.0.dev0)"><span>Installation</span></a>) and you
-will install the appropriate versions of the <code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-amazon</span></code> and
-<code class="docutils literal notranslate"><span class="pre">apache-airflow-providers-google</span></code> packages together with <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>.</p>
-<p>Some of the community providers have cross-provider dependencies as well. Those are not required
-dependencies, they might simply enable certain features (for example transfer operators often create
-dependency between different providers. Again, the general approach here is that the providers are backwards
-compatible, including cross-dependencies. Any kind of breaking changes and requirements on particular versions of other
-provider packages are automatically documented in the release notes of every provider.</p>
-<div class="admonition note">
-<p class="admonition-title">Note</p>
-<p>For Airflow 1.10 We also provided <code class="docutils literal notranslate"><span class="pre">apache-airflow-backport-providers</span></code> packages that could be installed
-with those versions Those were the same providers as for 2.0 but automatically back-ported to work for
-Airflow 1.10. The last release of backport providers was done on March 17, 2021 and the backport
-providers will no longer be released, since Airflow 1.10 has reached End-Of-Life as of June 17, 2021.</p>
-</div>
-<p>If you want to contribute to <code class="docutils literal notranslate"><span class="pre">Apache</span> <span class="pre">Airflow</span></code>, you can see how to build and extend community
-managed providers in <a class="reference internal" href="howto/create-update-providers.html"><span class="doc">Community Providers</span></a>.</p>
-</div>
-<div class="section" id="custom-provider-packages">
-<span id="providers-community-maintained-providers"></span><h2>Custom provider packages<a class="headerlink" href="#custom-provider-packages" title="Permalink to this heading">¶</a></h2>
-<p>You can develop and release your own providers. Your custom operators, hooks, sensors, transfer operators
-can be packaged together in a standard airflow package and installed using the same mechanisms.
-Moreover they can also use the same mechanisms to extend the Airflow Core with auth backends,
-custom connections, logging, secret backends and extra operator links as described in the previous chapter.</p>
-</div>
-</div>
-<div class="section" id="how-to-create-your-own-provider">
-<h1>How to create your own provider<a class="headerlink" href="#how-to-create-your-own-provider" title="Permalink to this heading">¶</a></h1>
-<p>As mentioned in the <a class="reference external" href="http://airflow.apache.org/docs/apache-airflow-providers/index.html">Providers</a>
-documentation, custom providers can extend Airflow core - they can add extra links to operators as well
-as custom connections. You can use build your own providers and install them as packages if you would like
-to use the mechanism for your own, custom providers.</p>
-<p>Adding a provider to Airflow is just a matter of building a Python package and adding the right meta-data to
-the package. We are using standard mechanism of python to define
-<a class="reference external" href="https://docs.python.org/3/library/importlib.metadata.html#entry-points">entry points</a> . Your package
-needs to define appropriate entry-point <code class="docutils literal notranslate"><span class="pre">apache_airflow_provider</span></code> which has to point to a callable
-implemented by your package and return a dictionary containing the list of discoverable capabilities
-of your package. The dictionary has to follow the
-<a class="reference external" href="https://github.com/apache/airflow/blob/main/airflow/provider_info.schema.json">json-schema specification</a>.</p>
-<p>Most of the schema provides extension point for the documentation (which you might want to also use for
-your own purpose) but the important fields from the extensibility point of view are those:</p>
-<p>Displaying package information in CLI/API:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">package-name</span></code> - Name of the package for the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code> - Human-friendly name of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">description</span></code> - Additional description of the provider.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">version</span></code> - List of versions of the package (in reverse-chronological order). The first version in the
-list is the current package version. It is taken from the version of package installed, not from the
-provider_info information.</p></li>
-</ul>
-<p>Exposing customized functionality to the Airflow’s core:</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">extra-links</span></code> - this field should contain the list of all operator class names that are adding extra links
-capability. See <a class="reference external" href="/docs/apache-airflow/stable/howto/define-extra-link.html" title="(in apache-airflow v2.6.0.dev0)"><span>Define an operator extra link</span></a> for description of how to add extra link
-capability to the operators of yours.</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">connection-types</span></code> - this field should contain the list of all connection types together with hook
-class names implementing those custom connection types (providing custom extra fields and
-custom field behaviour). This field is available as of Airflow 2.2.0 and it replaces deprecated
-<code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code>. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details</p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> (deprecated) - this field should contain the list of all hook class names that provide
-custom connection types with custom extra fields and field behaviour. The <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array
-is deprecated as of Airflow 2.2.0 (for optimization reasons) and will be removed in Airflow 3. If your
-providers are targeting Airflow 2.2.0+ you do not have to include the <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> array, if
-you want to also target earlier versions of Airflow 2, you should include both <code class="docutils literal notranslate"><span class="pre">hook-class-names</span></code> and
-<code class="docutils literal notranslate"><span class="pre">connection-types</span></code> arrays. See <a class="reference external" href="/docs/apache-airflow/stable/howto/connection.html" title="(in apache-airflow v2.6.0.dev0)"><span>Managing Connections</span></a> for more details.</p></li>
+ <div class="section" id="module-airflow.providers.amazon.aws.hooks.dynamodb">
+<span id="airflow-providers-amazon-aws-hooks-dynamodb"></span><h1><a class="reference internal" href="#module-airflow.providers.amazon.aws.hooks.dynamodb" title="airflow.providers.amazon.aws.hooks.dynamodb"><code class="xref py py-mod docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.dynamodb</span></code></a><a class="headerlink" href="#module-airflow.providers.amazon.aws.hooks.dynamodb" title="Permalink to this heading">¶</a></h1>
+<p>This module contains the Amazon DynamoDB Hook</p>
+<div class="section" id="module-contents">
+<h2>Module Contents<a class="headerlink" href="#module-contents" title="Permalink to this heading">¶</a></h2>
+<div class="section" id="classes">
+<h3>Classes<a class="headerlink" href="#classes" title="Permalink to this heading">¶</a></h3>
+<table class="autosummary longtable docutils align-default">
+<colgroup>
+<col style="width: 10%" />
+<col style="width: 90%" />
+</colgroup>
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#airflow.providers.amazon.aws.hooks.dynamodb.DynamoDBHook" title="airflow.providers.amazon.aws.hooks.dynamodb.DynamoDBHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">DynamoDBHook</span></code></a></p></td>
+<td><p>Interact with Amazon DynamoDB.</p></td>
+</tr>
+</tbody>
+</table>
+<dl class="py class">
+<dt class="sig sig-object py" id="airflow.providers.amazon.aws.hooks.dynamodb.DynamoDBHook">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">airflow.providers.amazon.aws.hooks.dynamodb.</span></span><span class="sig-name descname"><span class="pre">DynamoDBHook</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ta [...]
+<dd><p>Bases: <a class="reference internal" href="../base_aws/index.html#airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook" title="airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook"><code class="xref py py-obj docutils literal notranslate"><span class="pre">airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook</span></code></a></p>
+<p>Interact with Amazon DynamoDB.
+Provide thick wrapper around
+<a class="reference external" href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/service-resource/index.html#DynamoDB.ServiceResource" title="(in Boto3 v1.26.120)"><code class="xref py py-class docutils literal notranslate"><span class="pre">boto3.resource("dynamodb")</span></code></a>.</p>
... 425591 lines suppressed ...