You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/23 13:59:02 UTC
[GitHub] [airflow] vincbeck opened a new pull request #22489: SalesforceToS3Operator: update sample dag and doc
vincbeck opened a new pull request #22489:
URL: https://github.com/apache/airflow/pull/22489
Update sample dag and documentation on `SalesforceToS3Operator`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on a change in pull request #22489: SalesforceToS3Operator: update sample dag and doc
Posted by GitBox <gi...@apache.org>.
vincbeck commented on a change in pull request #22489:
URL: https://github.com/apache/airflow/pull/22489#discussion_r833676299
##########
File path: airflow/providers/amazon/aws/example_dags/example_salesforce_to_s3.py
##########
@@ -15,60 +15,35 @@
# specific language governing permissions and limitations
# under the License.
-# Ignore missing args provided by default_args
-# type: ignore[call-arg]
-
"""
This is a basic example DAG for using `SalesforceToS3Operator` to retrieve Salesforce customer
Review comment:
Nice one. Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #22489: SalesforceToS3Operator: update sample dag and doc
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #22489:
URL: https://github.com/apache/airflow/pull/22489#discussion_r833380504
##########
File path: docs/apache-airflow-providers-amazon/operators/transfer/salesforce_to_s3.rst
##########
@@ -15,35 +15,33 @@
specific language governing permissions and limitations
under the License.
-Salesforce To S3 Operator
-==============================
-
-.. _howto/operator:SalesforceToS3Operator:
-
-Overview
---------
+Salesforce To Amazon S3 Transfer Operator
+=========================================
Use the
:class:`~airflow.providers.amazon.aws.transfers.salesforce_to_s3.SalesforceToS3Operator`
-to execute a Salesforce query to fetch data and upload to S3. The results of the query
-are initially written to a local, temporary directory and then uploaded to an S3 bucket.
+to execute a Salesforce query to fetch data and upload to an Amazon S3 bucket.
+
+Prerequisite Tasks
+^^^^^^^^^^^^^^^^^^
-Extract Customer Data from Salesforce
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. include:: ../_partials/prerequisite_tasks.rst
-The following example demonstrates a use case of extracting customer data from a Salesforce
-instance and upload to a "landing" bucket in S3.
+.. _howto/operator:SalesforceToS3Operator:
+
+Extract Account data from Salesforce
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following example demonstrates a use case of extracting account data from a Salesforce
+instance and upload to an Amazon S3 bucket.
.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_salesforce_to_s3.py
:language: python
- :start-after: [START howto_operator_salesforce_to_s3_transfer]
- :end-before: [END howto_operator_salesforce_to_s3_transfer]
+ :dedent: 4
+ :start-after: [START howto_transfer_salesforce_to_s3]
+ :end-before: [END howto_transfer_salesforce_to_s3]
Reference
---------
-This operator uses the :class:`~airflow.providers.salesforce.hooks.salesforce.SalesforceHook`
-to interact with Salesforce. This hook is built with functionality from the Simple Salesforce
-package.
-
-For further information, review the `Simple Salesforce Documentation <https://simple-salesforce.readthedocs.io/en/latest/>`__.
Review comment:
Just curious, why remove this context completely? Seems a little empty just having a link to the simple-salesforce lib.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on a change in pull request #22489: SalesforceToS3Operator: update sample dag and doc
Posted by GitBox <gi...@apache.org>.
vincbeck commented on a change in pull request #22489:
URL: https://github.com/apache/airflow/pull/22489#discussion_r833432250
##########
File path: docs/apache-airflow-providers-amazon/operators/transfer/salesforce_to_s3.rst
##########
@@ -15,35 +15,33 @@
specific language governing permissions and limitations
under the License.
-Salesforce To S3 Operator
-==============================
-
-.. _howto/operator:SalesforceToS3Operator:
-
-Overview
---------
+Salesforce To Amazon S3 Transfer Operator
+=========================================
Use the
:class:`~airflow.providers.amazon.aws.transfers.salesforce_to_s3.SalesforceToS3Operator`
-to execute a Salesforce query to fetch data and upload to S3. The results of the query
-are initially written to a local, temporary directory and then uploaded to an S3 bucket.
+to execute a Salesforce query to fetch data and upload to an Amazon S3 bucket.
+
+Prerequisite Tasks
+^^^^^^^^^^^^^^^^^^
-Extract Customer Data from Salesforce
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. include:: ../_partials/prerequisite_tasks.rst
-The following example demonstrates a use case of extracting customer data from a Salesforce
-instance and upload to a "landing" bucket in S3.
+.. _howto/operator:SalesforceToS3Operator:
+
+Extract Account data from Salesforce
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The following example demonstrates a use case of extracting account data from a Salesforce
+instance and upload to an Amazon S3 bucket.
.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_salesforce_to_s3.py
:language: python
- :start-after: [START howto_operator_salesforce_to_s3_transfer]
- :end-before: [END howto_operator_salesforce_to_s3_transfer]
+ :dedent: 4
+ :start-after: [START howto_transfer_salesforce_to_s3]
+ :end-before: [END howto_transfer_salesforce_to_s3]
Reference
---------
-This operator uses the :class:`~airflow.providers.salesforce.hooks.salesforce.SalesforceHook`
-to interact with Salesforce. This hook is built with functionality from the Simple Salesforce
-package.
-
-For further information, review the `Simple Salesforce Documentation <https://simple-salesforce.readthedocs.io/en/latest/>`__.
Review comment:
It just, to me, kind of obvious information. Most of operator use hooks under the hood to communicate with the underlying service. I just thought it was useless but I am happy to put it back if you think otherwise
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #22489: SalesforceToS3Operator: update sample dag and doc
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #22489:
URL: https://github.com/apache/airflow/pull/22489
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on a change in pull request #22489: SalesforceToS3Operator: update sample dag and doc
Posted by GitBox <gi...@apache.org>.
josh-fell commented on a change in pull request #22489:
URL: https://github.com/apache/airflow/pull/22489#discussion_r833372992
##########
File path: airflow/providers/amazon/aws/example_dags/example_salesforce_to_s3.py
##########
@@ -15,60 +15,35 @@
# specific language governing permissions and limitations
# under the License.
-# Ignore missing args provided by default_args
-# type: ignore[call-arg]
-
"""
This is a basic example DAG for using `SalesforceToS3Operator` to retrieve Salesforce customer
Review comment:
```suggestion
This is a basic example DAG for using `SalesforceToS3Operator` to retrieve Salesforce account
```
Since the query is changing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org