You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "ishiis (via GitHub)" <gi...@apache.org> on 2023/10/17 10:45:21 UTC

[I] SFTPToGCSOperator doesn't fail like `SFTPOperator` if the source file doesn't exist [airflow]

ishiis opened a new issue, #34989:
URL: https://github.com/apache/airflow/issues/34989

   ### Apache Airflow version
   
   2.7.2
   
   ### What happened
   
   When source_path contains a wildcard, SFTPToGCSOperator succeeds if the file does not exist in the specified path.
   
   ### What you think should happen instead
   
   If the file does not exist, the operator should fail.
   
   ### How to reproduce
   
   1. Include wildcards in source_path.
   2. If the file does not exist in source_path, it will be reproduced.
   
   ```python
   SFTPToGCSOperator(
       task_id=f"sftp_to_gcs_task",
       gcp_conn_id="gcp_conn_id",
       sftp_conn_id="sftp_conn_id",
       source_path="/var/path/to/*",    # wildcard within source_path
       destination_bucket="my_bucket",
       destination_path="path/to"
   )
   ```
   
   ### Operating System
   
   GCP Container
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-google==10.5.0
   
   ### Deployment
   
   Google Cloud Composer
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] SFTPToGCSOperator doesn't fail like `SFTPOperator` if the source file doesn't exist [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] closed issue #34989: SFTPToGCSOperator doesn't fail like `SFTPOperator` if the source file doesn't exist
URL: https://github.com/apache/airflow/issues/34989


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] SFTPToGCSOperator doesn't fail like `SFTPOperator` if the source file doesn't exist [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on issue #34989:
URL: https://github.com/apache/airflow/issues/34989#issuecomment-1813498430

   This issue has been closed because it has not received response from the issue author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] SFTPToGCSOperator doesn't fail like `SFTPOperator` if the source file doesn't exist [airflow]

Posted by "Taragolis (via GitHub)" <gi...@apache.org>.

Taragolis commented on issue #34989:
URL: https://github.com/apache/airflow/issues/34989#issuecomment-1766715414

   What behaviour then expected if required to move object (`move=True`)?
   If task completed successfully and after that it restarted then with your suggestion the task failed, which might breaks someone pipeline.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] SFTPToGCSOperator doesn't fail like `SFTPOperator` if the source file doesn't exist [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on issue #34989:
URL: https://github.com/apache/airflow/issues/34989#issuecomment-1802944152

   This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org