You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/14 16:34:09 UTC
[GitHub] [airflow] potiuk opened a new issue #17617: Switch Alibaba tests to use Mocks
potiuk opened a new issue #17617:
URL: https://github.com/apache/airflow/issues/17617
We should also take a look if there is a way to mock the calls rather than having to communicate with the reaal "Alibaba Cloud". This way our unit tests would be even more stable. For AWS we are using `moto` library. Seems that Alibaba has built-in way to use Mock server as backend (?) https://www.alibabacloud.com/help/doc-detail/48978.htm
CC: @Gabriel39 -> would you like to pick on that task? We had a number of transient failures of the unit tests over the last week - cause the unit tests were actually reaching out to cn-hengzou Alibaba region's servers, which has a lot of stability issues. I am switching to us-east-1 in https://github.com/apache/airflow/pull/17616 but ideally we should not reach out to "real" services in unit tests (we are working on standardizing system tests that will do that and reach-out to real services but it should not be happening for unit tests).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17617:
URL: https://github.com/apache/airflow/issues/17617#issuecomment-1060093167
Merged it . This looks cool. Thanks! Please proceed with other tests :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] EricGao888 commented on issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on issue #17617:
URL: https://github.com/apache/airflow/issues/17617#issuecomment-1065062989
> Merged it . This looks cool. Thanks! Please proceed with other tests :)
All unit tests in alibaba provider have been fixed now : ) @potiuk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #17617:
URL: https://github.com/apache/airflow/issues/17617#issuecomment-898915853
Can I assign you that one @Gabriel39 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #17617:
URL: https://github.com/apache/airflow/issues/17617#issuecomment-982788006
@Gabriel39 are you still working on it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17617:
URL: https://github.com/apache/airflow/issues/17617#issuecomment-1053650510
Feel Free!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #17617:
URL: https://github.com/apache/airflow/issues/17617
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] EricGao888 commented on issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on issue #17617:
URL: https://github.com/apache/airflow/issues/17617#issuecomment-1059128105
> Feel Free!
@potiuk Hello, I've switched test cases in oss hook to use Mocks in [this PR](https://github.com/apache/airflow/pull/21992). If this works, I will fix unit tests in [oss_task_handler](https://github.com/apache/airflow/pull/21785), oss operator and oss sensor in future PRs and commits. THX : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17617:
URL: https://github.com/apache/airflow/issues/17617#issuecomment-898915853
Can I assign you to that one @Gabriel39 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Gabriel39 commented on issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
Gabriel39 commented on issue #17617:
URL: https://github.com/apache/airflow/issues/17617#issuecomment-898996858
OK, I'll take this task and will submit next PR in next few days. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] EricGao888 commented on issue #17617: Switch Alibaba tests to use Mocks
Posted by GitBox <gi...@apache.org>.
EricGao888 commented on issue #17617:
URL: https://github.com/apache/airflow/issues/17617#issuecomment-1050672562
> We should also take a look if there is a way to mock the calls rather than having to communicate with the reaal "Alibaba Cloud". This way our unit tests would be even more stable. For AWS we are using `moto` library. Seems that Alibaba has built-in way to use Mock server as backend (?) https://www.alibabacloud.com/help/doc-detail/48978.htm
>
> CC: @Gabriel39 -> would you like to pick on that task? We had a number of transient failures of the unit tests over the last week - cause the unit tests were actually reaching out to cn-hengzou Alibaba region's servers, which has a lot of stability issues. I am switching to us-east-1 in #17616 but ideally we should not reach out to "real" services in unit tests (we are working on standardizing system tests that will do that and reach-out to real services but it should not be happening for unit tests).
Maybe we could just simply @mock.patch this get_connection() function in unit tests and there will be no need to connect to a real oss. I'd like to work on this issue if currently there's no one working on this. Please assign it to me, thx : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org