You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/05/07 22:37:13 UTC
[GitHub] [airflow] ashb opened a new issue #8774: Move singularity out of main CI tests and into separate docker image/system test
ashb opened a new issue #8774:
URL: https://github.com/apache/airflow/issues/8774
A CI build failed because we couldn't download a release from https://github.com/sylabs/singularity -- which while it was a random network blib, there is no _need_ for the tests for the singularity operators to live in our CI image.
We should instead create "system tests" for singularity, and ensure that the unit tests use mocking.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #8774: Move singularity out of main CI tests and into separate docker image/system test
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #8774:
URL: https://github.com/apache/airflow/issues/8774#issuecomment-625770630
Not sure if it is still needed. Maybe @vsoch can tell more about it whether it is actually needed in the image. I think we had a least few "integration" kind of tests where we run it against a real singluarity installation?
Comment:
In this case it is more of an "integration" tests rather than "system" test (at least in the currently used "integration" test definition that we have). "System" tests (at least how we have defined now) are tests that are reaching out to something that is externally available and we do not have a locally running - via docker compose. Maybe eventually we might want to change those names somehow after we finish all of those "clean-ups" but for now, it's the classification we use :). This is another - unrelated topic on making an order also in the "unit" tests we have - some of them require backend db for example, some of them no, some of them require the whole airflow environment configured, so they are not technically "unit" tests either.
But I am sure we will get there and get the classification right and commonly agreed eventually :). It's just not the right time to do it.
I plan to take a look at this after some of the most important production image changes are handled (this is much higher priority). I created an umbrella task for it #8785 and linked it with all the sub-tasks (minicluster, Hadoop, singularity, kubectl, kind) as all of them can be moved out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb closed issue #8774: Move singularity out of main CI tests and into separate docker image
Posted by GitBox <gi...@apache.org>.
ashb closed issue #8774:
URL: https://github.com/apache/airflow/issues/8774
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #8774: Move singularity out of main CI tests and into separate docker image/system test
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #8774:
URL: https://github.com/apache/airflow/issues/8774#issuecomment-625532382
Wait, I've just looked at the singularity operator tests - it already seems to mock it, so does anyone know why we include this in our CI image? @potiuk ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org