You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/10/31 10:31:50 UTC
[GitHub] [airflow] VladaZakharova opened a new pull request, #27405: Add deferrable mode to TestMLEngineStartTrainingJobOperator
VladaZakharova opened a new pull request, #27405:
URL: https://github.com/apache/airflow/pull/27405
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1345518123
Rebased after fixing the "public" tests last week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1337432454
Restarted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova closed pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova closed pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
URL: https://github.com/apache/airflow/pull/27405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1337426847
@potiuk
Thank you for having a look and helping with this problem!
Unfortunately, fix with adding this warning to "known" category didn't help and some of the tests are still failing. Can you please restart them once again?
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358028677
@Taragolis
Thank you for having a look!
Have you faced the same problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358158556
OK. I fixed it (and I found that the test missed one assert). PR is coming.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
Taragolis commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1357990959
Interesting, the reason of failing tests could be some kind of race condition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova closed pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova closed pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
URL: https://github.com/apache/airflow/pull/27405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova closed pull request #27405: Add deferrable mode to TestMLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova closed pull request #27405: Add deferrable mode to TestMLEngineStartTrainingJobOperator
URL: https://github.com/apache/airflow/pull/27405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358056365
@XD-DENG - maybe you can also take a look to see for a possible race condition it could have caused? It seems like a side-effect - especially that when it fails it fails because ir returns several namespaces (airflow and default) so it is rather appropriate for the multi-namespace change being the culprit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova closed pull request #27405: Add deferrable mode to TestMLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova closed pull request #27405: Add deferrable mode to TestMLEngineStartTrainingJobOperator
URL: https://github.com/apache/airflow/pull/27405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358051182
> @Taragolis Thank you for having a look! Have you faced the same problem?
It started to happen recently randomly in a number of PRs and usually goes away afer re-running, typical flakey test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by "VladaZakharova (via GitHub)" <gi...@apache.org>.
VladaZakharova commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1403230279
@potiuk
Hello Team,
Is it possible to review and merge this changes? Thanks! :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1345628311
@potiuk
Thank you for rebasing :)
Still seems like some errors that are not related to my changes but with Kubernetes executor:
https://github.com/apache/airflow/actions/runs/3668592118/jobs/6201917454#step:6:7783
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
Taragolis commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358025835
Seems like the issue come from this class, which changed by this PR: https://github.com/apache/airflow/pull/28047:
https://github.com/apache/airflow/blob/672264b0af4874274bd130ba42a78e8e72f3d3ff/airflow/executors/kubernetes_executor.py#L66-L77
During the tests time to time additional namespace `default` come into this singleton.
I do not know is it issue with some tests itself or problem with some internals of k8s executor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1330339309
Tests :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1336532020
For : https://github.com/apache/airflow/actions/runs/3523584902/jobs/5908291978 - the steps to follow are described in the instructions after the error is displayed. Just follow. The problem is really about new "warning" generated and you just need to add the warning as "known" warning to the list of known warnings.
The other failures are - I think - just intermitteent errors - this seems to be a problem when you run the whole suite of tests as a non-commiter, sometimes (and we have not figured out when) the tests are failing intermittently in groups - but when we re-run them they work. those are flaky tests that we need to fix eventually.
Please rebase and fix the warning and if other tests fail next time, ping me and I will re-run them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358166398
Fix to the flaky exception here: https://github.com/apache/airflow/pull/28475
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1330351549
>
Hello @potiuk
This is not a problem of tests themselves, i suppose, but the problem of new package that i have added to providers.yaml file (gcloud-aio-auth): https://github.com/apache/airflow/actions/runs/3523584902/jobs/5908291978#step:10:1997
It seems that the version of currently used pyarrow is not compatible with version of pyarrow that is used in this new package.
So the solution here is may be to upgrade version of pyarrow in gcloud-aio-auth, but i am not sure how to do it correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1378589317
@potiuk
Hi Team :)
Can we please review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358475180
@potiuk @XD-DENG
Team, thanks a lot for so quick response and helping with this problem :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1337432018
Yeah. I will take a look at those failing "public runners" tests shortly (cc: @Taragolis - unless you want to take a look following my description).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1367870652
@potiuk
Hi! Can you please review this PR?
Many thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1358058017
Sure I will take a look @potiuk . Please ping me for reminding if you don't hear from me later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #27405:
URL: https://github.com/apache/airflow/pull/27405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova closed pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova closed pull request #27405: Add deferrable mode to MLEngineStartTrainingJobOperator
URL: https://github.com/apache/airflow/pull/27405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] VladaZakharova closed pull request #27405: Add deferrable mode to TestMLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
VladaZakharova closed pull request #27405: Add deferrable mode to TestMLEngineStartTrainingJobOperator
URL: https://github.com/apache/airflow/pull/27405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #27405: Add deferrable mode to TestMLEngineStartTrainingJobOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #27405:
URL: https://github.com/apache/airflow/pull/27405#issuecomment-1311067439
conflicts :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org