You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/20 10:26:52 UTC
[GitHub] [airflow] IvanUkhov opened a new issue #13192: Generalize MLEngine(Training|StartTrainingJob)Operator to custom images
IvanUkhov opened a new issue #13192:
URL: https://github.com/apache/airflow/issues/13192
The operator is arguably unnecessarily limited to AI Platform’s standard images. The only change that is required to lift this constraint is making `package_uris` and `training_python_module` optional with default values `[]` and `None`, respectively. Then, using `master_config`, one can supply `imageUri` and run any image of choice.
If the above sounds reasonable, I can open pull requests to both version 1 and version 2.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek removed a comment on issue #13192: Generalize MLEngineStartTrainingJobOperator to custom images
Posted by GitBox <gi...@apache.org>.
turbaszek removed a comment on issue #13192:
URL: https://github.com/apache/airflow/issues/13192#issuecomment-748602410
Please check the discussion:
https://github.com/apache/airflow/discussions/12610
I don't think the operator is limited as you can do:
```python
master_config={"imageUri" : "gcr.io/my-pytorch-image-uri"}
```
Please, reopen if I'm wrong 👌
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] IvanUkhov edited a comment on issue #13192: Generalize MLEngineStartTrainingJobOperator to custom images
Posted by GitBox <gi...@apache.org>.
IvanUkhov edited a comment on issue #13192:
URL: https://github.com/apache/airflow/issues/13192#issuecomment-748603401
Those two are mutually exclusive ways of working with AI Platform. In other words, it does not make sense to specify both `training_python_module` and `imageUri`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] IvanUkhov edited a comment on issue #13192: Generalize MLEngineStartTrainingJobOperator to custom images
Posted by GitBox <gi...@apache.org>.
IvanUkhov edited a comment on issue #13192:
URL: https://github.com/apache/airflow/issues/13192#issuecomment-748603401
Those two are mutually exclusive ways of working with AI Platform. In other words, it does not make sense to specify both `training_python_module` together with `imageUri`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #13192: Generalize MLEngine(Training|StartTrainingJob)Operator to custom images
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13192:
URL: https://github.com/apache/airflow/issues/13192#issuecomment-748589165
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #13192: Generalize MLEngineStartTrainingJobOperator to custom images
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #13192:
URL: https://github.com/apache/airflow/issues/13192
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek closed issue #13192: Generalize MLEngineStartTrainingJobOperator to custom images
Posted by GitBox <gi...@apache.org>.
turbaszek closed issue #13192:
URL: https://github.com/apache/airflow/issues/13192
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] IvanUkhov commented on issue #13192: Generalize MLEngineStartTrainingJobOperator to custom images
Posted by GitBox <gi...@apache.org>.
IvanUkhov commented on issue #13192:
URL: https://github.com/apache/airflow/issues/13192#issuecomment-748603401
Those two are mutually exclusive ways of working with AI Platform. In other words, it does not make sense to specify `training_python_module` together with `imageUri`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #13192: Generalize MLEngineStartTrainingJobOperator to custom images
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #13192:
URL: https://github.com/apache/airflow/issues/13192#issuecomment-748602410
Please check the discussion:
https://github.com/apache/airflow/discussions/12610
I don't think the operator is limited as you can do:
```python
master_config={"imageUri" : "gcr.io/my-pytorch-image-uri"}
```
Please, reopen if I'm wrong 👌
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #13192: Generalize MLEngineStartTrainingJobOperator to custom images
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #13192:
URL: https://github.com/apache/airflow/issues/13192#issuecomment-749163358
@IvanUkhov would you mind raising a PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #13192: Generalize MLEngineStartTrainingJobOperator to custom images
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #13192:
URL: https://github.com/apache/airflow/issues/13192#issuecomment-748602528
Do I correctly understand that this:
```python
master_config={"imageUri" : "gcr.io/my-pytorch-image-uri"}
```
Won't work currently?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org