You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/06/25 20:48:22 UTC
[GitHub] [airflow] gardner-dev opened a new issue #9518: Docker operator command
gardner-dev opened a new issue #9518:
URL: https://github.com/apache/airflow/issues/9518
Hi, when I am using command as a one element list it generated unhandled, strange error. Docs don't say, that list can not consist only one element. I've strugled two days until I resolve it. Can it be fixed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] gardner-dev commented on issue #9518: Docker operator command
Posted by GitBox <gi...@apache.org>.
gardner-dev commented on issue #9518:
URL: https://github.com/apache/airflow/issues/9518#issuecomment-651801755
Thank you @mik-laj , issue could be closed then
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj edited a comment on issue #9518: Docker operator command
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #9518:
URL: https://github.com/apache/airflow/issues/9518#issuecomment-650747654
> Docs don't say, that list can not consist only one element
It can be only one element, but it should be an executable file.
```
command=["./my-awesome-script.sh"]
```
You can also run a script called `my awesome script.py` (name contains spaces)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] gardner-dev closed issue #9518: Docker operator command
Posted by GitBox <gi...@apache.org>.
gardner-dev closed issue #9518:
URL: https://github.com/apache/airflow/issues/9518
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #9518: Docker operator command
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #9518:
URL: https://github.com/apache/airflow/issues/9518#issuecomment-649809657
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #9518: Docker operator command
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #9518:
URL: https://github.com/apache/airflow/issues/9518#issuecomment-650747401
You should use
```
command="python owid_API.py -date {{ ds }}"
```
or
```
command=["python", "owid_API.py", "-date", "{{ ds }}"]
```
I don't know what we could do to prevent a similar problem in the future.
A similar problem will occur in CLI if you provide an invalid value.
```
docker run -ti --rm openjdk:8-jre-slim 'bash -c "echo 1"'
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"bash -c \\\"echo 1\\\"\": executable file not found in $PATH": unknown.
```
We may require this argument to be a list, but it is a breaking change. And this breaks one special feature of this operator.
You can put an array into XCOM and as an argument.
For xcom = `["bash", "-c", "echo 123"]`
and
```
command="{{ task_instance.xcom_pull("create_dataset_task", key="dataset_id") }}"
```
it will run `bash -c 'echo 123'`. Please note that the number of array elements can be dynamic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] gardner-dev commented on issue #9518: Docker operator command
Posted by GitBox <gi...@apache.org>.
gardner-dev commented on issue #9518:
URL: https://github.com/apache/airflow/issues/9518#issuecomment-650744447
Sure @nullhack! In the documentation regarding docker operaort we can read
```
command (str or list) – Command to be run in the container. (templated)
```
When I run docker operator with command such as:
`command=["python owid_API.py -date {{ ds }}"]`
it generates error
> docker.errors.APIError: 500 Server Error: Internal Server Error ("OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"python owid_API.py -date 2020-03-14\": executable file not found in $PATH": unknown")
The clear problem is that list is one-element list. But it should be specified in docs that list doesn't work for one command
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nullhack commented on issue #9518: Docker operator command
Posted by GitBox <gi...@apache.org>.
nullhack commented on issue #9518:
URL: https://github.com/apache/airflow/issues/9518#issuecomment-650137848
Hi @gardner-dev Thank you for opening this report. Can you give some details?
Like, What is the strange error you mentioned? How have you solved It? Can you provide a simple example of DAG which can reproduce the issue? Which version of airflow are you using?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #9518: Docker operator command
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #9518:
URL: https://github.com/apache/airflow/issues/9518#issuecomment-650747654
> Docs don't say, that list can not consist only one element
It can be only one element, but it should be an executable file.
```
command=["./my-awesome-script.sh"]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org