You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/09 17:44:59 UTC
[GitHub] [airflow] afonit opened a new issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
afonit opened a new issue #11309:
URL: https://github.com/apache/airflow/issues/11309
airflow 1.10.12
ubuntu
I am puzzled by this error and am trying to determine what file/directory of 'airflow' is not being seen. I initially started off trying to build out papermill dag's when I noticed I was getting this issue, so I have reduced it down to use all the basics found in the documentation to try and find what is wrong.
I have used most of the defaults.
* home directory has a folder of `airflow`
* when in there I performed:
```bash
airflow initdb
```
Within the airflow folder I have placed a file called `airflow.env` the contents of which are:
```
AIRFLOW_HOME=/home/ubuntu/airflow/
```
I have created the systemd file in /etc/systemd/system/airflow-scheduler.service, the contents of which are:
```
[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service
[Service]
EnvironmentFile=/home/ubuntu/airflow/airflow.env
User=ubuntu
Group=ubuntu
Type=simple
ExecStart=/home/ubuntu/anaconda3/envs/data_analysis_lab/bin/airflow scheduler
Restart=always
RestartSec=5s
[Install]
WantedBy=multi-user.target
```
Manually triggering the buildin example of: `example_python_operator` fails:
journal -u airflow-scheduler.service shows:
```
Oct 09 17:32:38 aa airflow[27741]: [2020-10-09 17:32:38,759] {scheduler_job.py:1195} INFO - Sending ('example_python_operator', 'print_the_context', datetime.datetime(2020, 10, 9, 17, 26, 57, 296322, tzinfo=<Timezone [UTC]>), 1) to execu
Oct 09 17:32:38 aa airflow[27741]: [2020-10-09 17:32:38,760] {base_executor.py:58} INFO - Adding to queue: ['airflow', 'run', 'example_python_operator', 'print_the_context', '2020-10-09T17:26:57.296322+00:00', '--local', '--pool', 'defau
Oct 09 17:32:38 aa airflow[27741]: [2020-10-09 17:32:38,762] {sequential_executor.py:54} INFO - Executing command: ['airflow', 'run', 'example_python_operator', 'print_the_context', '2020-10-09T17:26:57.296322+00:00', '--local', '--pool'
Oct 09 17:32:38 aa airflow[27741]: [2020-10-09 17:32:38,787] {scheduler_job.py:1401} ERROR - Exception when executing execute_helper
Oct 09 17:32:38 aa airflow[27741]: Traceback (most recent call last):
Oct 09 17:32:38 aa airflow[27741]: File "/home/ubuntu/anaconda3/envs/data_analysis_lab/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 1399, in _execute
Oct 09 17:32:38 aa airflow[27741]: self._execute_helper()
Oct 09 17:32:38 aa airflow[27741]: File "/home/ubuntu/anaconda3/envs/data_analysis_lab/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 1470, in _execute_helper
Oct 09 17:32:38 aa airflow[27741]: if not self._validate_and_run_task_instances(simple_dag_bag=simple_dag_bag):
Oct 09 17:32:38 aa airflow[27741]: File "/home/ubuntu/anaconda3/envs/data_analysis_lab/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line 1532, in _validate_and_run_task_instances
Oct 09 17:32:38 aa airflow[27741]: self.executor.heartbeat()
Oct 09 17:32:38 aa airflow[27741]: File "/home/ubuntu/anaconda3/envs/data_analysis_lab/lib/python3.7/site-packages/airflow/executors/base_executor.py", line 134, in heartbeat
Oct 09 17:32:38 aa airflow[27741]: self.sync()
Oct 09 17:32:38 aa airflow[27741]: File "/home/ubuntu/anaconda3/envs/data_analysis_lab/lib/python3.7/site-packages/airflow/executors/sequential_executor.py", line 57, in sync
Oct 09 17:32:38 aa airflow[27741]: subprocess.check_call(command, close_fds=True)
Oct 09 17:32:38 aa airflow[27741]: File "/home/ubuntu/anaconda3/envs/data_analysis_lab/lib/python3.7/subprocess.py", line 358, in check_call
Oct 09 17:32:38 aa airflow[27741]: retcode = call(*popenargs, **kwargs)
Oct 09 17:32:38 aa airflow[27741]: File "/home/ubuntu/anaconda3/envs/data_analysis_lab/lib/python3.7/subprocess.py", line 339, in call
Oct 09 17:32:38 aa airflow[27741]: with Popen(*popenargs, **kwargs) as p:
Oct 09 17:32:38 aa airflow[27741]: File "/home/ubuntu/anaconda3/envs/data_analysis_lab/lib/python3.7/subprocess.py", line 800, in __init__
Oct 09 17:32:38 aa airflow[27741]: restore_signals, start_new_session)
Oct 09 17:32:38 aa airflow[27741]: File "/home/ubuntu/anaconda3/envs/data_analysis_lab/lib/python3.7/subprocess.py", line 1551, in _execute_child
Oct 09 17:32:38 aa airflow[27741]: raise child_exception_type(errno_num, err_msg, err_filename)
Oct 09 17:32:38 aa airflow[27741]: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
```
Interestingly if I don't use systemd, and just launch the `airflow scheduler` from the terminal in my home directory, I don't get the errors - so I am assuming it has something to do with the systemd configuration?
I have, as of yet, been able to get more detail on: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow`
Any ideas for additional troubleshooting?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] john-james-sf commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
john-james-sf commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-895593063
I'll join the bandwagon. I edited the bash file setting the location of the home directory; however, when I run airflow info, I get filenotfound for the default location, not the location set in the script. Please advise/educate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] afonit commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
afonit commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-891191282
@zacharya19 ,
I created mine here:
```
/etc/systemd/system/airflow-scheduler.service
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] afonit closed issue #11309: WorkingDirectory= in systemd
Posted by GitBox <gi...@apache.org>.
afonit closed issue #11309:
URL: https://github.com/apache/airflow/issues/11309
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] afonit commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
afonit commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-707112755
For others who come across this.
The error was due to, on Ubuntu, The '[Service]` section needs to be `Environment="PATH=/...."` not `EnvironmentFile=...`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ajays1991 commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
ajays1991 commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-888465909
i am also facing the similar issue. What exactly did you put in `Environment="PATH=/...."` @afonit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] afonit commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
afonit commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-888473822
@ajays1991 here is the full systemd file contents:
```
[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service
[Service]
Environment="PATH=/home/ubuntu/miniconda3/envs/data_analysis_lab/bin:$PATH:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
EnvironmentFile=/home/ubuntu/notebooks/airflow/airflow
WorkingDirectory=/home/ubuntu/notebooks
User=ubuntu
Group=ubuntu
Type=simple
ExecStart=/home/ubuntu/miniconda3/envs/data_analysis_lab/bin/airflow scheduler
Restart=always
RestartSec=5s
[Install]
WantedBy=multi-user.target
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] afonit closed issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
afonit closed issue #11309:
URL: https://github.com/apache/airflow/issues/11309
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] idnoclip commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
idnoclip commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-1080474228
I had similar issue on Raspberry Pi - it seemed that somehow service used different PATH variable than the user running it. I managed to work it around by modyfing /etc/systemd/system/airflow-scheduler.service file, replacing the line:
`ExecStart=/bin/bash -c '/path/to/airflow scheduler'`
with
`ExecStart=/bin/bash -c 'PATH=$PATH:/path/to/airflow; /path/to/airflow scheduler'`
This way i was sure that the PATH variable contains path to the airflow executable.
Then I had to stop services, reload service daemon and start them again:
`sudo service airflow-webserver stop`
`sudo service airflow-scheduler stop`
`sudo systemctl daemon-reload`
`sudo service airflow-webserver start`
`sudo service airflow-scheduler start`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] tanujdhiman commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
tanujdhiman commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-892454313
Hello everybody !!
I'm also facing an error of `FileNotFound Error` when using Airflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] tanujdhiman edited a comment on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
tanujdhiman edited a comment on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-892454313
Hello everybody !!
I'm also facing an error of `FileNotFound Error` when using Airflow.
Can you please tell me how I set my path for a saving a particular file at a specific location??
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Zacharyr41 commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
Zacharyr41 commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-891136281
> @ajays1991 here is the full systemd file contents:
>
> ```
> [Unit]
> Description=Airflow scheduler daemon
> After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
> Wants=postgresql.service mysql.service redis.service rabbitmq-server.service
>
> [Service]
> Environment="PATH=/home/ubuntu/miniconda3/envs/data_analysis_lab/bin:$PATH:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
> EnvironmentFile=/home/ubuntu/notebooks/airflow/airflow
> WorkingDirectory=/home/ubuntu/notebooks
> User=ubuntu
> Group=ubuntu
> Type=simple
> ExecStart=/home/ubuntu/miniconda3/envs/data_analysis_lab/bin/airflow scheduler
> Restart=always
> RestartSec=5s
>
> [Install]
> WantedBy=multi-user.target
> ```
This may be a silly question, but where can I edit this systemd file contents?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] afonit edited a comment on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
afonit edited a comment on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-707112755
For others who come across this.
The error was due to, on Ubuntu, The `[Service]` section needs to be `Environment="PATH=/...."` not `EnvironmentFile=...`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #11309: WorkingDirectory= in systemd
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-704386363
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11309:
URL: https://github.com/apache/airflow/issues/11309#issuecomment-895850195
@john-james-sf - this issue is closed and I think your have a different environment/error. Please open a new discussion (best) rather than issue and describe your circumstances, what you've done, how you installed airflow, which version and all the details you can get.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org