You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "W Hasnaoui (JIRA)" <ji...@apache.org> on 2019/07/11 12:37:00 UTC
[jira] [Reopened] (AIRFLOW-4584) Error when using ssh operateur to
execute a sh script from an remote server
[ https://issues.apache.org/jira/browse/AIRFLOW-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
W Hasnaoui reopened AIRFLOW-4584:
---------------------------------
Hi,
Please, i find this post AIRFLOW-4743. it's about adding variables environnement.
Can i fix myself the changes in the sshoperator.py.
Manay thanksfor your usual help
> Error when using ssh operateur to execute a sh script from an remote server
> ---------------------------------------------------------------------------
>
> Key: AIRFLOW-4584
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4584
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
> Affects Versions: 1.10.2
> Reporter: W Hasnaoui
> Assignee: Ash Berlin-Taylor
> Priority: Major
> Fix For: 1.10.4
>
>
> hello guys;
> i need yr help please, i'm new in apache airflow, and i'm trying to ssh operateur to execute a shell script from a remote server, my code looks like this:
>
> t4 = SSHOperator(
> ssh_conn_id='test_ssh',
> task_id= 'Execute_transfert',
> command="""sh 'scripts/jwi/test.sh'""",
> dag=dag )
>
> the only thing is inside my script (test.sh) i called a pentaho job (.kjb extention), the line command gives:
> LOGFILE="/xxx2/xxx3/logs/migxxx__`date "+%Y-%m-%d-%H%M"`.log"
> JOBFILE="/xxx2/xxx3/xxx4/migxxx/avxxx.kjb"
> PDI_LEVEL=Detailed
> /folder1/folder2/kitchen.sh -file:$JOBFILE -level:$PDI_LEVEL -logfile:$LOGFILE
> when running and afeter establishing connection to the remote server, the execution faild, a snapshot of the log:
> {{[2019-05-27 20:02:02,651] \{logging_mixin.py:95} INFO - [2019-05-27 20:02:02,651] \{transport.py:1746} INFO - Connected (version 2.0, client OpenSSH_4.3) }}
> {{[2019-05-27 20:02:05,877] \{logging_mixin.py:95} INFO - [2019-05-27 20:02:05,877] \{transport.py:1746} INFO - Authentication (publickey) failed. }}
> {{[2019-05-27 20:02:05,897] \{logging_mixin.py:95} INFO - [2019-05-27 20:02:05,897] \{transport.py:1746} INFO - Authentication (password) successful! }}
> {{[2019-05-27 20:02:06,640] \{ssh_operator.py:133} INFO - INFO 27-05 18:22:07,371 - Using "/tmp/vfs_cache" as temporary files store. }}
> {{[2019-05-27 20:02:06,777] \{models.py:1788} ERROR - SSH operator error: 'utf8' codec can't decode byte 0xe9 in position 63: invalid continuation byte }}
> {{Traceback (most recent call last): }}
> {{File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1657, in _run_raw_task }}
> {{result = task_copy.execute(context=context)}}
> {{ File "/usr/lib/python2.7/site-packages/airflow/contrib/operators/ssh_operator.py", line 167, in execute}}
> {{ raise AirflowException("SSH operator error: \{0}".format(str(e)))}}
> {{ AirflowException: SSH operator error: 'utf8' codec can't decode byte 0xe9 in position 63: invalid continuation byte }}
> {{[2019-05-27 20:02:06,780] \{models.py:1817} INFO - All retries failed; marking task as FAILED [2019-05-27 20:02:06,795] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert Traceback (most recent call last): }}
> {{[2019-05-27 20:02:06,796] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert File "/usr/bin/airflow", line 32, in <module> [2019-05-27 20:02:06,796] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert args.func(args) }}
> {{[2019-05-27 20:02:06,796] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/utils/cli.py", line 74, in wrapper [2019-05-27 20:02:06,797] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert return f(*args, **kwargs) }}
> {{[2019-05-27 20:02:06,797] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py", line 526, in run [2019-05-27 20:02:06,798] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert _run(args, dag, ti) }}
> {{[2019-05-27 20:02:06,798] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py", line 445, in _run [2019-05-27 20:02:06,798] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert pool=args.pool, }}
> {{[2019-05-27 20:02:06,799] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/utils/db.py", line 73, in wrapper [2019-05-27 20:02:06,799] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert return func(*args, **kwargs) }}
> {{[2019-05-27 20:02:06,799] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1657, in _run_raw_task [2019-05-27 20:02:06,799] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert result = task_copy.execute(context=context) }}
> {{[2019-05-27 20:02:06,800] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert File "/usr/lib/python2.7/site-packages/airflow/contrib/operators/ssh_operator.py", line 167, in execute [2019-05-27 20:02:06,800] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert raise AirflowException("SSH operator error: \{0}".format(str(e))) }}
> {{[2019-05-27 20:02:06,800] \{base_task_runner.py:101} INFO - Job 1146: Subtask Execute_transfert airflow.exceptions.AirflowException: SSH operator error: 'utf8' codec can't decode byte 0xe9 in position 63: invalid continuation byte}}
> please help how can resolve this issue.
>
> Many thanks
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)