[jira] [Created] (AIRFLOW-3750) SSHOperator Not Ending Process On Remote Machine

["pranav agrawal (JIRA)" <ji...@apache.org>]

pranav agrawal created AIRFLOW-3750:
---------------------------------------

             Summary: SSHOperator Not Ending Process On Remote Machine
                 Key: AIRFLOW-3750
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3750
             Project: Apache Airflow
          Issue Type: Bug
          Components: operators
    Affects Versions: 1.10.1
            Reporter: pranav agrawal
         Attachments: Screen Shot 2019-01-22 at 5.17.27 PM.png

*Airflow Version: 1.10.1*

I have run a DAG containing SSHOperator,


{code:java}
from airflow.contrib.operators.ssh_operator import SSHOperator
SSHOperator(
task_id='mongo_transformation',
ssh_conn_id='datapl_utils',
command='java -cp /home/airflowclient/application-0.0.1-SNAPSHOT/application-0.0.1-SNAPSHOT-jar-with-dependencies.jar com.oyo.spark.application.MongoIngestionNew {}'.format(Variable.get('mongo_new_transformation')),
retries=99999,
dag=dag)
{code}

(ATTACHED ssh_conn_id details as image)



It starts to run as expected on remote machine,

 
{code:java}
root 7575 2638 0 10:55 ? 00:00:00 sshd: airflowclient [priv]
airflow+ 7577 7575 0 10:55 ? 00:00:00 sshd: airflowclient@notty
airflow+ 7578 7577 23 10:55 ? 00:00:12 java -cp /home/airflowclient/application-0.0.1-SNAPSHOT/application-0.0.1-SNAPSHOT-jar-with-dependencies.jar c
{code}
 
After a while, I marked the DAG as Success via UI, I get following log in airflow logs,


{code:java}
[2019-01-22 11:14:20,866] {logging_mixin.py:95} INFO - [2019-01-22 11:14:20,866] {jobs.py:2695} WARNING - State of this instance has been externally set to success. Taking the poison pill.
[2019-01-22 11:14:20,911] {helpers.py:240} INFO - Sending 15 to GPID 8260
[2019-01-22 11:14:20,911] {models.py:1636} ERROR - Received SIGTERM. Terminating subprocesses.
[2019-01-22 11:14:22,341] {helpers.py:230} INFO - Process psutil.Process(pid=8260 (terminated)) (8260) terminated with exit code 0
[2019-01-22 11:14:22,342] {logging_mixin.py:95} INFO - [2019-01-22 11:14:22,342] {jobs.py:2627} INFO - Task exited with return code 0{code}


But on the remote machine, the java process is still running even after several minutes,


{code:java}
airflow+ 7578 1 99 10:55 ? 00:58:02 java -cp /home/airflowclient/application-0.0.1-SNAPSHOT/application-0.0.1-SNAPSHOT-jar-with-dependencies.jar com.oyo.spark.application.MongoIngestionNew prod 10.20.3.33 owner_portal app-jan21-test-1{code}


server details,
{code:java}
[airflowclient@prod-datapl-common-utils ~]$ cat /etc/system-release
Amazon Linux release 2 (Karoo){code}

Please help me with fixing this at the earliest.

Thanks!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)