[jira] [Created] (ZEPPELIN-3542) Stop a pyspark paragraph cancels all the pyspark jobs of other users

["Jhon Cardenas (JIRA)" <ji...@apache.org>]

Jhon Cardenas created ZEPPELIN-3542:
---------------------------------------

             Summary: Stop a pyspark paragraph cancels all the pyspark jobs of other users
                 Key: ZEPPELIN-3542
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3542
             Project: Zeppelin
          Issue Type: Bug
          Components: pySpark
    Affects Versions: 0.8.0
         Environment: This happen when the spark context is shared between users (scoped).
            Reporter: Jhon Cardenas


Stop a pyspark paragraph cancels all the pyspark jobs of other users.

Cancel button in pyspark paragraph cancels Spark jobs for all users.

This happen when the spark context is shared between users (scoped).

It seems to be related with [the solution|https://github.com/apache/zeppelin/commit/9f22db91c279b7daf6a13b2d805a874074b070fd] for the task [ZEPPELIN-2075|https://issues.apache.org/jira/browse/ZEPPELIN-2075].

This solution is causing that when one particular user cancels his py-spark job, the py-spark jobs from all the users are being canceled !!.

When a py-spark job is cancelled, the method PySparkInterpreter interrupt() is invoked, and then the SIGINT event is called, causing that all the jobs in the same spark context be cancelled:

context.py:

# create a signal handler which would be invoked on receiving SIGINT
def signal_handler(signal, frame):
    self.cancelAllJobs()
    raise KeyboardInterrupt()



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)