[jira] [Created] (SPARK-15499) Add python testsuite with remote debug and single test parameter to help developer debug code easier.

["Weichen Xu (JIRA)" <ji...@apache.org>]

Weichen Xu created SPARK-15499:
----------------------------------

             Summary: Add python testsuite with remote debug and single test parameter to help developer debug code easier.
                 Key: SPARK-15499
                 URL: https://issues.apache.org/jira/browse/SPARK-15499
             Project: Spark
          Issue Type: Improvement
          Components: PySpark, Tests
    Affects Versions: 2.0.0
            Reporter: Weichen Xu


To python/run-tests.py script, I add the following parameters:

  --single-test=SINGLE_TEST
                        specify a python module to run single python test.
  --debug-server=DEBUG_SERVER
                        debug server host, only used in single test.
  --debug-port=DEBUG_PORT
                        debug server port, only used in single test.

Now, for example, I want to debug only pyspark.tests, I can use the following command:
(first startup debug server in your python IDE such as pycharm or pydev, your IED on machine which has host MY_DEV_MACHINE and using debug port 5678, and make sure the machine running test can connect to MY_DEV_MACHINE)

python/run-tests --python-executables=python2.7 --single-test=pyspark.tests --debug-server=MY_DEV_MACHINE --debug-port=5678

the parameter --single-test specify the single python testsuite you want to test, currently you can use the following value:
pyspark.tests
pyspark.ml.tests
pyspark.mllib.tests
pyspark.sql.tests
pyspark.streaming.tests

Adding parameters and develop and debug python script will be quite easier, I think.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org