[jira] [Created] (SPARK-24623) Hadoop - Spark Cluster - Python XGBoost - Not working in distributed mode

["Abhishek Reddy Chamakura (JIRA)" <ji...@apache.org>]

Abhishek Reddy Chamakura created SPARK-24623:
------------------------------------------------

             Summary: Hadoop - Spark Cluster - Python XGBoost - Not working in distributed mode
                 Key: SPARK-24623
                 URL: https://issues.apache.org/jira/browse/SPARK-24623
             Project: Spark
          Issue Type: Bug
          Components: Deploy
    Affects Versions: 2.1.1
         Environment: Hadoop - Hortonworks Cluster

 

Total Nodes - 18

Worker Nodes - 13
            Reporter: Abhishek Reddy Chamakura


Hi


We recently installed python on the Hadoop cluster with lot of data science python modules including xgboost , spicy , scikit learn , pandas
Using pyspark the data scientists are able to test there scoring models in the distributed mode on the Hadoop cluster. But with python - xgboost the pyspark job is not getting distributed and it is trying to run only on one instance.
we are trying to achieve the distributed mode when using python xgboost via pyspark. 
It would be a great help if you can direct me on how to achieve this.

Thanks,
Abhishek



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org