Re: Error with a Cluster : ConnectionRefused

[Pat Ferrel <pa...@occamsmachete.com>]

We (ActionML) do contract and consulting based on PIO and I can assure you from many installations that it works quite well with clustered HDFS, HBase, Spark, and Elasticsearch. For truly scalable setup I’d recommend clusters for Spark, HBase+HDFS, and ES 5.x. So 3 different ones.

If you are using the ActionML setup instructions you must be using the Universal Recommender? This was broken by the latest release of PIO 0.12.0 but we have an RC that will be released with the next Mahout release (should be next week). You can use it now by following some interim build instructions (see the UR-0.7.0-SNAPSHOT readme here: https://github.com/actionml/universal-recommender/tree/0.7.0-SNAPSHOT <https://github.com/actionml/universal-recommender/tree/0.7.0-SNAPSHOT>). BTW the page you reference is for PIO 0.11.0 and is being updated as I write.

If you are using the UR you do not want an HDFS storage backend, which is being checked by `pio status`. Can you share your pio-env? Even if you are not using the UR my theory is that something is setup in pio-env wrong because using clustered HDFS for a store is quite typical and well tested.


On Nov 28, 2017, at 8:49 AM, Thibaut Gensollen - Choose <th...@appchoose.io> wrote:

Hi Guys,

I tried to set up a Predictionio cluster, working with an elasticsearch and hadoop clusters. 

The fact is, everything is working well (Hbase, ES or Hadoop/Spark), and as soon as I tried « pio-start-all » I am getting these errors:




And then, my hadoop is not working anymore :

hdfs dfs -mkdir test
mkdir: Call From master.c.choose-ninja-01.internal/10.128.0.4 to master.c.choose-ninja-01.internal:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused <http://wiki.apache.org/hadoop/ConnectionRefused>

I tried working with a gcloud dataproc, or following your tutorial http://actionml.com/docs/small_ha_cluster <http://actionml.com/docs/small_ha_cluster> with some vm engines (still on gcloud), but the results are the same.. Any Idea ? We tried so many times that I am beginning thinking that we cannot use predictionio with an hadoop and elasticsearch clusters :/

Thanks for your help,
Regards

Thibaut
—





Thibaut Gensollen
CTO
+33 6 76 73 08 50 <https://appchoose.io/>