You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@predictionio.apache.org by "VI, Tran Tan Phong" <tp...@prosodie.com> on 2018/03/27 10:09:30 UTC

Error when training The Universal Recommender 0.7.0 with PredictionIO 0.12.0-incubating

Hi,

I am trying to build and train UR 0.7.0 with PredictionIO 0.12.0-incubating on a local "Small HA Cluster" (http://actionml.com/docs/small_ha_cluster) using Elasticsearch 5.5.2.
By following different steps of the how-to, I success to execute the "pio build" command of U.R 7.0. But I am getting some errors on the following step of "pio train".

Here are the principal errors:
...
[INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
[INFO] [HttpMethodDirector] Retrying request
[INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
[INFO] [HttpMethodDirector] Retrying request
[INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
[INFO] [HttpMethodDirector] Retrying request
[ERROR] [NetworkClient] Node [127.0.0.1:9200] failed (Connection refused (Connection refused)); no other nodes left - aborting...
...

Exception in thread "main" org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
...
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[127.0.0.1:9200]]

The cluster Elasticsearch (aml-elasticsearch) is up, but is not listening on localhost.

Here under is my config of ES 5.5.2
PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=aml-elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=aml-master,aml-slave-1,aml-slave-2
PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200,9200,9200
PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/usr/local/elasticsearch

Did somebody get this kind of error before? Any help or suggestion would be appreciated.

Thanks,
VI Tran Tan Phong
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

RE: Error when training The Universal Recommender 0.7.0 with PredictionIO 0.12.0-incubating

Posted by "VI, Tran Tan Phong" <tp...@prosodie.com>.
Yeah, this command passes, thank you!

Phong

De : Pat Ferrel [mailto:pat@occamsmachete.com]
Envoyé : mardi 27 mars 2018 17:44
À : user@predictionio.apache.org; VI, Tran Tan Phong; actionml-user
Objet : Re: Error when training The Universal Recommender 0.7.0 with PredictionIO 0.12.0-incubating

Pio build requires that ES hosts are known to Spark, which write the model to ES. You can pass these in on the `pio train` command line:

pio train … -- --conf spark.es<http://spark.es>.nodes=“node1,node2,node3”

notice no spaces in the quoted list of hosts, also notice the double dash, which separates pio parameters from Spark parameters.

There is a way to pass this in using the sparkConf section in engine.json but this is unreliable due to how the commas are treated in ES. The site description for the UR in the small HA cluster has not been updated for 0.7.0 because we are expecting a Mahout release, which will greatly simplfy the build process described in the README.


From: VI, Tran Tan Phong <tp...@prosodie.com>
Reply: user@predictionio.apache.org<ma...@predictionio.apache.org> <us...@predictionio.apache.org>
Date: March 27, 2018 at 3:09:30 AM
To: user@predictionio.apache.org<ma...@predictionio.apache.org> <us...@predictionio.apache.org>
Subject:  Error when training The Universal Recommender 0.7.0 with PredictionIO 0.12.0-incubating


Hi,

I am trying to build and train UR 0.7.0 with PredictionIO 0.12.0-incubating on a local “Small HA Cluster” (http://actionml.com/docs/small_ha_cluster) using Elasticsearch 5.5.2.
By following different steps of the how-to, I success to execute the “pio build” command of U.R 7.0. But I am getting some errors on the following step of “pio train”.

Here are the principal errors:
…
[INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
[INFO] [HttpMethodDirector] Retrying request
[INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
[INFO] [HttpMethodDirector] Retrying request
[INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection refused (Connection refused)
[INFO] [HttpMethodDirector] Retrying request
[ERROR] [NetworkClient] Node [127.0.0.1:9200<http://127.0.0.1:9200>] failed (Connection refused (Connection refused)); no other nodes left - aborting...
…

Exception in thread "main" org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
…
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[127.0.0.1:9200<http://127.0.0.1:9200>]]

The cluster Elasticsearch (aml-elasticsearch) is up, but is not listening on localhost.

Here under is my config of ES 5.5.2
PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=aml-elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=aml-master,aml-slave-1,aml-slave-2
PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200,9200,9200
PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/usr/local/elasticsearch

Did somebody get this kind of error before? Any help or suggestion would be appreciated.

Thanks,
VI Tran Tan Phong
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.
This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message.

Re: Error when training The Universal Recommender 0.7.0 with PredictionIO 0.12.0-incubating

Posted by Pat Ferrel <pa...@occamsmachete.com>.
Pio build requires that ES hosts are known to Spark, which write the model
to ES. You can pass these in on the `pio train` command line:

pio train … -- --conf spark.es.nodes=“node1,node2,node3”

notice no spaces in the quoted list of hosts, also notice the double dash,
which separates pio parameters from Spark parameters.

There is a way to pass this in using the sparkConf section in engine.json
but this is unreliable due to how the commas are treated in ES. The site
description for the UR in the small HA cluster has not been updated for
0.7.0 because we are expecting a Mahout release, which will greatly simplfy
the build process described in the README.


From: VI, Tran Tan Phong <tp...@prosodie.com> <tp...@prosodie.com>
Reply: user@predictionio.apache.org <us...@predictionio.apache.org>
<us...@predictionio.apache.org>
Date: March 27, 2018 at 3:09:30 AM
To: user@predictionio.apache.org <us...@predictionio.apache.org>
<us...@predictionio.apache.org>
Subject:  Error when training The Universal Recommender 0.7.0 with
PredictionIO 0.12.0-incubating

Hi,



I am trying to build and train UR 0.7.0 with PredictionIO 0.12.0-incubating
on a local “Small HA Cluster” (http://actionml.com/docs/small_ha_cluster)
using Elasticsearch 5.5.2.

By following different steps of the how-to, I success to execute the “pio
build” command of U.R 7.0. But I am getting some errors on the following
step of “pio train”.



Here are the principal errors:

…

[INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException)
caught when processing request: Connection refused (Connection refused)

[INFO] [HttpMethodDirector] Retrying request

[INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException)
caught when processing request: Connection refused (Connection refused)

[INFO] [HttpMethodDirector] Retrying request

[INFO] [HttpMethodDirector] I/O exception (java.net.ConnectException)
caught when processing request: Connection refused (Connection refused)

[INFO] [HttpMethodDirector] Retrying request

[ERROR] [NetworkClient] Node [127.0.0.1:9200] failed (Connection refused
(Connection refused)); no other nodes left - aborting...

…



Exception in thread "main"
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES
version - typically this happens if the network/Elasticsearch cluster is
not accessible or when targeting a WAN/Cloud instance without the proper
setting 'es.nodes.wan.only'

…

        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException:
Connection error (check network and/or proxy settings)- all nodes failed;
tried [[127.0.0.1:9200]]



The cluster Elasticsearch (aml-elasticsearch) is up, but is not listening
on localhost.



Here under is my config of ES 5.5.2

PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch

PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=aml-elasticsearch

PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=aml-master,aml-slave-1,aml-slave-2

PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200,9200,9200

PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http

PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/usr/local/elasticsearch



Did somebody get this kind of error before? Any help or suggestion would be
appreciated.



Thanks,

VI Tran Tan Phong
This message contains information that may be privileged or confidential
and is the property of the Capgemini Group. It is intended only for the
person to whom it is addressed. If you are not the intended recipient, you
are not authorized to read, print, retain, copy, disseminate, distribute,
or use this message or any part thereof. If you receive this message in
error, please notify the sender immediately and delete all copies of this
message.