You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@predictionio.apache.org by Wojciech Kowalski <wo...@tomandco.co.uk> on 2018/03/08 15:31:05 UTC
UR 0.7.0 - problem with training
Hello, i am trying to set new UR 0.7.0 with predictionio 0.12.0 but all atempts are failing.
I cannot set in engine’s spark config section „es.config” as i get such error:
org.elasticsearch.index.mapper.MapperParsingException: object mapping for [sparkConf.es.nodes] tried to parse field [es.nodes] as object, but found a concrete value
If i don’t set this up engine fail to train because it cannot find elasticsearch on localhost as it’s running on a separate machine
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[localhost:9200]]
Passing es.nodes via cli –conf es.nodes=elasticsearch doesn’t help either :/
pio train -- --conf es.nodes=elasticsearch --driver-memory 8g --executor-memory 8g
Anyone would give any advice what am i doing wrong ?
I have separate docker containers for hadoop,hbase,elasticsearch,pio
Same setup was working fine on 0.10 and UR 0.5
Thanks,
Wojciech Kowalski
ODP: UR 0.7.0 - problem with training
Posted by Wojciech Kowalski <wo...@tomandco.co.uk>.
Thanks guys,
using cli param spark.es.nodes fixed problem
but still I think someone should look at engine.json param why it’s not parsing properly and causing issues, not sure if it’s related to PIO itself or UR engine tho.
Anyway using in train –conf spark.es.nodes=elasticsearch is working while engine.json es.nodes=”elasticsearch” throw error as posted in first mail:
org.elasticsearch.index.mapper.MapperParsingException: object mapping for [sparkConf.es.nodes] tried to parse field [es.nodes] as object, but found a concrete value
Thanks for all Your help guys!
Wojciech
Re: UR 0.7.0 - problem with training
Posted by Pat Ferrel <pa...@occamsmachete.com>.
BTW I think you may have to push setting on the cli by adding “spark” to
the beginning of the key name:
*pio train -- --conf spark.es.nodes=**“**localhost" --driver-memory 8g
--executor-memory 8g*
From: Pat Ferrel <pa...@occamsmachete.com> <pa...@occamsmachete.com>
Reply: user@predictionio.apache.org <us...@predictionio.apache.org>
<us...@predictionio.apache.org>
Date: March 8, 2018 at 11:04:55 AM
To: Wojciech Kowalski <wo...@tomandco.co.uk> <wo...@tomandco.co.uk>,
user@predictionio.apache.org <us...@predictionio.apache.org>
<us...@predictionio.apache.org>, user@predictionio.incubator.apache.org
<us...@predictionio.incubator.apache.org>
<us...@predictionio.incubator.apache.org>
Subject: Re: UR 0.7.0 - problem with training
es.nodes is supposed to be a string with hostnames separated by commas.
Depending on how your containers are set to communicate with the outside
world (Docker networking or port mapping) you may also need to set the
port, which is 9200 by default.
If your container is using port mapping and maps the container port 9200 to
the localhost port of 9200 you should be ok with only setting hostnames in
engine.json.
es.nodes=“localhost”
But I suspect you didn’t set your container communication strategy because
this is the fallback that would have been tried with no valid setting.
If this is true look up how you set Docker to communicate, port mapping is
the simplest for a single all-in-one machine.
From: Wojciech Kowalski <wo...@tomandco.co.uk> <wo...@tomandco.co.uk>
Reply: user@predictionio.apache.org <us...@predictionio.apache.org>
<us...@predictionio.apache.org>
Date: March 8, 2018 at 7:31:10 AM
To: user@predictionio.incubator.apache.org
<us...@predictionio.incubator.apache.org>
<us...@predictionio.incubator.apache.org>
Subject: UR 0.7.0 - problem with training
Hello, i am trying to set new UR 0.7.0 with predictionio 0.12.0 but all
atempts are failing.
I cannot set in engine’s spark config section „es.config” as i get such
error:
org.elasticsearch.index.mapper.MapperParsingException: object mapping for
[sparkConf.es.nodes] tried to parse field [es.nodes] as object, but found a
concrete value
If i don’t set this up engine fail to train because it cannot find
elasticsearch on localhost as it’s running on a separate machine
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException:
Connection error (check network and/or proxy settings)- all nodes failed;
tried [[localhost:9200]]
Passing es.nodes via cli –conf es.nodes=elasticsearch doesn’t help either :/
*pio train -- --conf es.nodes=elasticsearch --driver-memory 8g
--executor-memory 8g*
Anyone would give any advice what am i doing wrong ?
I have separate docker containers for hadoop,hbase,elasticsearch,pio
Same setup was working fine on 0.10 and UR 0.5
Thanks,
Wojciech Kowalski
Re: UR 0.7.0 - problem with training
Posted by Pat Ferrel <pa...@occamsmachete.com>.
es.nodes is supposed to be a string with hostnames separated by commas.
Depending on how your containers are set to communicate with the outside
world (Docker networking or port mapping) you may also need to set the
port, which is 9200 by default.
If your container is using port mapping and maps the container port 9200 to
the localhost port of 9200 you should be ok with only setting hostnames in
engine.json.
es.nodes=“localhost”
But I suspect you didn’t set your container communication strategy because
this is the fallback that would have been tried with no valid setting.
If this is true look up how you set Docker to communicate, port mapping is
the simplest for a single all-in-one machine.
From: Wojciech Kowalski <wo...@tomandco.co.uk> <wo...@tomandco.co.uk>
Reply: user@predictionio.apache.org <us...@predictionio.apache.org>
<us...@predictionio.apache.org>
Date: March 8, 2018 at 7:31:10 AM
To: user@predictionio.incubator.apache.org
<us...@predictionio.incubator.apache.org>
<us...@predictionio.incubator.apache.org>
Subject: UR 0.7.0 - problem with training
Hello, i am trying to set new UR 0.7.0 with predictionio 0.12.0 but all
atempts are failing.
I cannot set in engine’s spark config section „es.config” as i get such
error:
org.elasticsearch.index.mapper.MapperParsingException: object mapping for
[sparkConf.es.nodes] tried to parse field [es.nodes] as object, but found a
concrete value
If i don’t set this up engine fail to train because it cannot find
elasticsearch on localhost as it’s running on a separate machine
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException:
Connection error (check network and/or proxy settings)- all nodes failed;
tried [[localhost:9200]]
Passing es.nodes via cli –conf es.nodes=elasticsearch doesn’t help either :/
*pio train -- --conf es.nodes=elasticsearch --driver-memory 8g
--executor-memory 8g*
Anyone would give any advice what am i doing wrong ?
I have separate docker containers for hadoop,hbase,elasticsearch,pio
Same setup was working fine on 0.10 and UR 0.5
Thanks,
Wojciech Kowalski