You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@predictionio.apache.org by amal kumar <am...@gmail.com> on 2016/10/13 09:50:17 UTC
PreditionIO with Hbase as Event Server on Separate machine Error
Hi,
I am trying to use Hbase as event server with predictionIO. Which is
working fine as expected.
But I am getting error while point Hbase installed on a separate machine
other than predictionIO.
As per the understanding, I have done the below steps:
1. Copied the hbase conf folder (from the machin on Hbase is running) that
contains hbase-site.xml to any folder on prediction io server on some path
i.e. "/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf"
2. Updated HBASE_CONF_DIR in pio-env.sh to point to the path of copied
folder i.e.
HBASE_CONF_DIR=/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf
PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
PIO_STORAGE_SOURCES_HBASE_HOME=/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf
3. Updated the property "hbase.zookeeper.quorum" in hbase-site.xml on local
copied folder i.e.
"/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf" i.e.
<configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value><Master public DNS></value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://ip-xx-xx-xx-xxx.us-west-2.compute.internal:8020/user/hbase</value>
</property>
<property>
<name>dfs.support.append</name>
<value>true</value>
</property>
But I am getting the below error.
*[INFO] [Console$] Inspecting PredictionIO...*
*[INFO] [Console$] PredictionIO 0.9.6 is installed at
/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6*
*[INFO] [Console$] Inspecting Apache Spark...*
*[INFO] [Console$] Apache Spark is installed at
/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/vendors/spark-1.6.2-bin-hadoop2.6*
*[INFO] [Console$] Apache Spark 1.6.2 detected (meets minimum requirement
of 1.3.0)*
*[INFO] [Console$] Inspecting storage backend connections...*
*[INFO] [Storage$] Verifying Meta Data Backend (Source: PGSQL)...*
*[INFO] [Storage$] Verifying Model Data Backend (Source: PGSQL)...*
*[INFO] [Storage$] Verifying Event Data Backend (Source: HBASE)...*
*[INFO] [Storage$] Test writing to Event Store (App Id 0)...*
*[ERROR] [Console$] Unable to connect to all storage backends successfully.
The following shows the error message from the storage backend.*
*[ERROR] [Console$] Failed after attempts=1, exceptions:*
*Thu Oct 13 09:36:53 UTC 2016,
org.apache.hadoop.hbase.client.RpcRetryingCaller@11b377c5,
org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while
waiting for channel to be ready for connect. ch :
java.nio.channels.SocketChannel[connection-pending
remote=ip-xxx-xx-xx-xx.us-west-2.compute.internal/xxx.xx.xx.xx:16020]*
* (org.apache.hadoop.hbase.client.RetriesExhaustedException)*
*[ERROR] [Console$] Dumping configuration of initialized storage backend
sources. Please make sure they are correct.*
*[ERROR] [Console$] Source Name: PGSQL; Type: jdbc; Configuration: URL ->
jdbc:postgresql://localhost/pio, PASSWORD -> pio, TYPE -> jdbc, USERNAME ->
pio*
*[ERROR] [Console$] Source Name: HBASE; Type: hbase; Configuration: TYPE ->
hbase, HOSTS -> 0, HOME ->
/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf, PORTS -> 0*
Can you please help?
Thanks,
Amal Kumar
Re: PreditionIO with Hbase as Event Server on Separate machine Error
Posted by amal kumar <am...@gmail.com>.
Thanks.
This has been resolved. It was a security group issue.
On Sat, Oct 15, 2016 at 9:34 PM, Pat Ferrel <pa...@occamsmachete.com> wrote:
> can you send your pio-env.sh?
>
>
> On Oct 13, 2016, at 2:50 AM, amal kumar <am...@gmail.com> wrote:
>
> Hi,
>
> I am trying to use Hbase as event server with predictionIO. Which is
> working fine as expected.
>
> But I am getting error while point Hbase installed on a separate machine
> other than predictionIO.
>
> As per the understanding, I have done the below steps:
>
> 1. Copied the hbase conf folder (from the machin on Hbase is running) that
> contains hbase-site.xml to any folder on prediction io server on some path
> i.e. "/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf"
>
> 2. Updated HBASE_CONF_DIR in pio-env.sh to point to the path of copied
> folder i.e.
>
> HBASE_CONF_DIR=/home/ubuntu/hbasePrediction-io/
> PredictionIO-0.9.6/hbase/conf
>
> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>
> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
> PIO_STORAGE_SOURCES_HBASE_HOME=/home/ubuntu/hbasePrediction-io/
> PredictionIO-0.9.6/hbase/conf
>
> 3. Updated the property "hbase.zookeeper.quorum" in hbase-site.xml on
> local copied folder i.e. "/home/ubuntu/hbasePrediction-
> io/PredictionIO-0.9.6/hbase/conf" i.e.
>
>
> <configuration>
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> </property>
>
> <property>
> <name>hbase.zookeeper.quorum</name>
> <value><Master public DNS></value>
> </property>
>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://ip-xx-xx-xx-xxx.us-west-2.compute.internal:
> 8020/user/hbase</value>
> </property>
>
> <property>
> <name>dfs.support.append</name>
> <value>true</value>
> </property>
>
>
>
>
> But I am getting the below error.
>
>
> *[INFO] [Console$] Inspecting PredictionIO...*
> *[INFO] [Console$] PredictionIO 0.9.6 is installed at
> /home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6*
> *[INFO] [Console$] Inspecting Apache Spark...*
> *[INFO] [Console$] Apache Spark is installed at
> /home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/vendors/spark-1.6.2-bin-hadoop2.6*
> *[INFO] [Console$] Apache Spark 1.6.2 detected (meets minimum requirement
> of 1.3.0)*
> *[INFO] [Console$] Inspecting storage backend connections...*
> *[INFO] [Storage$] Verifying Meta Data Backend (Source: PGSQL)...*
> *[INFO] [Storage$] Verifying Model Data Backend (Source: PGSQL)...*
> *[INFO] [Storage$] Verifying Event Data Backend (Source: HBASE)...*
> *[INFO] [Storage$] Test writing to Event Store (App Id 0)...*
> *[ERROR] [Console$] Unable to connect to all storage backends
> successfully. The following shows the error message from the storage
> backend.*
> *[ERROR] [Console$] Failed after attempts=1, exceptions:*
> *Thu Oct 13 09:36:53 UTC 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@11b377c5,
> org.apache.hadoop.net
> <http://org.apache.hadoop.net>.ConnectTimeoutException: 20000 millis
> timeout while waiting for channel to be ready for connect. ch :
> java.nio.channels.SocketChannel[connection-pending
> remote=ip-xxx-xx-xx-xx.us
> <http://ip-xxx-xx-xx-xx.us>-west-2.compute.internal/xxx.xx.xx.xx:16020]*
> * (org.apache.hadoop.hbase.client.RetriesExhaustedException)*
> *[ERROR] [Console$] Dumping configuration of initialized storage backend
> sources. Please make sure they are correct.*
> *[ERROR] [Console$] Source Name: PGSQL; Type: jdbc; Configuration: URL ->
> jdbc:postgresql://localhost/pio, PASSWORD -> pio, TYPE -> jdbc, USERNAME ->
> pio*
> *[ERROR] [Console$] Source Name: HBASE; Type: hbase; Configuration: TYPE
> -> hbase, HOSTS -> 0, HOME ->
> /home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf, PORTS -> 0*
>
>
>
> Can you please help?
>
>
>
> Thanks,
> Amal Kumar
>
>
>
>
Re: PreditionIO with Hbase as Event Server on Separate machine Error
Posted by Pat Ferrel <pa...@occamsmachete.com>.
can you send your pio-env.sh?
On Oct 13, 2016, at 2:50 AM, amal kumar <am...@gmail.com> wrote:
Hi,
I am trying to use Hbase as event server with predictionIO. Which is working fine as expected.
But I am getting error while point Hbase installed on a separate machine other than predictionIO.
As per the understanding, I have done the below steps:
1. Copied the hbase conf folder (from the machin on Hbase is running) that contains hbase-site.xml to any folder on prediction io server on some path i.e. "/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf"
2. Updated HBASE_CONF_DIR in pio-env.sh to point to the path of copied folder i.e.
HBASE_CONF_DIR=/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf
PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
PIO_STORAGE_SOURCES_HBASE_HOME=/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf
3. Updated the property "hbase.zookeeper.quorum" in hbase-site.xml on local copied folder i.e. "/home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf" i.e.
<configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value><Master public DNS></value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://ip-xx-xx-xx-xxx.us-west-2.compute.internal:8020/user/hbase</value>
</property>
<property>
<name>dfs.support.append</name>
<value>true</value>
</property>
But I am getting the below error.
[INFO] [Console$] Inspecting PredictionIO...
[INFO] [Console$] PredictionIO 0.9.6 is installed at /home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6
[INFO] [Console$] Inspecting Apache Spark...
[INFO] [Console$] Apache Spark is installed at /home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/vendors/spark-1.6.2-bin-hadoop2.6
[INFO] [Console$] Apache Spark 1.6.2 detected (meets minimum requirement of 1.3.0)
[INFO] [Console$] Inspecting storage backend connections...
[INFO] [Storage$] Verifying Meta Data Backend (Source: PGSQL)...
[INFO] [Storage$] Verifying Model Data Backend (Source: PGSQL)...
[INFO] [Storage$] Verifying Event Data Backend (Source: HBASE)...
[INFO] [Storage$] Test writing to Event Store (App Id 0)...
[ERROR] [Console$] Unable to connect to all storage backends successfully. The following shows the error message from the storage backend.
[ERROR] [Console$] Failed after attempts=1, exceptions:
Thu Oct 13 09:36:53 UTC 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@11b377c5, org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=ip-xxx-xx-xx-xx.us-west-2.compute.internal/xxx.xx.xx.xx:16020]
(org.apache.hadoop.hbase.client.RetriesExhaustedException)
[ERROR] [Console$] Dumping configuration of initialized storage backend sources. Please make sure they are correct.
[ERROR] [Console$] Source Name: PGSQL; Type: jdbc; Configuration: URL -> jdbc:postgresql://localhost/pio, PASSWORD -> pio, TYPE -> jdbc, USERNAME -> pio
[ERROR] [Console$] Source Name: HBASE; Type: hbase; Configuration: TYPE -> hbase, HOSTS -> 0, HOME -> /home/ubuntu/hbasePrediction-io/PredictionIO-0.9.6/hbase/conf, PORTS -> 0
Can you please help?
Thanks,
Amal Kumar