You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@predictionio.apache.org by Pawan Agnihotri <pa...@gmail.com> on 2018/03/07 06:21:03 UTC
[ERROR] [Storage$] Required repository (METADATA) configuration is missing.
Hello,
I need your help to configure predictionIO on linux 7.2 -
I am using
http://predictionio.apache.org/install/install-sourcecode/ link for steps
and installed spark, elastic search and hbase but getting below error
[mapr@valvcshad004vm bin]$ ./pio status
/dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89:
/opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such file
or directory
/dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90:
generate_compatible_classpath: command not found
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/dfs/pawan_scala/mapr-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/data/opt/mapr/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[INFO] [Console$] Inspecting PredictionIO...
[INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at
/dfs/pawan_scala/mapr-predictionio
[INFO] [Console$] Inspecting Apache Spark...
[INFO] [Console$] Apache Spark is installed at
/dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6
[INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum requirement of
1.3.0)
[INFO] [Console$] Inspecting storage backend connections...
*[WARN] [Storage$] There is no properly configured repository.*
*[ERROR] [Storage$] Required repository (METADATA) configuration is
missing.*
*[ERROR] [Storage$] There were 1 configuration errors. Exiting.*
[mapr@valvcshad004vm bin]$
Here is my *pio-env.sh *file
[mapr@valvcshad004vm conf]$ cat pio-env.sh
# Default is to use PostgreSQL
#PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
#PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
#PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
#PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
#PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
#PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
# Storage Data Sources
# PostgreSQL Default Settings
# Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL
# Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
#PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
#PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
#PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
#$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
# MySQL Example
# PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
# PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
# PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
# PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
# Elasticsearch Example
PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch_cluster_name
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2
# Local File System Example
PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
# HBase Example
PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/hbase-1.2.6
[mapr@valvcshad004vm conf]$
--
Thanks,
Pawan Agnihotri
Re: [ERROR] [Storage$] Required repository (METADATA) configuration
is missing.
Posted by Pawan Agnihotri <pa...@gmail.com>.
Thank you Mars.
On Fri, Mar 9, 2018 at 12:42 PM, Mars Hall <ma...@salesforce.com> wrote:
> It looks like you're using an old version of PIO:
>
> PredictionIO 0.10.0-SNAPSHOT
>>
>
> Elasticsearch 5 does not work with that old version.
>
> Please upgrade to PIO 0.12.0. Download/install the newer distribution, and
> then update the engine's `build.sbt` & `template.json`.
>
> On Fri, Mar 9, 2018 at 12:14 AM, Pawan Agnihotri <
> pawan.agnihotri@gmail.com> wrote:
>
>> Thanks Mars for the response. I tried adding the scheme and port to 9200
>> for elasticsearch in conf/pio-env.sh but still getting the same error which
>> checking the status.
>>
>> here is my conf for elasticsearch and other logs. Please let me know if
>> you need more details. thank you.
>>
>> *PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch*
>> *#PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch*
>> *PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost*
>> *PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http*
>> *PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200*
>>
>> *PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2*
>>
>>
>> [mapr@valvcshad004vm bin]$ ./pio status
>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89:
>> /opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such
>> file or directory
>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90:
>> generate_compatible_classpath: command not found
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in [jar:file:/dfs/pawan_scala/map
>> r-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT.jar!/
>> org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in [jar:file:/data/opt/mapr/lib/s
>> lf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> [INFO] [Console$] Inspecting PredictionIO...
>> [INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at
>> /dfs/pawan_scala/mapr-predictionio
>> [INFO] [Console$] Inspecting Apache Spark...
>> [INFO] [Console$] Apache Spark is installed at
>> /dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6
>> [INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum requirement
>> of 1.3.0)
>> [INFO] [Console$] Inspecting storage backend connections...
>> [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
>> [ERROR] [Console$] Unable to connect to all storage backends
>> successfully. The following shows the error message from the storage
>> backend.
>> [ERROR] [Console$] None of the configured nodes are available: []
>> (org.elasticsearch.client.transport.NoNodeAvailableException)
>> [ERROR] [Console$] Dumping configuration of initialized storage backend
>> sources. Please make sure they are correct.
>> [ERROR] [Console$] Source Name: ELASTICSEARCH; Type: elasticsearch;
>> Configuration: HOME -> /dfs/pawan_scala/mapr-predicti
>> onio/vendors/elasticsearch-5.5.2, HOSTS -> localhost, PORTS -> 9200,
>> SCHEMES -> http, TYPE -> elasticsearch
>> [mapr@valvcshad004vm bin]$
>>
>> ------------
>> [mapr@valvcshad004vm bin]$ cat pio.log
>>
>> 2018-03-08 19:08:54,371 INFO org.apache.predictionio.tools.console.Console$
>> [main] - Inspecting PredictionIO...
>> 2018-03-08 19:08:54,375 INFO org.apache.predictionio.tools.console.Console$
>> [main] - PredictionIO 0.10.0-SNAPSHOT is installed at
>> /dfs/pawan_scala/mapr-predictionio
>> 2018-03-08 19:08:54,376 INFO org.apache.predictionio.tools.console.Console$
>> [main] - Inspecting Apache Spark...
>> 2018-03-08 19:08:54,389 INFO org.apache.predictionio.tools.console.Console$
>> [main] - Apache Spark is installed at /dfs/pawan_scala/mapr-predicti
>> onio/vendors/spark-2.1.1-bin-hadoop2.6
>> 2018-03-08 19:08:54,427 INFO org.apache.predictionio.tools.console.Console$
>> [main] - Apache Spark 2.1.1 detected (meets minimum requirement of 1.3.0)
>> 2018-03-08 19:08:54,428 INFO org.apache.predictionio.tools.console.Console$
>> [main] - Inspecting storage backend connections...
>> 2018-03-08 19:08:54,450 INFO org.apache.predictionio.data.storage.Storage$
>> [main] - Verifying Meta Data Backend (Source: ELASTICSEARCH)...
>> 2018-03-08 19:08:55,636 ERROR org.apache.predictionio.tools.console.Console$
>> [main] - Unable to connect to all storage backends successfully. The
>> following shows the error message from the storage backend.
>> 2018-03-08 19:08:55,638 ERROR org.apache.predictionio.tools.console.Console$
>> [main] - None of the configured nodes are available: []
>> (org.elasticsearch.client.transport.NoNodeAvailableException)
>> org.elasticsearch.client.transport.NoNodeAvailableException: None of the
>> configured nodes are available: []
>> at org.elasticsearch.client.transport.TransportClientNodesServi
>> ce.ensureNodesAreAvailable(TransportClientNodesService.java:305)
>> at org.elasticsearch.client.transport.TransportClientNodesServi
>> ce.execute(TransportClientNodesService.java:200)
>> at org.elasticsearch.client.transport.support.InternalTransport
>> IndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86)
>> at org.elasticsearch.client.support.AbstractIndicesAdminClient.
>> exists(AbstractIndicesAdminClient.java:178)
>> at org.elasticsearch.action.admin.indices.exists.indices.Indice
>> sExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:53)
>> at org.elasticsearch.action.ActionRequestBuilder.execute(Action
>> RequestBuilder.java:91)
>> at org.elasticsearch.action.ActionRequestBuilder.execute(Action
>> RequestBuilder.java:65)
>> at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequ
>> estBuilder.java:73)
>> at org.apache.predictionio.data.storage.elasticsearch.ESEngineI
>> nstances.<init>(ESEngineInstances.scala:42)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native
>> ConstructorAccessorImpl.java:62)
>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De
>> legatingConstructorAccessorImpl.java:45)
>> at java.lang.reflect.Constructor.newInstance(Constructor.java:4
>> 23)
>> at org.apache.predictionio.data.storage.Storage$.getDataObject(
>> Storage.scala:306)
>> at org.apache.predictionio.data.storage.Storage$.getDataObjectF
>> romRepo(Storage.scala:266)
>> at org.apache.predictionio.data.storage.Storage$.getMetaDataEng
>> ineInstances(Storage.scala:367)
>> at org.apache.predictionio.data.storage.Storage$.verifyAllDataO
>> bjects(Storage.scala:342)
>> at org.apache.predictionio.tools.console.Console$.status(Consol
>> e.scala:1087)
>> at org.apache.predictionio.tools.console.Console$$anonfun$main$
>> 1.apply(Console.scala:737)
>> at org.apache.predictionio.tools.console.Console$$anonfun$main$
>> 1.apply(Console.scala:696)
>> at scala.Option.map(Option.scala:145)
>> at org.apache.predictionio.tools.console.Console$.main(Console.
>> scala:696)
>> at org.apache.predictionio.tools.console.Console.main(Console.s
>> cala)
>> 2018-03-08 19:08:55,641 ERROR org.apache.predictionio.tools.console.Console$
>> [main] - Dumping configuration of initialized storage backend sources.
>> Please make sure they are correct.
>> 2018-03-08 19:08:55,644 ERROR org.apache.predictionio.tools.console.Console$
>> [main] - Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration:
>> HOME -> /dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2,
>> HOSTS -> localhost, PORTS -> 9200, SCHEMES -> http, TYPE -> elasticsearch
>> [mapr@valvcshad004vm bin]$
>>
>>
>> On Thu, Mar 8, 2018 at 1:30 PM, Mars Hall <ma...@salesforce.com>
>> wrote:
>>
>>> We added support for Elasticsearch 5 last year, so current PredictionIO
>>> uses HTTP/REST protocol on port 9200, not the native protocol on port 9300.
>>>
>>> Here's the local dev config I have working with PIO 0.12.0 and
>>> Elasticsearch 5:
>>>
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/ela
>>> sticsearch
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
>>>
>>> I believe Elasticsearch 1 is only supported via native protocol on port
>>> 9300, which is eventually being removed in a future PIO release.
>>>
>>> On Thu, Mar 8, 2018 at 2:50 AM, Pawan Agnihotri <
>>> pawan.agnihotri@gmail.com> wrote:
>>>
>>>> Hello Donald and Team,
>>>>
>>>> I am working POC and I would like to use predictionIO.I know its
>>>> configuration issue with elasticsearch but I am kind of stuck with below
>>>> error so reaching out for help.
>>>>
>>>> I am in need of some quick hand here as the time is running out.
>>>> anything you feel I can try out or steps would be helpful please.
>>>>
>>>>
>>>> *2018-03-07 21:36:15,602 ERROR
>>>> org.apache.predictionio.tools.console.Console$ [main] - None of the
>>>> configured nodes are available: []
>>>> (org.elasticsearch.client.transport.NoNodeAvailableException)*
>>>> *org.elasticsearch.client.transport.NoNodeAvailableException: None of
>>>> the configured nodes are available: []*
>>>>
>>>>
>>>> *I am using the steps from
>>>> - http://predictionio.apache.org/install/install-sourcecode/
>>>> <http://predictionio.apache.org/install/install-sourcecode/>*
>>>>
>>>> *below is pio-env.sh and Error logs*
>>>>
>>>> *[mapr@valvcshad004vm conf]$ cat pio-env.sh*
>>>> #!/usr/bin/env bash
>>>> #
>>>> # Copy this file as pio-env.sh and edit it for your site's
>>>> configuration.
>>>> #
>>>> # Licensed to the Apache Software Foundation (ASF) under one or more
>>>> # contributor license agreements. See the NOTICE file distributed with
>>>> # this work for additional information regarding copyright ownership.
>>>> # The ASF licenses this file to You under the Apache License, Version
>>>> 2.0
>>>> # (the "License"); you may not use this file except in compliance with
>>>> # the License. You may obtain a copy of the License at
>>>> #
>>>> # http://www.apache.org/licenses/LICENSE-2.0
>>>> #
>>>> # Unless required by applicable law or agreed to in writing, software
>>>> # distributed under the License is distributed on an "AS IS" BASIS,
>>>> # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>> implied.
>>>> # See the License for the specific language governing permissions and
>>>> # limitations under the License.
>>>> #
>>>>
>>>> # PredictionIO Main Configuration
>>>> #
>>>> # This section controls core behavior of PredictionIO. It is very
>>>> likely that
>>>> # you need to change these to fit your site.
>>>>
>>>> # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
>>>> #SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6
>>>> SPARK_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/spark-
>>>> 2.1.1-bin-hadoop2.6
>>>> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.2.1.jar
>>>> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.37.jar
>>>>
>>>> # ES_CONF_DIR: You must configure this if you have advanced
>>>> configuration for
>>>> # your Elasticsearch setup.
>>>> # ES_CONF_DIR=/opt/elasticsearch
>>>>
>>>> # HADOOP_CONF_DIR: You must configure this if you intend to run
>>>> PredictionIO
>>>> # with Hadoop 2.
>>>> # HADOOP_CONF_DIR=/opt/hadoop
>>>>
>>>> # HBASE_CONF_DIR: You must configure this if you intend to run
>>>> PredictionIO
>>>> # with HBase on a remote cluster.
>>>> # HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf
>>>>
>>>> # Filesystem paths where PredictionIO uses as block storage.
>>>> PIO_FS_BASEDIR=$HOME/.pio_store
>>>> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
>>>> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>>>>
>>>> # PredictionIO Storage Configuration
>>>> #
>>>> # This section controls programs that make use of PredictionIO's
>>>> built-in
>>>> # storage facilities. Default values are shown below.
>>>> #
>>>> # For more information on storage configuration please refer to
>>>> # http://predictionio.incubator.apache.org/system/anotherdatastore/
>>>>
>>>> # Storage Repositories
>>>>
>>>> # Default is to use PostgreSQL
>>>> #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>>>> #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
>>>>
>>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
>>>>
>>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
>>>>
>>>> PIO_STORAGE_REPOSITORIES_METADATA_NAME=predictionio_metadata
>>>> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>>>>
>>>> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=predictionio_eventdata
>>>> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>>>>
>>>> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_
>>>> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
>>>>
>>>> # Storage Data Sources
>>>>
>>>>
>>>> # PostgreSQL Default Settings
>>>> # Please change "pio" to your database name in
>>>> PIO_STORAGE_SOURCES_PGSQL_URL
>>>> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>>>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>>>> #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
>>>> #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
>>>> #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
>>>> #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>>>>
>>>> # MySQL Example
>>>> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
>>>> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
>>>> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
>>>> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>>>>
>>>> # Elasticsearch Example
>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>>>> #PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch
>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/mapr
>>>> -predictionio/vendors/elasticsearch-5.5.2
>>>>
>>>> # Local File System Example
>>>> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
>>>> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>>>>
>>>> # HBase Example
>>>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
>>>> PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predict
>>>> ionio/vendors/hbase-1.2.6
>>>>
>>>>
>>>>
>>>> *ERROR in the logs---*
>>>>
>>>> 2018-03-07 21:36:14,491 INFO org.apache.predictionio.data.storage.Storage$
>>>> [main] - Verifying Meta Data Backend (Source: ELASTICSEARCH)...
>>>> 2018-03-07 21:36:15,601 ERROR org.apache.predictionio.tools.console.Console$
>>>> [main] - Unable to connect to all storage backends successfully. The
>>>> following shows the error message from the storage backend.
>>>> 2018-03-07 21:36:15,602 ERROR org.apache.predictionio.tools.console.Console$
>>>> [main] - None of the configured nodes are available: []
>>>> (org.elasticsearch.client.transport.NoNodeAvailableException)
>>>> org.elasticsearch.client.transport.NoNodeAvailableException: None of
>>>> the configured nodes are available: []
>>>> at org.elasticsearch.client.transport.TransportClientNodesServi
>>>> ce.ensureNodesAreAvailable(TransportClientNodesService.java:305)
>>>> at org.elasticsearch.client.transport.TransportClientNodesServi
>>>> ce.execute(TransportClientNodesService.java:200)
>>>> at org.elasticsearch.client.transport.support.InternalTransport
>>>> IndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86)
>>>> at org.elasticsearch.client.support.AbstractIndicesAdminClient.
>>>> exists(AbstractIndicesAdminClient.java:178)
>>>> at org.elasticsearch.action.admin.indices.exists.indices.Indice
>>>> sExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:53)
>>>> at org.elasticsearch.action.ActionRequestBuilder.execute(Action
>>>> RequestBuilder.java:91)
>>>> at org.elasticsearch.action.ActionRequestBuilder.execute(Action
>>>> RequestBuilder.java:65)
>>>> at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequ
>>>> estBuilder.java:73)
>>>> at org.apache.predictionio.data.storage.elasticsearch.ESEngineI
>>>> nstances.<init>(ESEngineInstances.scala:42)
>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>>> Method)
>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native
>>>> ConstructorAccessorImpl.java:62)
>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De
>>>> legatingConstructorAccessorImpl.java:45)
>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:4
>>>> 23)
>>>> at org.apache.predictionio.data.storage.Storage$.getDataObject(
>>>> Storage.scala:306)
>>>> at org.apache.predictionio.data.storage.Storage$.getDataObjectF
>>>> romRepo(Storage.scala:266)
>>>> at org.apache.predictionio.data.storage.Storage$.getMetaDataEng
>>>> ineInstances(Storage.scala:367)
>>>> at org.apache.predictionio.data.storage.Storage$.verifyAllDataO
>>>> bjects(Storage.scala:342)
>>>> at org.apache.predictionio.tools.console.Console$.status(Consol
>>>> e.scala:1087)
>>>> at org.apache.predictionio.tools.console.Console$$anonfun$main$
>>>> 1.apply(Console.scala:737)
>>>> at org.apache.predictionio.tools.console.Console$$anonfun$main$
>>>> 1.apply(Console.scala:696)
>>>> at scala.Option.map(Option.scala:145)
>>>> at org.apache.predictionio.tools.console.Console$.main(Console.
>>>> scala:696)
>>>> at org.apache.predictionio.tools.console.Console.main(Console.s
>>>> cala)
>>>> 2018-03-07 21:36:15,605 ERROR org.apache.predictionio.tools.console.Console$
>>>> [main] - Dumping configuration of initialized storage backend sources.
>>>> Please make sure they are correct.
>>>> 2018-03-07 21:36:15,607 ERROR org.apache.predictionio.tools.console.Console$
>>>> [main] - Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration:
>>>> HOSTS -> localhost, TYPE -> elasticsearch, HOME ->
>>>> /dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2, PORTS
>>>> -> 9300
>>>> 2018-03-07 21:36:42,649 INFO org.apache.predictionio.tools.console.Console$
>>>> [main] - Creating Event Server at 0.0.0.0:7070
>>>> 2018-03-07 21:36:43,417 ERROR org.apache.predictionio.data.storage.Storage$
>>>> [main] - Error initializing storage client for source HBASE
>>>> java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseC
>>>> onfiguration
>>>> at org.apache.predictionio.data.storage.hbase.StorageClient.<in
>>>> it>(StorageClient.scala:46)
>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>>> Method)
>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native
>>>> ConstructorAccessorImpl.java:62)
>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De
>>>> legatingConstructorAccessorImpl.java:45)
>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:4
>>>> 23)
>>>> at org.apache.predictionio.data.storage.Storage$.getClient(Stor
>>>> age.scala:220)
>>>> at org.apache.predictionio.data.storage.Storage$.org$apache$pre
>>>> dictionio$data$storage$Storage$$updateS2CM(Storage.scala:251)
>>>> at org.apache.predictionio.data.storage.Storage$$anonfun$source
>>>> sToClientMeta$1.apply(Storage.scala:212)
>>>> at org.apache.predictionio.data.storage.Storage$$anonfun$source
>>>> sToClientMeta$1.apply(Storage.scala:212)
>>>>
>>>>
>>>> On Wed, Mar 7, 2018 at 1:21 AM, Pawan Agnihotri <
>>>> pawan.agnihotri@gmail.com> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I need your help to configure predictionIO on linux 7.2 -
>>>>>
>>>>> I am using
>>>>> http://predictionio.apache.org/install/install-sourcecode/ link for
>>>>> steps and installed spark, elastic search and hbase but getting below error
>>>>>
>>>>>
>>>>> [mapr@valvcshad004vm bin]$ ./pio status
>>>>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89:
>>>>> /opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such
>>>>> file or directory
>>>>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90:
>>>>> generate_compatible_classpath: command not found
>>>>> SLF4J: Class path contains multiple SLF4J bindings.
>>>>> SLF4J: Found binding in [jar:file:/dfs/pawan_scala/map
>>>>> r-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT.jar!/or
>>>>> g/slf4j/impl/StaticLoggerBinder.class]
>>>>> SLF4J: Found binding in [jar:file:/data/opt/mapr/lib/s
>>>>> lf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>>>> explanation.
>>>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>>>> [INFO] [Console$] Inspecting PredictionIO...
>>>>> [INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at
>>>>> /dfs/pawan_scala/mapr-predictionio
>>>>> [INFO] [Console$] Inspecting Apache Spark...
>>>>> [INFO] [Console$] Apache Spark is installed at
>>>>> /dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6
>>>>> [INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum
>>>>> requirement of 1.3.0)
>>>>> [INFO] [Console$] Inspecting storage backend connections...
>>>>> *[WARN] [Storage$] There is no properly configured repository.*
>>>>> *[ERROR] [Storage$] Required repository (METADATA) configuration is
>>>>> missing.*
>>>>> *[ERROR] [Storage$] There were 1 configuration errors. Exiting.*
>>>>> [mapr@valvcshad004vm bin]$
>>>>>
>>>>> Here is my *pio-env.sh *file
>>>>>
>>>>> [mapr@valvcshad004vm conf]$ cat pio-env.sh
>>>>> # Default is to use PostgreSQL
>>>>> #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>>>>> #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
>>>>>
>>>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>>>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
>>>>>
>>>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>>>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
>>>>>
>>>>> # Storage Data Sources
>>>>>
>>>>> # PostgreSQL Default Settings
>>>>> # Please change "pio" to your database name in
>>>>> PIO_STORAGE_SOURCES_PGSQL_URL
>>>>> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>>>>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>>>>> #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
>>>>> #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
>>>>> #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
>>>>> #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>>>>>
>>>>> # MySQL Example
>>>>> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
>>>>> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
>>>>> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
>>>>> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>>>>>
>>>>> # Elasticsearch Example
>>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch
>>>>> _cluster_name
>>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/map
>>>>> r-predictionio/vendors/elasticsearch-5.5.2
>>>>>
>>>>> # Local File System Example
>>>>> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
>>>>> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>>>>>
>>>>> # HBase Example
>>>>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
>>>>> PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predic
>>>>> tionio/vendors/hbase-1.2.6
>>>>>
>>>>> [mapr@valvcshad004vm conf]$
>>>>>
>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Pawan Agnihotri
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Pawan Agnihotri
>>>>
>>>
>>>
>>>
>>> --
>>> *Mars Hall
>>> 415-818-7039 <(415)%20818-7039>
>>> Customer Facing Architect
>>> Salesforce Platform / Heroku
>>> San Francisco, California
>>>
>>
>>
>>
>> --
>> Thanks,
>> Pawan Agnihotri
>>
>
>
>
> --
> *Mars Hall
> 415-818-7039 <(415)%20818-7039>
> Customer Facing Architect
> Salesforce Platform / Heroku
> San Francisco, California
>
--
Thanks,
Pawan Agnihotri
Re: [ERROR] [Storage$] Required repository (METADATA) configuration
is missing.
Posted by Mars Hall <ma...@salesforce.com>.
It looks like you're using an old version of PIO:
PredictionIO 0.10.0-SNAPSHOT
>
Elasticsearch 5 does not work with that old version.
Please upgrade to PIO 0.12.0. Download/install the newer distribution, and
then update the engine's `build.sbt` & `template.json`.
On Fri, Mar 9, 2018 at 12:14 AM, Pawan Agnihotri <pa...@gmail.com>
wrote:
> Thanks Mars for the response. I tried adding the scheme and port to 9200
> for elasticsearch in conf/pio-env.sh but still getting the same error which
> checking the status.
>
> here is my conf for elasticsearch and other logs. Please let me know if
> you need more details. thank you.
>
> *PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch*
> *#PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch*
> *PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost*
> *PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http*
> *PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200*
>
> *PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2*
>
>
> [mapr@valvcshad004vm bin]$ ./pio status
> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89:
> /opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such file
> or directory
> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90:
> generate_compatible_classpath: command not found
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/dfs/pawan_scala/
> mapr-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT.
> jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/data/opt/mapr/lib/
> slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> [INFO] [Console$] Inspecting PredictionIO...
> [INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at
> /dfs/pawan_scala/mapr-predictionio
> [INFO] [Console$] Inspecting Apache Spark...
> [INFO] [Console$] Apache Spark is installed at /dfs/pawan_scala/mapr-
> predictionio/vendors/spark-2.1.1-bin-hadoop2.6
> [INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum requirement
> of 1.3.0)
> [INFO] [Console$] Inspecting storage backend connections...
> [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
> [ERROR] [Console$] Unable to connect to all storage backends successfully.
> The following shows the error message from the storage backend.
> [ERROR] [Console$] None of the configured nodes are available: []
> (org.elasticsearch.client.transport.NoNodeAvailableException)
> [ERROR] [Console$] Dumping configuration of initialized storage backend
> sources. Please make sure they are correct.
> [ERROR] [Console$] Source Name: ELASTICSEARCH; Type: elasticsearch;
> Configuration: HOME -> /dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2,
> HOSTS -> localhost, PORTS -> 9200, SCHEMES -> http, TYPE -> elasticsearch
> [mapr@valvcshad004vm bin]$
>
> ------------
> [mapr@valvcshad004vm bin]$ cat pio.log
>
> 2018-03-08 19:08:54,371 INFO org.apache.predictionio.tools.console.Console$
> [main] - Inspecting PredictionIO...
> 2018-03-08 19:08:54,375 INFO org.apache.predictionio.tools.console.Console$
> [main] - PredictionIO 0.10.0-SNAPSHOT is installed at /dfs/pawan_scala/mapr-
> predictionio
> 2018-03-08 19:08:54,376 INFO org.apache.predictionio.tools.console.Console$
> [main] - Inspecting Apache Spark...
> 2018-03-08 19:08:54,389 INFO org.apache.predictionio.tools.console.Console$
> [main] - Apache Spark is installed at /dfs/pawan_scala/mapr-
> predictionio/vendors/spark-2.1.1-bin-hadoop2.6
> 2018-03-08 19:08:54,427 INFO org.apache.predictionio.tools.console.Console$
> [main] - Apache Spark 2.1.1 detected (meets minimum requirement of 1.3.0)
> 2018-03-08 19:08:54,428 INFO org.apache.predictionio.tools.console.Console$
> [main] - Inspecting storage backend connections...
> 2018-03-08 19:08:54,450 INFO org.apache.predictionio.data.storage.Storage$
> [main] - Verifying Meta Data Backend (Source: ELASTICSEARCH)...
> 2018-03-08 19:08:55,636 ERROR org.apache.predictionio.tools.console.Console$
> [main] - Unable to connect to all storage backends successfully. The
> following shows the error message from the storage backend.
> 2018-03-08 19:08:55,638 ERROR org.apache.predictionio.tools.console.Console$
> [main] - None of the configured nodes are available: []
> (org.elasticsearch.client.transport.NoNodeAvailableException)
> org.elasticsearch.client.transport.NoNodeAvailableException: None of the
> configured nodes are available: []
> at org.elasticsearch.client.transport.TransportClientNodesService.
> ensureNodesAreAvailable(TransportClientNodesService.java:305)
> at org.elasticsearch.client.transport.TransportClientNodesService.
> execute(TransportClientNodesService.java:200)
> at org.elasticsearch.client.transport.support.
> InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminC
> lient.java:86)
> at org.elasticsearch.client.support.AbstractIndicesAdminClient.
> exists(AbstractIndicesAdminClient.java:178)
> at org.elasticsearch.action.admin.indices.exists.indices.
> IndicesExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:53)
> at org.elasticsearch.action.ActionRequestBuilder.execute(
> ActionRequestBuilder.java:91)
> at org.elasticsearch.action.ActionRequestBuilder.execute(
> ActionRequestBuilder.java:65)
> at org.elasticsearch.action.ActionRequestBuilder.get(
> ActionRequestBuilder.java:73)
> at org.apache.predictionio.data.storage.elasticsearch.
> ESEngineInstances.<init>(ESEngineInstances.scala:42)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(
> NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.predictionio.data.storage.Storage$.
> getDataObject(Storage.scala:306)
> at org.apache.predictionio.data.storage.Storage$.
> getDataObjectFromRepo(Storage.scala:266)
> at org.apache.predictionio.data.storage.Storage$.
> getMetaDataEngineInstances(Storage.scala:367)
> at org.apache.predictionio.data.storage.Storage$.
> verifyAllDataObjects(Storage.scala:342)
> at org.apache.predictionio.tools.console.Console$.status(
> Console.scala:1087)
> at org.apache.predictionio.tools.console.Console$$anonfun$main$
> 1.apply(Console.scala:737)
> at org.apache.predictionio.tools.console.Console$$anonfun$main$
> 1.apply(Console.scala:696)
> at scala.Option.map(Option.scala:145)
> at org.apache.predictionio.tools.console.Console$.main(Console.
> scala:696)
> at org.apache.predictionio.tools.console.Console.main(Console.
> scala)
> 2018-03-08 19:08:55,641 ERROR org.apache.predictionio.tools.console.Console$
> [main] - Dumping configuration of initialized storage backend sources.
> Please make sure they are correct.
> 2018-03-08 19:08:55,644 ERROR org.apache.predictionio.tools.console.Console$
> [main] - Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration:
> HOME -> /dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2,
> HOSTS -> localhost, PORTS -> 9200, SCHEMES -> http, TYPE -> elasticsearch
> [mapr@valvcshad004vm bin]$
>
>
> On Thu, Mar 8, 2018 at 1:30 PM, Mars Hall <ma...@salesforce.com>
> wrote:
>
>> We added support for Elasticsearch 5 last year, so current PredictionIO
>> uses HTTP/REST protocol on port 9200, not the native protocol on port 9300.
>>
>> Here's the local dev config I have working with PIO 0.12.0 and
>> Elasticsearch 5:
>>
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/ela
>> sticsearch
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
>>
>> I believe Elasticsearch 1 is only supported via native protocol on port
>> 9300, which is eventually being removed in a future PIO release.
>>
>> On Thu, Mar 8, 2018 at 2:50 AM, Pawan Agnihotri <
>> pawan.agnihotri@gmail.com> wrote:
>>
>>> Hello Donald and Team,
>>>
>>> I am working POC and I would like to use predictionIO.I know its
>>> configuration issue with elasticsearch but I am kind of stuck with below
>>> error so reaching out for help.
>>>
>>> I am in need of some quick hand here as the time is running out.
>>> anything you feel I can try out or steps would be helpful please.
>>>
>>>
>>> *2018-03-07 21:36:15,602 ERROR
>>> org.apache.predictionio.tools.console.Console$ [main] - None of the
>>> configured nodes are available: []
>>> (org.elasticsearch.client.transport.NoNodeAvailableException)*
>>> *org.elasticsearch.client.transport.NoNodeAvailableException: None of
>>> the configured nodes are available: []*
>>>
>>>
>>> *I am using the steps from
>>> - http://predictionio.apache.org/install/install-sourcecode/
>>> <http://predictionio.apache.org/install/install-sourcecode/>*
>>>
>>> *below is pio-env.sh and Error logs*
>>>
>>> *[mapr@valvcshad004vm conf]$ cat pio-env.sh*
>>> #!/usr/bin/env bash
>>> #
>>> # Copy this file as pio-env.sh and edit it for your site's configuration.
>>> #
>>> # Licensed to the Apache Software Foundation (ASF) under one or more
>>> # contributor license agreements. See the NOTICE file distributed with
>>> # this work for additional information regarding copyright ownership.
>>> # The ASF licenses this file to You under the Apache License, Version 2.0
>>> # (the "License"); you may not use this file except in compliance with
>>> # the License. You may obtain a copy of the License at
>>> #
>>> # http://www.apache.org/licenses/LICENSE-2.0
>>> #
>>> # Unless required by applicable law or agreed to in writing, software
>>> # distributed under the License is distributed on an "AS IS" BASIS,
>>> # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>> implied.
>>> # See the License for the specific language governing permissions and
>>> # limitations under the License.
>>> #
>>>
>>> # PredictionIO Main Configuration
>>> #
>>> # This section controls core behavior of PredictionIO. It is very likely
>>> that
>>> # you need to change these to fit your site.
>>>
>>> # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
>>> #SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6
>>> SPARK_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/spark-
>>> 2.1.1-bin-hadoop2.6
>>> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.2.1.jar
>>> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.37.jar
>>>
>>> # ES_CONF_DIR: You must configure this if you have advanced
>>> configuration for
>>> # your Elasticsearch setup.
>>> # ES_CONF_DIR=/opt/elasticsearch
>>>
>>> # HADOOP_CONF_DIR: You must configure this if you intend to run
>>> PredictionIO
>>> # with Hadoop 2.
>>> # HADOOP_CONF_DIR=/opt/hadoop
>>>
>>> # HBASE_CONF_DIR: You must configure this if you intend to run
>>> PredictionIO
>>> # with HBase on a remote cluster.
>>> # HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf
>>>
>>> # Filesystem paths where PredictionIO uses as block storage.
>>> PIO_FS_BASEDIR=$HOME/.pio_store
>>> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
>>> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>>>
>>> # PredictionIO Storage Configuration
>>> #
>>> # This section controls programs that make use of PredictionIO's built-in
>>> # storage facilities. Default values are shown below.
>>> #
>>> # For more information on storage configuration please refer to
>>> # http://predictionio.incubator.apache.org/system/anotherdatastore/
>>>
>>> # Storage Repositories
>>>
>>> # Default is to use PostgreSQL
>>> #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>>> #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
>>>
>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
>>>
>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
>>>
>>> PIO_STORAGE_REPOSITORIES_METADATA_NAME=predictionio_metadata
>>> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>>>
>>> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=predictionio_eventdata
>>> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>>>
>>> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_
>>> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
>>>
>>> # Storage Data Sources
>>>
>>>
>>> # PostgreSQL Default Settings
>>> # Please change "pio" to your database name in
>>> PIO_STORAGE_SOURCES_PGSQL_URL
>>> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>>> #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
>>> #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
>>> #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
>>> #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>>>
>>> # MySQL Example
>>> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
>>> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
>>> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
>>> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>>>
>>> # Elasticsearch Example
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>>> #PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/mapr
>>> -predictionio/vendors/elasticsearch-5.5.2
>>>
>>> # Local File System Example
>>> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
>>> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>>>
>>> # HBase Example
>>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
>>> PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predict
>>> ionio/vendors/hbase-1.2.6
>>>
>>>
>>>
>>> *ERROR in the logs---*
>>>
>>> 2018-03-07 21:36:14,491 INFO org.apache.predictionio.data.storage.Storage$
>>> [main] - Verifying Meta Data Backend (Source: ELASTICSEARCH)...
>>> 2018-03-07 21:36:15,601 ERROR org.apache.predictionio.tools.console.Console$
>>> [main] - Unable to connect to all storage backends successfully. The
>>> following shows the error message from the storage backend.
>>> 2018-03-07 21:36:15,602 ERROR org.apache.predictionio.tools.console.Console$
>>> [main] - None of the configured nodes are available: []
>>> (org.elasticsearch.client.transport.NoNodeAvailableException)
>>> org.elasticsearch.client.transport.NoNodeAvailableException: None of
>>> the configured nodes are available: []
>>> at org.elasticsearch.client.transport.TransportClientNodesServi
>>> ce.ensureNodesAreAvailable(TransportClientNodesService.java:305)
>>> at org.elasticsearch.client.transport.TransportClientNodesServi
>>> ce.execute(TransportClientNodesService.java:200)
>>> at org.elasticsearch.client.transport.support.InternalTransport
>>> IndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86)
>>> at org.elasticsearch.client.support.AbstractIndicesAdminClient.
>>> exists(AbstractIndicesAdminClient.java:178)
>>> at org.elasticsearch.action.admin.indices.exists.indices.Indice
>>> sExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:53)
>>> at org.elasticsearch.action.ActionRequestBuilder.execute(Action
>>> RequestBuilder.java:91)
>>> at org.elasticsearch.action.ActionRequestBuilder.execute(Action
>>> RequestBuilder.java:65)
>>> at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequ
>>> estBuilder.java:73)
>>> at org.apache.predictionio.data.storage.elasticsearch.ESEngineI
>>> nstances.<init>(ESEngineInstances.scala:42)
>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>> Method)
>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native
>>> ConstructorAccessorImpl.java:62)
>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De
>>> legatingConstructorAccessorImpl.java:45)
>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:4
>>> 23)
>>> at org.apache.predictionio.data.storage.Storage$.getDataObject(
>>> Storage.scala:306)
>>> at org.apache.predictionio.data.storage.Storage$.getDataObjectF
>>> romRepo(Storage.scala:266)
>>> at org.apache.predictionio.data.storage.Storage$.getMetaDataEng
>>> ineInstances(Storage.scala:367)
>>> at org.apache.predictionio.data.storage.Storage$.verifyAllDataO
>>> bjects(Storage.scala:342)
>>> at org.apache.predictionio.tools.console.Console$.status(Consol
>>> e.scala:1087)
>>> at org.apache.predictionio.tools.console.Console$$anonfun$main$
>>> 1.apply(Console.scala:737)
>>> at org.apache.predictionio.tools.console.Console$$anonfun$main$
>>> 1.apply(Console.scala:696)
>>> at scala.Option.map(Option.scala:145)
>>> at org.apache.predictionio.tools.console.Console$.main(Console.
>>> scala:696)
>>> at org.apache.predictionio.tools.console.Console.main(Console.s
>>> cala)
>>> 2018-03-07 21:36:15,605 ERROR org.apache.predictionio.tools.console.Console$
>>> [main] - Dumping configuration of initialized storage backend sources.
>>> Please make sure they are correct.
>>> 2018-03-07 21:36:15,607 ERROR org.apache.predictionio.tools.console.Console$
>>> [main] - Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration:
>>> HOSTS -> localhost, TYPE -> elasticsearch, HOME ->
>>> /dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2, PORTS
>>> -> 9300
>>> 2018-03-07 21:36:42,649 INFO org.apache.predictionio.tools.console.Console$
>>> [main] - Creating Event Server at 0.0.0.0:7070
>>> 2018-03-07 21:36:43,417 ERROR org.apache.predictionio.data.storage.Storage$
>>> [main] - Error initializing storage client for source HBASE
>>> java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseC
>>> onfiguration
>>> at org.apache.predictionio.data.storage.hbase.StorageClient.<in
>>> it>(StorageClient.scala:46)
>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>>> Method)
>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native
>>> ConstructorAccessorImpl.java:62)
>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De
>>> legatingConstructorAccessorImpl.java:45)
>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:4
>>> 23)
>>> at org.apache.predictionio.data.storage.Storage$.getClient(Stor
>>> age.scala:220)
>>> at org.apache.predictionio.data.storage.Storage$.org$apache$pre
>>> dictionio$data$storage$Storage$$updateS2CM(Storage.scala:251)
>>> at org.apache.predictionio.data.storage.Storage$$anonfun$source
>>> sToClientMeta$1.apply(Storage.scala:212)
>>> at org.apache.predictionio.data.storage.Storage$$anonfun$source
>>> sToClientMeta$1.apply(Storage.scala:212)
>>>
>>>
>>> On Wed, Mar 7, 2018 at 1:21 AM, Pawan Agnihotri <
>>> pawan.agnihotri@gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I need your help to configure predictionIO on linux 7.2 -
>>>>
>>>> I am using
>>>> http://predictionio.apache.org/install/install-sourcecode/ link for
>>>> steps and installed spark, elastic search and hbase but getting below error
>>>>
>>>>
>>>> [mapr@valvcshad004vm bin]$ ./pio status
>>>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89:
>>>> /opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such
>>>> file or directory
>>>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90:
>>>> generate_compatible_classpath: command not found
>>>> SLF4J: Class path contains multiple SLF4J bindings.
>>>> SLF4J: Found binding in [jar:file:/dfs/pawan_scala/map
>>>> r-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT.jar!/or
>>>> g/slf4j/impl/StaticLoggerBinder.class]
>>>> SLF4J: Found binding in [jar:file:/data/opt/mapr/lib/s
>>>> lf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>>> explanation.
>>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>>> [INFO] [Console$] Inspecting PredictionIO...
>>>> [INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at
>>>> /dfs/pawan_scala/mapr-predictionio
>>>> [INFO] [Console$] Inspecting Apache Spark...
>>>> [INFO] [Console$] Apache Spark is installed at
>>>> /dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6
>>>> [INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum
>>>> requirement of 1.3.0)
>>>> [INFO] [Console$] Inspecting storage backend connections...
>>>> *[WARN] [Storage$] There is no properly configured repository.*
>>>> *[ERROR] [Storage$] Required repository (METADATA) configuration is
>>>> missing.*
>>>> *[ERROR] [Storage$] There were 1 configuration errors. Exiting.*
>>>> [mapr@valvcshad004vm bin]$
>>>>
>>>> Here is my *pio-env.sh *file
>>>>
>>>> [mapr@valvcshad004vm conf]$ cat pio-env.sh
>>>> # Default is to use PostgreSQL
>>>> #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>>>> #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
>>>>
>>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
>>>>
>>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
>>>>
>>>> # Storage Data Sources
>>>>
>>>> # PostgreSQL Default Settings
>>>> # Please change "pio" to your database name in
>>>> PIO_STORAGE_SOURCES_PGSQL_URL
>>>> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>>>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>>>> #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
>>>> #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
>>>> #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
>>>> #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>>>>
>>>> # MySQL Example
>>>> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
>>>> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
>>>> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
>>>> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>>>>
>>>> # Elasticsearch Example
>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch
>>>> _cluster_name
>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/map
>>>> r-predictionio/vendors/elasticsearch-5.5.2
>>>>
>>>> # Local File System Example
>>>> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
>>>> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>>>>
>>>> # HBase Example
>>>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
>>>> PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predic
>>>> tionio/vendors/hbase-1.2.6
>>>>
>>>> [mapr@valvcshad004vm conf]$
>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Pawan Agnihotri
>>>>
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Pawan Agnihotri
>>>
>>
>>
>>
>> --
>> *Mars Hall
>> 415-818-7039 <(415)%20818-7039>
>> Customer Facing Architect
>> Salesforce Platform / Heroku
>> San Francisco, California
>>
>
>
>
> --
> Thanks,
> Pawan Agnihotri
>
--
*Mars Hall
415-818-7039
Customer Facing Architect
Salesforce Platform / Heroku
San Francisco, California
Re: [ERROR] [Storage$] Required repository (METADATA) configuration
is missing.
Posted by Pawan Agnihotri <pa...@gmail.com>.
Thanks Mars for the response. I tried adding the scheme and port to 9200
for elasticsearch in conf/pio-env.sh but still getting the same error which
checking the status.
here is my conf for elasticsearch and other logs. Please let me know if you
need more details. thank you.
*PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch*
*#PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch*
*PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost*
*PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http*
*PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200*
*PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2*
[mapr@valvcshad004vm bin]$ ./pio status
/dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89:
/opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such file
or directory
/dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90:
generate_compatible_classpath: command not found
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/dfs/pawan_scala/mapr-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/data/opt/mapr/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[INFO] [Console$] Inspecting PredictionIO...
[INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at
/dfs/pawan_scala/mapr-predictionio
[INFO] [Console$] Inspecting Apache Spark...
[INFO] [Console$] Apache Spark is installed at
/dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6
[INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum requirement of
1.3.0)
[INFO] [Console$] Inspecting storage backend connections...
[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
[ERROR] [Console$] Unable to connect to all storage backends successfully.
The following shows the error message from the storage backend.
[ERROR] [Console$] None of the configured nodes are available: []
(org.elasticsearch.client.transport.NoNodeAvailableException)
[ERROR] [Console$] Dumping configuration of initialized storage backend
sources. Please make sure they are correct.
[ERROR] [Console$] Source Name: ELASTICSEARCH; Type: elasticsearch;
Configuration: HOME ->
/dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2, HOSTS ->
localhost, PORTS -> 9200, SCHEMES -> http, TYPE -> elasticsearch
[mapr@valvcshad004vm bin]$
------------
[mapr@valvcshad004vm bin]$ cat pio.log
2018-03-08 19:08:54,371 INFO
org.apache.predictionio.tools.console.Console$ [main] - Inspecting
PredictionIO...
2018-03-08 19:08:54,375 INFO
org.apache.predictionio.tools.console.Console$ [main] - PredictionIO
0.10.0-SNAPSHOT is installed at /dfs/pawan_scala/mapr-predictionio
2018-03-08 19:08:54,376 INFO
org.apache.predictionio.tools.console.Console$ [main] - Inspecting Apache
Spark...
2018-03-08 19:08:54,389 INFO
org.apache.predictionio.tools.console.Console$ [main] - Apache Spark is
installed at
/dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6
2018-03-08 19:08:54,427 INFO
org.apache.predictionio.tools.console.Console$ [main] - Apache Spark 2.1.1
detected (meets minimum requirement of 1.3.0)
2018-03-08 19:08:54,428 INFO
org.apache.predictionio.tools.console.Console$ [main] - Inspecting storage
backend connections...
2018-03-08 19:08:54,450 INFO org.apache.predictionio.data.storage.Storage$
[main] - Verifying Meta Data Backend (Source: ELASTICSEARCH)...
2018-03-08 19:08:55,636 ERROR
org.apache.predictionio.tools.console.Console$ [main] - Unable to connect
to all storage backends successfully. The following shows the error message
from the storage backend.
2018-03-08 19:08:55,638 ERROR
org.apache.predictionio.tools.console.Console$ [main] - None of the
configured nodes are available: []
(org.elasticsearch.client.transport.NoNodeAvailableException)
org.elasticsearch.client.transport.NoNodeAvailableException: None of the
configured nodes are available: []
at
org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:305)
at
org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:200)
at
org.elasticsearch.client.transport.support.InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86)
at
org.elasticsearch.client.support.AbstractIndicesAdminClient.exists(AbstractIndicesAdminClient.java:178)
at
org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:53)
at
org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)
at
org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65)
at
org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:73)
at
org.apache.predictionio.data.storage.elasticsearch.ESEngineInstances.<init>(ESEngineInstances.scala:42)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:306)
at
org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:266)
at
org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:367)
at
org.apache.predictionio.data.storage.Storage$.verifyAllDataObjects(Storage.scala:342)
at
org.apache.predictionio.tools.console.Console$.status(Console.scala:1087)
at
org.apache.predictionio.tools.console.Console$$anonfun$main$1.apply(Console.scala:737)
at
org.apache.predictionio.tools.console.Console$$anonfun$main$1.apply(Console.scala:696)
at scala.Option.map(Option.scala:145)
at
org.apache.predictionio.tools.console.Console$.main(Console.scala:696)
at org.apache.predictionio.tools.console.Console.main(Console.scala)
2018-03-08 19:08:55,641 ERROR
org.apache.predictionio.tools.console.Console$ [main] - Dumping
configuration of initialized storage backend sources. Please make sure they
are correct.
2018-03-08 19:08:55,644 ERROR
org.apache.predictionio.tools.console.Console$ [main] - Source Name:
ELASTICSEARCH; Type: elasticsearch; Configuration: HOME ->
/dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2, HOSTS ->
localhost, PORTS -> 9200, SCHEMES -> http, TYPE -> elasticsearch
[mapr@valvcshad004vm bin]$
On Thu, Mar 8, 2018 at 1:30 PM, Mars Hall <ma...@salesforce.com> wrote:
> We added support for Elasticsearch 5 last year, so current PredictionIO
> uses HTTP/REST protocol on port 9200, not the native protocol on port 9300.
>
> Here's the local dev config I have working with PIO 0.12.0 and
> Elasticsearch 5:
>
> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch
> PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
>
> I believe Elasticsearch 1 is only supported via native protocol on port
> 9300, which is eventually being removed in a future PIO release.
>
> On Thu, Mar 8, 2018 at 2:50 AM, Pawan Agnihotri <pawan.agnihotri@gmail.com
> > wrote:
>
>> Hello Donald and Team,
>>
>> I am working POC and I would like to use predictionIO.I know its
>> configuration issue with elasticsearch but I am kind of stuck with below
>> error so reaching out for help.
>>
>> I am in need of some quick hand here as the time is running out. anything
>> you feel I can try out or steps would be helpful please.
>>
>>
>> *2018-03-07 21:36:15,602 ERROR
>> org.apache.predictionio.tools.console.Console$ [main] - None of the
>> configured nodes are available: []
>> (org.elasticsearch.client.transport.NoNodeAvailableException)*
>> *org.elasticsearch.client.transport.NoNodeAvailableException: None of the
>> configured nodes are available: []*
>>
>>
>> *I am using the steps from
>> - http://predictionio.apache.org/install/install-sourcecode/
>> <http://predictionio.apache.org/install/install-sourcecode/>*
>>
>> *below is pio-env.sh and Error logs*
>>
>> *[mapr@valvcshad004vm conf]$ cat pio-env.sh*
>> #!/usr/bin/env bash
>> #
>> # Copy this file as pio-env.sh and edit it for your site's configuration.
>> #
>> # Licensed to the Apache Software Foundation (ASF) under one or more
>> # contributor license agreements. See the NOTICE file distributed with
>> # this work for additional information regarding copyright ownership.
>> # The ASF licenses this file to You under the Apache License, Version 2.0
>> # (the "License"); you may not use this file except in compliance with
>> # the License. You may obtain a copy of the License at
>> #
>> # http://www.apache.org/licenses/LICENSE-2.0
>> #
>> # Unless required by applicable law or agreed to in writing, software
>> # distributed under the License is distributed on an "AS IS" BASIS,
>> # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> # See the License for the specific language governing permissions and
>> # limitations under the License.
>> #
>>
>> # PredictionIO Main Configuration
>> #
>> # This section controls core behavior of PredictionIO. It is very likely
>> that
>> # you need to change these to fit your site.
>>
>> # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
>> #SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6
>> SPARK_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/spark-
>> 2.1.1-bin-hadoop2.6
>> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.2.1.jar
>> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.37.jar
>>
>> # ES_CONF_DIR: You must configure this if you have advanced configuration
>> for
>> # your Elasticsearch setup.
>> # ES_CONF_DIR=/opt/elasticsearch
>>
>> # HADOOP_CONF_DIR: You must configure this if you intend to run
>> PredictionIO
>> # with Hadoop 2.
>> # HADOOP_CONF_DIR=/opt/hadoop
>>
>> # HBASE_CONF_DIR: You must configure this if you intend to run
>> PredictionIO
>> # with HBase on a remote cluster.
>> # HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf
>>
>> # Filesystem paths where PredictionIO uses as block storage.
>> PIO_FS_BASEDIR=$HOME/.pio_store
>> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
>> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>>
>> # PredictionIO Storage Configuration
>> #
>> # This section controls programs that make use of PredictionIO's built-in
>> # storage facilities. Default values are shown below.
>> #
>> # For more information on storage configuration please refer to
>> # http://predictionio.incubator.apache.org/system/anotherdatastore/
>>
>> # Storage Repositories
>>
>> # Default is to use PostgreSQL
>> #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>> #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
>>
>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
>>
>> #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>> #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
>>
>> PIO_STORAGE_REPOSITORIES_METADATA_NAME=predictionio_metadata
>> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>>
>> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=predictionio_eventdata
>> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>>
>> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_
>> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
>>
>> # Storage Data Sources
>>
>>
>> # PostgreSQL Default Settings
>> # Please change "pio" to your database name in
>> PIO_STORAGE_SOURCES_PGSQL_URL
>> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>> #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
>> #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
>> #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
>> #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>>
>> # MySQL Example
>> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
>> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
>> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
>> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>>
>> # Elasticsearch Example
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>> #PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/
>> mapr-predictionio/vendors/elasticsearch-5.5.2
>>
>> # Local File System Example
>> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
>> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>>
>> # HBase Example
>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
>> PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predict
>> ionio/vendors/hbase-1.2.6
>>
>>
>>
>> *ERROR in the logs---*
>>
>> 2018-03-07 21:36:14,491 INFO org.apache.predictionio.data.storage.Storage$
>> [main] - Verifying Meta Data Backend (Source: ELASTICSEARCH)...
>> 2018-03-07 21:36:15,601 ERROR org.apache.predictionio.tools.console.Console$
>> [main] - Unable to connect to all storage backends successfully. The
>> following shows the error message from the storage backend.
>> 2018-03-07 21:36:15,602 ERROR org.apache.predictionio.tools.console.Console$
>> [main] - None of the configured nodes are available: []
>> (org.elasticsearch.client.transport.NoNodeAvailableException)
>> org.elasticsearch.client.transport.NoNodeAvailableException: None of the
>> configured nodes are available: []
>> at org.elasticsearch.client.transport.TransportClientNodesServi
>> ce.ensureNodesAreAvailable(TransportClientNodesService.java:305)
>> at org.elasticsearch.client.transport.TransportClientNodesServi
>> ce.execute(TransportClientNodesService.java:200)
>> at org.elasticsearch.client.transport.support.InternalTransport
>> IndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86)
>> at org.elasticsearch.client.support.AbstractIndicesAdminClient.
>> exists(AbstractIndicesAdminClient.java:178)
>> at org.elasticsearch.action.admin.indices.exists.indices.Indice
>> sExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:53)
>> at org.elasticsearch.action.ActionRequestBuilder.execute(Action
>> RequestBuilder.java:91)
>> at org.elasticsearch.action.ActionRequestBuilder.execute(Action
>> RequestBuilder.java:65)
>> at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequ
>> estBuilder.java:73)
>> at org.apache.predictionio.data.storage.elasticsearch.ESEngineI
>> nstances.<init>(ESEngineInstances.scala:42)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native
>> ConstructorAccessorImpl.java:62)
>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De
>> legatingConstructorAccessorImpl.java:45)
>> at java.lang.reflect.Constructor.newInstance(Constructor.java:4
>> 23)
>> at org.apache.predictionio.data.storage.Storage$.getDataObject(
>> Storage.scala:306)
>> at org.apache.predictionio.data.storage.Storage$.getDataObjectF
>> romRepo(Storage.scala:266)
>> at org.apache.predictionio.data.storage.Storage$.getMetaDataEng
>> ineInstances(Storage.scala:367)
>> at org.apache.predictionio.data.storage.Storage$.verifyAllDataO
>> bjects(Storage.scala:342)
>> at org.apache.predictionio.tools.console.Console$.status(Consol
>> e.scala:1087)
>> at org.apache.predictionio.tools.console.Console$$anonfun$main$
>> 1.apply(Console.scala:737)
>> at org.apache.predictionio.tools.console.Console$$anonfun$main$
>> 1.apply(Console.scala:696)
>> at scala.Option.map(Option.scala:145)
>> at org.apache.predictionio.tools.console.Console$.main(Console.
>> scala:696)
>> at org.apache.predictionio.tools.console.Console.main(Console.s
>> cala)
>> 2018-03-07 21:36:15,605 ERROR org.apache.predictionio.tools.console.Console$
>> [main] - Dumping configuration of initialized storage backend sources.
>> Please make sure they are correct.
>> 2018-03-07 21:36:15,607 ERROR org.apache.predictionio.tools.console.Console$
>> [main] - Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration:
>> HOSTS -> localhost, TYPE -> elasticsearch, HOME ->
>> /dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2, PORTS ->
>> 9300
>> 2018-03-07 21:36:42,649 INFO org.apache.predictionio.tools.console.Console$
>> [main] - Creating Event Server at 0.0.0.0:7070
>> 2018-03-07 21:36:43,417 ERROR org.apache.predictionio.data.storage.Storage$
>> [main] - Error initializing storage client for source HBASE
>> java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseC
>> onfiguration
>> at org.apache.predictionio.data.storage.hbase.StorageClient.<in
>> it>(StorageClient.scala:46)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native
>> ConstructorAccessorImpl.java:62)
>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De
>> legatingConstructorAccessorImpl.java:45)
>> at java.lang.reflect.Constructor.newInstance(Constructor.java:4
>> 23)
>> at org.apache.predictionio.data.storage.Storage$.getClient(Stor
>> age.scala:220)
>> at org.apache.predictionio.data.storage.Storage$.org$apache$pre
>> dictionio$data$storage$Storage$$updateS2CM(Storage.scala:251)
>> at org.apache.predictionio.data.storage.Storage$$anonfun$source
>> sToClientMeta$1.apply(Storage.scala:212)
>> at org.apache.predictionio.data.storage.Storage$$anonfun$source
>> sToClientMeta$1.apply(Storage.scala:212)
>>
>>
>> On Wed, Mar 7, 2018 at 1:21 AM, Pawan Agnihotri <
>> pawan.agnihotri@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I need your help to configure predictionIO on linux 7.2 -
>>>
>>> I am using
>>> http://predictionio.apache.org/install/install-sourcecode/ link for
>>> steps and installed spark, elastic search and hbase but getting below error
>>>
>>>
>>> [mapr@valvcshad004vm bin]$ ./pio status
>>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89:
>>> /opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such
>>> file or directory
>>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90:
>>> generate_compatible_classpath: command not found
>>> SLF4J: Class path contains multiple SLF4J bindings.
>>> SLF4J: Found binding in [jar:file:/dfs/pawan_scala/map
>>> r-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT.jar!/or
>>> g/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in [jar:file:/data/opt/mapr/lib/s
>>> lf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>> explanation.
>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>> [INFO] [Console$] Inspecting PredictionIO...
>>> [INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at
>>> /dfs/pawan_scala/mapr-predictionio
>>> [INFO] [Console$] Inspecting Apache Spark...
>>> [INFO] [Console$] Apache Spark is installed at
>>> /dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6
>>> [INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum requirement
>>> of 1.3.0)
>>> [INFO] [Console$] Inspecting storage backend connections...
>>> *[WARN] [Storage$] There is no properly configured repository.*
>>> *[ERROR] [Storage$] Required repository (METADATA) configuration is
>>> missing.*
>>> *[ERROR] [Storage$] There were 1 configuration errors. Exiting.*
>>> [mapr@valvcshad004vm bin]$
>>>
>>> Here is my *pio-env.sh *file
>>>
>>> [mapr@valvcshad004vm conf]$ cat pio-env.sh
>>> # Default is to use PostgreSQL
>>> #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>>> #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
>>>
>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
>>>
>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>>> #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
>>>
>>> # Storage Data Sources
>>>
>>> # PostgreSQL Default Settings
>>> # Please change "pio" to your database name in
>>> PIO_STORAGE_SOURCES_PGSQL_URL
>>> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>>> #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
>>> #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
>>> #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
>>> #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>>>
>>> # MySQL Example
>>> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
>>> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
>>> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
>>> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>>>
>>> # Elasticsearch Example
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch
>>> _cluster_name
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/map
>>> r-predictionio/vendors/elasticsearch-5.5.2
>>>
>>> # Local File System Example
>>> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
>>> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>>>
>>> # HBase Example
>>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
>>> PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predic
>>> tionio/vendors/hbase-1.2.6
>>>
>>> [mapr@valvcshad004vm conf]$
>>>
>>>
>>> --
>>> Thanks,
>>> Pawan Agnihotri
>>>
>>
>>
>>
>> --
>> Thanks,
>> Pawan Agnihotri
>>
>
>
>
> --
> *Mars Hall
> 415-818-7039 <(415)%20818-7039>
> Customer Facing Architect
> Salesforce Platform / Heroku
> San Francisco, California
>
--
Thanks,
Pawan Agnihotri
Re: [ERROR] [Storage$] Required repository (METADATA) configuration
is missing.
Posted by Mars Hall <ma...@salesforce.com>.
We added support for Elasticsearch 5 last year, so current PredictionIO
uses HTTP/REST protocol on port 9200, not the native protocol on port 9300.
Here's the local dev config I have working with PIO 0.12.0 and
Elasticsearch 5:
PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=$PIO_HOME/vendors/elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
I believe Elasticsearch 1 is only supported via native protocol on port
9300, which is eventually being removed in a future PIO release.
On Thu, Mar 8, 2018 at 2:50 AM, Pawan Agnihotri <pa...@gmail.com>
wrote:
> Hello Donald and Team,
>
> I am working POC and I would like to use predictionIO.I know its
> configuration issue with elasticsearch but I am kind of stuck with below
> error so reaching out for help.
>
> I am in need of some quick hand here as the time is running out. anything
> you feel I can try out or steps would be helpful please.
>
>
> *2018-03-07 21:36:15,602 ERROR
> org.apache.predictionio.tools.console.Console$ [main] - None of the
> configured nodes are available: []
> (org.elasticsearch.client.transport.NoNodeAvailableException)*
> *org.elasticsearch.client.transport.NoNodeAvailableException: None of the
> configured nodes are available: []*
>
>
> *I am using the steps from
> - http://predictionio.apache.org/install/install-sourcecode/
> <http://predictionio.apache.org/install/install-sourcecode/>*
>
> *below is pio-env.sh and Error logs*
>
> *[mapr@valvcshad004vm conf]$ cat pio-env.sh*
> #!/usr/bin/env bash
> #
> # Copy this file as pio-env.sh and edit it for your site's configuration.
> #
> # Licensed to the Apache Software Foundation (ASF) under one or more
> # contributor license agreements. See the NOTICE file distributed with
> # this work for additional information regarding copyright ownership.
> # The ASF licenses this file to You under the Apache License, Version 2.0
> # (the "License"); you may not use this file except in compliance with
> # the License. You may obtain a copy of the License at
> #
> # http://www.apache.org/licenses/LICENSE-2.0
> #
> # Unless required by applicable law or agreed to in writing, software
> # distributed under the License is distributed on an "AS IS" BASIS,
> # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> # See the License for the specific language governing permissions and
> # limitations under the License.
> #
>
> # PredictionIO Main Configuration
> #
> # This section controls core behavior of PredictionIO. It is very likely
> that
> # you need to change these to fit your site.
>
> # SPARK_HOME: Apache Spark is a hard dependency and must be configured.
> #SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6
> SPARK_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/
> spark-2.1.1-bin-hadoop2.6
> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.2.1.jar
> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.37.jar
>
> # ES_CONF_DIR: You must configure this if you have advanced configuration
> for
> # your Elasticsearch setup.
> # ES_CONF_DIR=/opt/elasticsearch
>
> # HADOOP_CONF_DIR: You must configure this if you intend to run
> PredictionIO
> # with Hadoop 2.
> # HADOOP_CONF_DIR=/opt/hadoop
>
> # HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO
> # with HBase on a remote cluster.
> # HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf
>
> # Filesystem paths where PredictionIO uses as block storage.
> PIO_FS_BASEDIR=$HOME/.pio_store
> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>
> # PredictionIO Storage Configuration
> #
> # This section controls programs that make use of PredictionIO's built-in
> # storage facilities. Default values are shown below.
> #
> # For more information on storage configuration please refer to
> # http://predictionio.incubator.apache.org/system/anotherdatastore/
>
> # Storage Repositories
>
> # Default is to use PostgreSQL
> #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
> #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
>
> #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
>
> #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
> #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
>
> PIO_STORAGE_REPOSITORIES_METADATA_NAME=predictionio_metadata
> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>
> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=predictionio_eventdata
> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
>
> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_
> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
>
> # Storage Data Sources
>
>
> # PostgreSQL Default Settings
> # Please change "pio" to your database name in
> PIO_STORAGE_SOURCES_PGSQL_URL
> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
> #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
> #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
> #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
> #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>
> # MySQL Example
> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>
> # Elasticsearch Example
> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> #PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/mapr-predictionio/
> vendors/elasticsearch-5.5.2
>
> # Local File System Example
> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>
> # HBase Example
> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
> PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-
> predictionio/vendors/hbase-1.2.6
>
>
>
> *ERROR in the logs---*
>
> 2018-03-07 21:36:14,491 INFO org.apache.predictionio.data.storage.Storage$
> [main] - Verifying Meta Data Backend (Source: ELASTICSEARCH)...
> 2018-03-07 21:36:15,601 ERROR org.apache.predictionio.tools.console.Console$
> [main] - Unable to connect to all storage backends successfully. The
> following shows the error message from the storage backend.
> 2018-03-07 21:36:15,602 ERROR org.apache.predictionio.tools.console.Console$
> [main] - None of the configured nodes are available: []
> (org.elasticsearch.client.transport.NoNodeAvailableException)
> org.elasticsearch.client.transport.NoNodeAvailableException: None of the
> configured nodes are available: []
> at org.elasticsearch.client.transport.TransportClientNodesService.
> ensureNodesAreAvailable(TransportClientNodesService.java:305)
> at org.elasticsearch.client.transport.TransportClientNodesService.
> execute(TransportClientNodesService.java:200)
> at org.elasticsearch.client.transport.support.
> InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminC
> lient.java:86)
> at org.elasticsearch.client.support.AbstractIndicesAdminClient.
> exists(AbstractIndicesAdminClient.java:178)
> at org.elasticsearch.action.admin.indices.exists.indices.
> IndicesExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:53)
> at org.elasticsearch.action.ActionRequestBuilder.execute(
> ActionRequestBuilder.java:91)
> at org.elasticsearch.action.ActionRequestBuilder.execute(
> ActionRequestBuilder.java:65)
> at org.elasticsearch.action.ActionRequestBuilder.get(
> ActionRequestBuilder.java:73)
> at org.apache.predictionio.data.storage.elasticsearch.
> ESEngineInstances.<init>(ESEngineInstances.scala:42)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(
> NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.predictionio.data.storage.Storage$.
> getDataObject(Storage.scala:306)
> at org.apache.predictionio.data.storage.Storage$.
> getDataObjectFromRepo(Storage.scala:266)
> at org.apache.predictionio.data.storage.Storage$.
> getMetaDataEngineInstances(Storage.scala:367)
> at org.apache.predictionio.data.storage.Storage$.
> verifyAllDataObjects(Storage.scala:342)
> at org.apache.predictionio.tools.console.Console$.status(
> Console.scala:1087)
> at org.apache.predictionio.tools.console.Console$$anonfun$main$
> 1.apply(Console.scala:737)
> at org.apache.predictionio.tools.console.Console$$anonfun$main$
> 1.apply(Console.scala:696)
> at scala.Option.map(Option.scala:145)
> at org.apache.predictionio.tools.console.Console$.main(Console.
> scala:696)
> at org.apache.predictionio.tools.console.Console.main(Console.
> scala)
> 2018-03-07 21:36:15,605 ERROR org.apache.predictionio.tools.console.Console$
> [main] - Dumping configuration of initialized storage backend sources.
> Please make sure they are correct.
> 2018-03-07 21:36:15,607 ERROR org.apache.predictionio.tools.console.Console$
> [main] - Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration:
> HOSTS -> localhost, TYPE -> elasticsearch, HOME -> /dfs/pawan_scala/mapr-
> predictionio/vendors/elasticsearch-5.5.2, PORTS -> 9300
> 2018-03-07 21:36:42,649 INFO org.apache.predictionio.tools.console.Console$
> [main] - Creating Event Server at 0.0.0.0:7070
> 2018-03-07 21:36:43,417 ERROR org.apache.predictionio.data.storage.Storage$
> [main] - Error initializing storage client for source HBASE
> java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
> at org.apache.predictionio.data.storage.hbase.StorageClient.<
> init>(StorageClient.scala:46)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(
> NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.predictionio.data.storage.Storage$.getClient(
> Storage.scala:220)
> at org.apache.predictionio.data.storage.Storage$.org$apache$
> predictionio$data$storage$Storage$$updateS2CM(Storage.scala:251)
> at org.apache.predictionio.data.storage.Storage$$anonfun$
> sourcesToClientMeta$1.apply(Storage.scala:212)
> at org.apache.predictionio.data.storage.Storage$$anonfun$
> sourcesToClientMeta$1.apply(Storage.scala:212)
>
>
> On Wed, Mar 7, 2018 at 1:21 AM, Pawan Agnihotri <pawan.agnihotri@gmail.com
> > wrote:
>
>> Hello,
>>
>> I need your help to configure predictionIO on linux 7.2 -
>>
>> I am using
>> http://predictionio.apache.org/install/install-sourcecode/ link for
>> steps and installed spark, elastic search and hbase but getting below error
>>
>>
>> [mapr@valvcshad004vm bin]$ ./pio status
>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89:
>> /opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such
>> file or directory
>> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90:
>> generate_compatible_classpath: command not found
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in [jar:file:/dfs/pawan_scala/map
>> r-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT.jar!/
>> org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in [jar:file:/data/opt/mapr/lib/s
>> lf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> [INFO] [Console$] Inspecting PredictionIO...
>> [INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at
>> /dfs/pawan_scala/mapr-predictionio
>> [INFO] [Console$] Inspecting Apache Spark...
>> [INFO] [Console$] Apache Spark is installed at
>> /dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6
>> [INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum requirement
>> of 1.3.0)
>> [INFO] [Console$] Inspecting storage backend connections...
>> *[WARN] [Storage$] There is no properly configured repository.*
>> *[ERROR] [Storage$] Required repository (METADATA) configuration is
>> missing.*
>> *[ERROR] [Storage$] There were 1 configuration errors. Exiting.*
>> [mapr@valvcshad004vm bin]$
>>
>> Here is my *pio-env.sh *file
>>
>> [mapr@valvcshad004vm conf]$ cat pio-env.sh
>> # Default is to use PostgreSQL
>> #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>> #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
>>
>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>> #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
>>
>> #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>> #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
>>
>> # Storage Data Sources
>>
>> # PostgreSQL Default Settings
>> # Please change "pio" to your database name in
>> PIO_STORAGE_SOURCES_PGSQL_URL
>> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
>> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
>> #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
>> #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
>> #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
>> #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>>
>> # MySQL Example
>> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
>> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
>> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
>> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>>
>> # Elasticsearch Example
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch_cluster_name
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/
>> mapr-predictionio/vendors/elasticsearch-5.5.2
>>
>> # Local File System Example
>> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
>> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>>
>> # HBase Example
>> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
>> PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predic
>> tionio/vendors/hbase-1.2.6
>>
>> [mapr@valvcshad004vm conf]$
>>
>>
>> --
>> Thanks,
>> Pawan Agnihotri
>>
>
>
>
> --
> Thanks,
> Pawan Agnihotri
>
--
*Mars Hall
415-818-7039
Customer Facing Architect
Salesforce Platform / Heroku
San Francisco, California
Re: [ERROR] [Storage$] Required repository (METADATA) configuration
is missing.
Posted by Pawan Agnihotri <pa...@gmail.com>.
Hello Donald and Team,
I am working POC and I would like to use predictionIO.I know its
configuration issue with elasticsearch but I am kind of stuck with below
error so reaching out for help.
I am in need of some quick hand here as the time is running out. anything
you feel I can try out or steps would be helpful please.
*2018-03-07 21:36:15,602 ERROR
org.apache.predictionio.tools.console.Console$ [main] - None of the
configured nodes are available: []
(org.elasticsearch.client.transport.NoNodeAvailableException)*
*org.elasticsearch.client.transport.NoNodeAvailableException: None of the
configured nodes are available: []*
*I am using the steps from
- http://predictionio.apache.org/install/install-sourcecode/
<http://predictionio.apache.org/install/install-sourcecode/>*
*below is pio-env.sh and Error logs*
*[mapr@valvcshad004vm conf]$ cat pio-env.sh*
#!/usr/bin/env bash
#
# Copy this file as pio-env.sh and edit it for your site's configuration.
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# PredictionIO Main Configuration
#
# This section controls core behavior of PredictionIO. It is very likely
that
# you need to change these to fit your site.
# SPARK_HOME: Apache Spark is a hard dependency and must be configured.
#SPARK_HOME=$PIO_HOME/vendors/spark-1.5.1-bin-hadoop2.6
SPARK_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/spark-2.1.1-bin-hadoop2.6
POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.2.1.jar
MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.37.jar
# ES_CONF_DIR: You must configure this if you have advanced configuration
for
# your Elasticsearch setup.
# ES_CONF_DIR=/opt/elasticsearch
# HADOOP_CONF_DIR: You must configure this if you intend to run PredictionIO
# with Hadoop 2.
# HADOOP_CONF_DIR=/opt/hadoop
# HBASE_CONF_DIR: You must configure this if you intend to run PredictionIO
# with HBase on a remote cluster.
# HBASE_CONF_DIR=$PIO_HOME/vendors/hbase-1.0.0/conf
# Filesystem paths where PredictionIO uses as block storage.
PIO_FS_BASEDIR=$HOME/.pio_store
PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
# PredictionIO Storage Configuration
#
# This section controls programs that make use of PredictionIO's built-in
# storage facilities. Default values are shown below.
#
# For more information on storage configuration please refer to
# http://predictionio.incubator.apache.org/system/anotherdatastore/
# Storage Repositories
# Default is to use PostgreSQL
#PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
#PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
#PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
#PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
#PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
#PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
PIO_STORAGE_REPOSITORIES_METADATA_NAME=predictionio_metadata
PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=predictionio_eventdata
PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=HBASE
PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_
PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=LOCALFS
# Storage Data Sources
# PostgreSQL Default Settings
# Please change "pio" to your database name in PIO_STORAGE_SOURCES_PGSQL_URL
# Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
# PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
#PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
#PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
#PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
#$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
# MySQL Example
# PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
# PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
# PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
# PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
# Elasticsearch Example
PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
#PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2
# Local File System Example
PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
# HBase Example
PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-predictionio/vendors/hbase-1.2.6
*ERROR in the logs---*
2018-03-07 21:36:14,491 INFO org.apache.predictionio.data.storage.Storage$
[main] - Verifying Meta Data Backend (Source: ELASTICSEARCH)...
2018-03-07 21:36:15,601 ERROR
org.apache.predictionio.tools.console.Console$ [main] - Unable to connect
to all storage backends successfully. The following shows the error message
from the storage backend.
2018-03-07 21:36:15,602 ERROR
org.apache.predictionio.tools.console.Console$ [main] - None of the
configured nodes are available: []
(org.elasticsearch.client.transport.NoNodeAvailableException)
org.elasticsearch.client.transport.NoNodeAvailableException: None of the
configured nodes are available: []
at
org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:305)
at
org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:200)
at
org.elasticsearch.client.transport.support.InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86)
at
org.elasticsearch.client.support.AbstractIndicesAdminClient.exists(AbstractIndicesAdminClient.java:178)
at
org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:53)
at
org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)
at
org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65)
at
org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:73)
at
org.apache.predictionio.data.storage.elasticsearch.ESEngineInstances.<init>(ESEngineInstances.scala:42)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:306)
at
org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storage.scala:266)
at
org.apache.predictionio.data.storage.Storage$.getMetaDataEngineInstances(Storage.scala:367)
at
org.apache.predictionio.data.storage.Storage$.verifyAllDataObjects(Storage.scala:342)
at
org.apache.predictionio.tools.console.Console$.status(Console.scala:1087)
at
org.apache.predictionio.tools.console.Console$$anonfun$main$1.apply(Console.scala:737)
at
org.apache.predictionio.tools.console.Console$$anonfun$main$1.apply(Console.scala:696)
at scala.Option.map(Option.scala:145)
at
org.apache.predictionio.tools.console.Console$.main(Console.scala:696)
at org.apache.predictionio.tools.console.Console.main(Console.scala)
2018-03-07 21:36:15,605 ERROR
org.apache.predictionio.tools.console.Console$ [main] - Dumping
configuration of initialized storage backend sources. Please make sure they
are correct.
2018-03-07 21:36:15,607 ERROR
org.apache.predictionio.tools.console.Console$ [main] - Source Name:
ELASTICSEARCH; Type: elasticsearch; Configuration: HOSTS -> localhost, TYPE
-> elasticsearch, HOME ->
/dfs/pawan_scala/mapr-predictionio/vendors/elasticsearch-5.5.2, PORTS ->
9300
2018-03-07 21:36:42,649 INFO
org.apache.predictionio.tools.console.Console$ [main] - Creating Event
Server at 0.0.0.0:7070
2018-03-07 21:36:43,417 ERROR org.apache.predictionio.data.storage.Storage$
[main] - Error initializing storage client for source HBASE
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
at
org.apache.predictionio.data.storage.hbase.StorageClient.<init>(StorageClient.scala:46)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.predictionio.data.storage.Storage$.getClient(Storage.scala:220)
at
org.apache.predictionio.data.storage.Storage$.org$apache$predictionio$data$storage$Storage$$updateS2CM(Storage.scala:251)
at
org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:212)
at
org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$1.apply(Storage.scala:212)
On Wed, Mar 7, 2018 at 1:21 AM, Pawan Agnihotri <pa...@gmail.com>
wrote:
> Hello,
>
> I need your help to configure predictionIO on linux 7.2 -
>
> I am using
> http://predictionio.apache.org/install/install-sourcecode/ link for
> steps and installed spark, elastic search and hbase but getting below error
>
>
> [mapr@valvcshad004vm bin]$ ./pio status
> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 89:
> /opt/mapr/spark/spark-2.1.0/mapr-util/generate-classpath.sh: No such file
> or directory
> /dfs/pawan_scala/mapr-predictionio/bin/pio-class: line 90:
> generate_compatible_classpath: command not found
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/dfs/pawan_scala/
> mapr-predictionio/assembly/pio-assembly-0.10.0-SNAPSHOT.
> jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/data/opt/mapr/lib/
> slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> [INFO] [Console$] Inspecting PredictionIO...
> [INFO] [Console$] PredictionIO 0.10.0-SNAPSHOT is installed at
> /dfs/pawan_scala/mapr-predictionio
> [INFO] [Console$] Inspecting Apache Spark...
> [INFO] [Console$] Apache Spark is installed at /dfs/pawan_scala/mapr-
> predictionio/vendors/spark-2.1.1-bin-hadoop2.6
> [INFO] [Console$] Apache Spark 2.1.1 detected (meets minimum requirement
> of 1.3.0)
> [INFO] [Console$] Inspecting storage backend connections...
> *[WARN] [Storage$] There is no properly configured repository.*
> *[ERROR] [Storage$] Required repository (METADATA) configuration is
> missing.*
> *[ERROR] [Storage$] There were 1 configuration errors. Exiting.*
> [mapr@valvcshad004vm bin]$
>
> Here is my *pio-env.sh *file
>
> [mapr@valvcshad004vm conf]$ cat pio-env.sh
> # Default is to use PostgreSQL
> #PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
> #PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=PGSQL
>
> #PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> #PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=PGSQL
>
> #PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
> #PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=PGSQL
>
> # Storage Data Sources
>
> # PostgreSQL Default Settings
> # Please change "pio" to your database name in
> PIO_STORAGE_SOURCES_PGSQL_URL
> # Please change PIO_STORAGE_SOURCES_PGSQL_USERNAME and
> # PIO_STORAGE_SOURCES_PGSQL_PASSWORD accordingly
> #PIO_STORAGE_SOURCES_PGSQL_TYPE=jdbc
> #PIO_STORAGE_SOURCES_PGSQL_URL=jdbc:postgresql://localhost/pio
> #PIO_STORAGE_SOURCES_PGSQL_USERNAME=pio
> #$PIO_STORAGE_SOURCES_PGSQL_PASSWORD=pio
>
> # MySQL Example
> # PIO_STORAGE_SOURCES_MYSQL_TYPE=jdbc
> # PIO_STORAGE_SOURCES_MYSQL_URL=jdbc:mysql://localhost/pio
> # PIO_STORAGE_SOURCES_MYSQL_USERNAME=pio
> # PIO_STORAGE_SOURCES_MYSQL_PASSWORD=pio
>
> # Elasticsearch Example
> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> PIO_STORAGE_SOURCES_ELASTICSEARCH_CLUSTERNAME=elasticsearch_cluster_name
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9300
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/dfs/pawan_
> scala/mapr-predictionio/vendors/elasticsearch-5.5.2
>
> # Local File System Example
> PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
> PIO_STORAGE_SOURCES_LOCALFS_PATH=$PIO_FS_BASEDIR/models
>
> # HBase Example
> PIO_STORAGE_SOURCES_HBASE_TYPE=hbase
> PIO_STORAGE_SOURCES_HBASE_HOME=/dfs/pawan_scala/mapr-
> predictionio/vendors/hbase-1.2.6
>
> [mapr@valvcshad004vm conf]$
>
>
> --
> Thanks,
> Pawan Agnihotri
>
--
Thanks,
Pawan Agnihotri