You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@predictionio.apache.org by Praveen Bhasker <ne...@gmail.com> on 2018/09/20 14:06:25 UTC

Duplicate field 'Status' when I try to run 'pio status'

When ever I do pio status I get the following error

[INFO] [Management$] Inspecting PredictionIO...
[INFO] [Management$] PredictionIO 0.13.0 is installed at
/Users/prvns/tools/PredictionIO-0.13.0
[INFO] [Management$] Inspecting Apache Spark...
[INFO] [Management$] Apache Spark is installed at
/Users/prvns/tools/PredictionIO-0.13.0/vendors/spark-2.3.1-bin-hadoop2.7
[INFO] [Management$] Apache Spark 2.3.1 detected (meets minimum
requirement of 1.6.3)
[INFO] [Management$] Inspecting storage backend connections...
[INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
[ERROR] [Management$] Unable to connect to all storage backends successfully.
The following shows the error message from the storage backend.

PUT http://localhost:9200/pio_meta/_mapping/engine_instances: HTTP/1.1
400 Bad Request
{"error":{"root_cause":[{"type":"parse_exception","reason":"Failed to
parse content to map"}],"type":"parse_exception","reason":"Failed to
parse content to
map","caused_by":{"type":"json_parse_exception","reason":"Duplicate
field 'status'\n at [Source:
org.elasticsearch.common.compress.DeflateCompressor$1@6b496f00; line:
1, column: 462]"}},"status":400}
(org.apache.predictionio.shaded.org.elasticsearch.client.ResponseException)

Dumping configuration of initialized storage backend sources.
Please make sure they are correct.

Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME
-> /usr/local/Cellar/elasticsearch/6.2.4/, HOSTS -> localhost, PORTS
-> 9200, SCHEMES -> http, TYPE -> elasticsearch

My pio-env.sh looks like

SPARK_HOME=$PIO_HOME/vendors/spark-2.3.1-bin-hadoop2.7
POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
PIO_FS_BASEDIR=$HOME/.pio_store
PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=ELASTICSEARCH
PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=ELASTICSEARCH
PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/usr/local/Cellar/elasticsearch/6.2.4/

Why is this not working?

Re: Duplicate field 'Status' when I try to run 'pio status'

Posted by Alex Merritt <em...@apache.org>.

FYI, I have changes (and will PR them) that enable PIO to work with ES 6
and are backwards-compatible with earlier ES versions.
The downside, and the reason this should go in a non-patch release is that
ES 6 necessitates changes to the metadata structure, and this breaks
existing PIO apps.
This is because ES 6 no longer allows creation of indices with multiple
types (see
https://www.elastic.co/blog/index-type-parent-child-join-now-future-in-elasticsearch
)

Alex

On Thu, Sep 20, 2018 at 10:49 AM Praveen Bhasker <ne...@gmail.com>
wrote:

> Hi Donald,
>
> That worked. Thanks!
>
> - Praveen
>
> On Thu, Sep 20, 2018 at 11:39 AM Donald Szeto <do...@apache.org> wrote:
>
>> Hi Praveen,
>>
>> It looks like you are using Elasticsearch 6. I think there are some
>> breaking changes with ES6 API compared to ES5, and PredictionIO does not
>> officially support ES6 yet.
>>
>> Regards,
>> Donald
>>
>> On Thu, Sep 20, 2018 at 7:06 AM Praveen Bhasker <ne...@gmail.com>
>> wrote:
>>
>>> When ever I do pio status I get the following error
>>>
>>> [INFO] [Management$] Inspecting PredictionIO...
>>> [INFO] [Management$] PredictionIO 0.13.0 is installed at /Users/prvns/tools/PredictionIO-0.13.0
>>> [INFO] [Management$] Inspecting Apache Spark...
>>> [INFO] [Management$] Apache Spark is installed at /Users/prvns/tools/PredictionIO-0.13.0/vendors/spark-2.3.1-bin-hadoop2.7
>>> [INFO] [Management$] Apache Spark 2.3.1 detected (meets minimum requirement of 1.6.3)
>>> [INFO] [Management$] Inspecting storage backend connections...
>>> [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
>>> [ERROR] [Management$] Unable to connect to all storage backends successfully.
>>> The following shows the error message from the storage backend.
>>>
>>> PUT http://localhost:9200/pio_meta/_mapping/engine_instances: HTTP/1.1 400 Bad Request
>>> {"error":{"root_cause":[{"type":"parse_exception","reason":"Failed to parse content to map"}],"type":"parse_exception","reason":"Failed to parse content to map","caused_by":{"type":"json_parse_exception","reason":"Duplicate field 'status'\n at [Source: org.elasticsearch.common.compress.DeflateCompressor$1@6b496f00; line: 1, column: 462]"}},"status":400} (org.apache.predictionio.shaded.org.elasticsearch.client.ResponseException)
>>>
>>> Dumping configuration of initialized storage backend sources.
>>> Please make sure they are correct.
>>>
>>> Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> /usr/local/Cellar/elasticsearch/6.2.4/, HOSTS -> localhost, PORTS -> 9200, SCHEMES -> http, TYPE -> elasticsearch
>>>
>>> My pio-env.sh looks like
>>>
>>> SPARK_HOME=$PIO_HOME/vendors/spark-2.3.1-bin-hadoop2.7
>>> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
>>> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
>>> PIO_FS_BASEDIR=$HOME/.pio_store
>>> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
>>> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>>> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>>> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>>> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>>> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=ELASTICSEARCH
>>> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>>> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=ELASTICSEARCH
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
>>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/usr/local/Cellar/elasticsearch/6.2.4/
>>>
>>> Why is this not working?
>>>
>>

Re: Duplicate field 'Status' when I try to run 'pio status'

Posted by Praveen Bhasker <ne...@gmail.com>.

Hi Donald,

That worked. Thanks!

- Praveen

On Thu, Sep 20, 2018 at 11:39 AM Donald Szeto <do...@apache.org> wrote:

> Hi Praveen,
>
> It looks like you are using Elasticsearch 6. I think there are some
> breaking changes with ES6 API compared to ES5, and PredictionIO does not
> officially support ES6 yet.
>
> Regards,
> Donald
>
> On Thu, Sep 20, 2018 at 7:06 AM Praveen Bhasker <ne...@gmail.com>
> wrote:
>
>> When ever I do pio status I get the following error
>>
>> [INFO] [Management$] Inspecting PredictionIO...
>> [INFO] [Management$] PredictionIO 0.13.0 is installed at /Users/prvns/tools/PredictionIO-0.13.0
>> [INFO] [Management$] Inspecting Apache Spark...
>> [INFO] [Management$] Apache Spark is installed at /Users/prvns/tools/PredictionIO-0.13.0/vendors/spark-2.3.1-bin-hadoop2.7
>> [INFO] [Management$] Apache Spark 2.3.1 detected (meets minimum requirement of 1.6.3)
>> [INFO] [Management$] Inspecting storage backend connections...
>> [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
>> [ERROR] [Management$] Unable to connect to all storage backends successfully.
>> The following shows the error message from the storage backend.
>>
>> PUT http://localhost:9200/pio_meta/_mapping/engine_instances: HTTP/1.1 400 Bad Request
>> {"error":{"root_cause":[{"type":"parse_exception","reason":"Failed to parse content to map"}],"type":"parse_exception","reason":"Failed to parse content to map","caused_by":{"type":"json_parse_exception","reason":"Duplicate field 'status'\n at [Source: org.elasticsearch.common.compress.DeflateCompressor$1@6b496f00; line: 1, column: 462]"}},"status":400} (org.apache.predictionio.shaded.org.elasticsearch.client.ResponseException)
>>
>> Dumping configuration of initialized storage backend sources.
>> Please make sure they are correct.
>>
>> Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> /usr/local/Cellar/elasticsearch/6.2.4/, HOSTS -> localhost, PORTS -> 9200, SCHEMES -> http, TYPE -> elasticsearch
>>
>> My pio-env.sh looks like
>>
>> SPARK_HOME=$PIO_HOME/vendors/spark-2.3.1-bin-hadoop2.7
>> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
>> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
>> PIO_FS_BASEDIR=$HOME/.pio_store
>> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
>> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
>> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
>> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
>> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
>> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=ELASTICSEARCH
>> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
>> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=ELASTICSEARCH
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
>> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/usr/local/Cellar/elasticsearch/6.2.4/
>>
>> Why is this not working?
>>
>

Re: Duplicate field 'Status' when I try to run 'pio status'

Posted by Donald Szeto <do...@apache.org>.

Hi Praveen,

It looks like you are using Elasticsearch 6. I think there are some
breaking changes with ES6 API compared to ES5, and PredictionIO does not
officially support ES6 yet.

Regards,
Donald

On Thu, Sep 20, 2018 at 7:06 AM Praveen Bhasker <ne...@gmail.com>
wrote:

> When ever I do pio status I get the following error
>
> [INFO] [Management$] Inspecting PredictionIO...
> [INFO] [Management$] PredictionIO 0.13.0 is installed at /Users/prvns/tools/PredictionIO-0.13.0
> [INFO] [Management$] Inspecting Apache Spark...
> [INFO] [Management$] Apache Spark is installed at /Users/prvns/tools/PredictionIO-0.13.0/vendors/spark-2.3.1-bin-hadoop2.7
> [INFO] [Management$] Apache Spark 2.3.1 detected (meets minimum requirement of 1.6.3)
> [INFO] [Management$] Inspecting storage backend connections...
> [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...
> [ERROR] [Management$] Unable to connect to all storage backends successfully.
> The following shows the error message from the storage backend.
>
> PUT http://localhost:9200/pio_meta/_mapping/engine_instances: HTTP/1.1 400 Bad Request
> {"error":{"root_cause":[{"type":"parse_exception","reason":"Failed to parse content to map"}],"type":"parse_exception","reason":"Failed to parse content to map","caused_by":{"type":"json_parse_exception","reason":"Duplicate field 'status'\n at [Source: org.elasticsearch.common.compress.DeflateCompressor$1@6b496f00; line: 1, column: 462]"}},"status":400} (org.apache.predictionio.shaded.org.elasticsearch.client.ResponseException)
>
> Dumping configuration of initialized storage backend sources.
> Please make sure they are correct.
>
> Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: HOME -> /usr/local/Cellar/elasticsearch/6.2.4/, HOSTS -> localhost, PORTS -> 9200, SCHEMES -> http, TYPE -> elasticsearch
>
> My pio-env.sh looks like
>
> SPARK_HOME=$PIO_HOME/vendors/spark-2.3.1-bin-hadoop2.7
> POSTGRES_JDBC_DRIVER=$PIO_HOME/lib/postgresql-42.0.0.jar
> MYSQL_JDBC_DRIVER=$PIO_HOME/lib/mysql-connector-java-5.1.41.jar
> PIO_FS_BASEDIR=$HOME/.pio_store
> PIO_FS_ENGINESDIR=$PIO_FS_BASEDIR/engines
> PIO_FS_TMPDIR=$PIO_FS_BASEDIR/tmp
> PIO_STORAGE_REPOSITORIES_METADATA_NAME=pio_meta
> PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH
> PIO_STORAGE_REPOSITORIES_EVENTDATA_NAME=pio_event
> PIO_STORAGE_REPOSITORIES_EVENTDATA_SOURCE=ELASTICSEARCH
> PIO_STORAGE_REPOSITORIES_MODELDATA_NAME=pio_model
> PIO_STORAGE_REPOSITORIES_MODELDATA_SOURCE=ELASTICSEARCH
> PIO_STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=localhost
> PIO_STORAGE_SOURCES_ELASTICSEARCH_PORTS=9200
> PIO_STORAGE_SOURCES_ELASTICSEARCH_SCHEMES=http
> PIO_STORAGE_SOURCES_ELASTICSEARCH_HOME=/usr/local/Cellar/elasticsearch/6.2.4/
>
> Why is this not working?
>