PutElasticSearch Error

[Aruna Sankaralingam <Ar...@Cormac-Corp.com>]

I had the port changed from 9200 to 9300. Still getting this error.

[cid:image004.png@01D384A7.F1CA53D0]

From: Aruna Sankaralingam [mailto:Aruna.Sankaralingam@Cormac-Corp.com]
Sent: Wednesday, January 03, 2018 3:27 PM
To: users@nifi.apache.org
Subject: ListS3 and FetchS3

I have 2 files in S3 - one is CSV and other is pdf. I want to fetch the pdf and load into Elastic Search. Even though I give the object key as the pdf file name, it is still taking all the files from S3. How do I make sure it takes only the file that is needed?

[cid:image005.png@01D384A7.F1CA53D0]

[cid:image006.png@01D384A7.F1CA53D0]

[cid:image007.png@01D384A7.F1CA53D0]

Re: PutElasticSearch Error

[Mike Thomsen <mi...@gmail.com>]

It just hit me that you're trying to post PDF and CSV files directly to
ElasticSearch. There is apparently a plugin you can install that will use
Tika behind the scenes to extract data from PDF, but unless you set that
up, you can't just throw a PDF file at ElasticSearch. Same with CSV data.
ElasticSearch expects a JSON document. So you're going to need to expand
the flow to detect the file type and handle it appropriately so that a JSON
document is what reaches PutElasticSearch.

On Wed, Jan 3, 2018 at 7:41 PM, Mike Thomsen <mi...@gmail.com> wrote:

> The error you're getting from PutElasticSearch is telling you that it
> cannot reach the ElasticSearch node(s) you've configured. The two likely
> reasons that come to mind are either that ElasticSearch is not able to
> respond to queries because it's stuck in a bad state or it's been bound to
> a particular IP address and is excluding localhost. You can open up ES
> completely and eliminate the second potential reason by editing
> elasticsearch.yml and setting this:
>
> network.host: 0.0.0.0
>
> Then restart ES and kick the tires in Kibana.
>
> FWIW, I would recommend switching to PutElasticSearchHttp/HttpRecord
> going forward. From what I understand, ElasticSearch's Java client is
> moving away from using the Transport API (port 9300) to a pure REST API, so
> you should see less pain by just using that and pointing it to port 9200.
>
> On Wed, Jan 3, 2018 at 3:32 PM, Aruna Sankaralingam <
> Aruna.Sankaralingam@cormac-corp.com> wrote:
>
>> I had the port changed from 9200 to 9300. Still getting this error.
>>
>>
>>
>>
>>
>> *From:* Aruna Sankaralingam [mailto:Aruna.Sankaralingam@Cormac-Corp.com]
>> *Sent:* Wednesday, January 03, 2018 3:27 PM
>> *To:* users@nifi.apache.org
>> *Subject:* ListS3 and FetchS3
>>
>>
>>
>> I have 2 files in S3 – one is CSV and other is pdf. I want to fetch the
>> pdf and load into Elastic Search. Even though I give the object key as the
>> pdf file name, it is still taking all the files from S3. How do I make sure
>> it takes only the file that is needed?
>>
>>
>>
>>
>>
>>
>>
>>
>

Re: PutElasticSearch Error

[Mike Thomsen <mi...@gmail.com>]

The error you're getting from PutElasticSearch is telling you that it
cannot reach the ElasticSearch node(s) you've configured. The two likely
reasons that come to mind are either that ElasticSearch is not able to
respond to queries because it's stuck in a bad state or it's been bound to
a particular IP address and is excluding localhost. You can open up ES
completely and eliminate the second potential reason by editing
elasticsearch.yml and setting this:

network.host: 0.0.0.0

Then restart ES and kick the tires in Kibana.

FWIW, I would recommend switching to PutElasticSearchHttp/HttpRecord going
forward. From what I understand, ElasticSearch's Java client is moving away
from using the Transport API (port 9300) to a pure REST API, so you should
see less pain by just using that and pointing it to port 9200.

On Wed, Jan 3, 2018 at 3:32 PM, Aruna Sankaralingam <
Aruna.Sankaralingam@cormac-corp.com> wrote:

> I had the port changed from 9200 to 9300. Still getting this error.
>
>
>
>
>
> *From:* Aruna Sankaralingam [mailto:Aruna.Sankaralingam@Cormac-Corp.com]
> *Sent:* Wednesday, January 03, 2018 3:27 PM
> *To:* users@nifi.apache.org
> *Subject:* ListS3 and FetchS3
>
>
>
> I have 2 files in S3 – one is CSV and other is pdf. I want to fetch the
> pdf and load into Elastic Search. Even though I give the object key as the
> pdf file name, it is still taking all the files from S3. How do I make sure
> it takes only the file that is needed?
>
>
>
>
>
>
>
>