You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Andrew McDonald <am...@ccri.com> on 2018/11/28 15:35:28 UTC
How do you use a custom region?
I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3 processors can not
be initialized.
Nifi 1.7.1 uses 1.11.319 and is throwing an IllegalArgumentException: no
region provided
The region I'm using isn't in the enum, so is it possible to use a
custom region?
Regards,Andrew
Re: How do you use a custom region?
Posted by Sivaprasanna <si...@gmail.com>.
Extremely sorry, Andrew. Was super busy for the past few days so wasn't
able to reply on time. Thanks for creating the Jira. I'll take a look.
Thanks once again.
-
Sivaprasanna
On Wed, Dec 12, 2018 at 7:18 PM Andrew McDonald <am...@ccri.com> wrote:
> I've created the JIRA https://issues.apache.org/jira/browse/NIFI-5893
>
> I couldn't find one if you had already made it so it could be a dup.
>
> Regards,Andrew
> On 11/30/18 11:58, Sivaprasanna wrote:
>
> I understand where the problem is. The reason why we replaced
> *this.client.setEndpoint(urlstr);* with *this.client.setEndpoint(urlstr,
> this.client.getServiceName(), this.region.getName()); *was that when
> working VPC enabled services, the endpoint URL wasn't properly parsed.
> Having said that, *this.region.getName()* shouldn't be there. I'll work
> on a solution. Thanks for bringing this up.
>
> -
> Sivaprasanna
>
> On Fri, Nov 30, 2018 at 9:49 PM Andrew McDonald <am...@ccri.com>
> wrote:
>
>> Setting line 289 in AbstractAWSProcessor.java to
>>
>> this.client.setEndpoint(urlstr);
>>
>> worked for me.
>>
>> Regards,Andrew
>> On 11/30/18 09:12, Andrew McDonald wrote:
>>
>> This is on an air-gapped system so I'll type in the essential part
>>
>> Failed to receive messages from Amazon SQS due to
>> com.amazonaws.services.sqs.model.AmazonSQSException: Credential should be
>> scoped to a valid region, not 'us-east-1'. (Service: AmazonSQS; Status
>> Code: 403; Error Code: SignatureDoesNotMatch; Request ID:
>> xxxxx-xxxx-xxxx-xxxx-xxxxxxxx).
>>
>> I believe that line 289 in AbstractAWSProcessor.java is setting the wrong
>> regionId
>>
>> this.client.setEndpoint(urlstr, this.client.getServiceName(), this.region.getName());
>>
>> The amazon docs for the third parameter (regionId) state:
>>
>> * @param regionId* The ID of the region in which this service resides AND the* overriding region for signing purposes.
>>
>> This would explain the SignatureDoesNotMatch error.
>>
>> The properties for the sqs processor work for nifi 1.4.0.
>>
>> Regards, Andrew
>>
>>
>> On 11/29/18 23:22, Sivaprasanna wrote:
>>
>> Andrew, Can you please share the complete error message and also show us
>> how the properties look for the GetSQS processor?
>>
>> Thanks.
>>
>> On Fri, Nov 30, 2018 at 3:45 AM Andrew McDonald <am...@ccri.com>
>> wrote:
>>
>>> So I've installed nifi 1.8.0 and endpoint override didn't help
>>>
>>> The GetSQS is throwing and exception
>>>
>>> com.amazonaws.services.sqs.model.AmazonSQSException: Credential should
>>> be scoped to a valid region, not 'us-east-1'.
>>>
>>> I'm guessing that the override endpoint is not overriding the region b/c
>>> 'us-east-1' is not my region.
>>>
>>> Regards, Andrew
>>> On 11/29/18 02:09, Sivaprasanna wrote:
>>>
>>> Yeah. This was added in 1.8.0 for SQS. However, the reason why a custom
>>> enum was added on the NiFi side[1] was to have a proper readable region
>>> instead of just the region code i.e., Asia Pacific (Singapore) instead
>>> of ap-southeast-1. However, I raised a request later to the AWS Java
>>> SDK team to have a readable name. It was added recently, if I remember
>>> correctly. So ideally this enum has to be removed and the one on the
>>> official AWS SDK has to be leveraged completely. I have created a Jira[2]
>>> and started working on it. I'll raise a PR soon.
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-5129
>>> [2] https://issues.apache.org/jira/browse/NIFI-5850
>>>
>>> Thanks,
>>>
>>> On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald <am...@ccri.com>
>>> wrote:
>>>
>>>> zenfenan added this later to 1.8.0, yay!
>>>> On 11/28/18 14:23, Andrew McDonald wrote:
>>>>
>>>>
>>>> This workaround doesn't work for sqs because it doesn't have endpoint
>>>> override URL property
>>>>
>>>>
>>>> On 11/28/18 12:16, Michael Moser wrote:
>>>>
>>>> Greetings! This JIRA ticket [1] describes the recommended work around
>>>> for AWS regions that aren't in the list.
>>>>
>>>> -- Mike
>>>>
>>>> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>>>
>>>>
>>>>
>>>> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan <jm...@buffalo.edu> wrote:
>>>>
>>>>> Andrew,
>>>>>
>>>>> I know there's a few regions not in the list. I'm not sure which
>>>>> region you're targeting, but at least for the case of one of the new
>>>>> regions, I submitted a PR for this. I haven't dug into it deeply, but it
>>>>> seems like a better way to do this might be to remove the enum entirely and
>>>>> get the region list via the AWS API, or allow a free-form entry.
>>>>>
>>>>> https://github.com/apache/nifi/pull/3187
>>>>>
>>>>>
>>>>> Jon
>>>>>
>>>>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald <am...@ccri.com>
>>>>> wrote:
>>>>>
>>>>>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3 processors can
>>>>>> not
>>>>>> be initialized.
>>>>>>
>>>>>> Nifi 1.7.1 uses 1.11.319 and is throwing an IllegalArgumentException:
>>>>>> no
>>>>>> region provided
>>>>>>
>>>>>> The region I'm using isn't in the enum, so is it possible to use a
>>>>>> custom region?
>>>>>>
>>>>>> Regards,Andrew
>>>>>>
>>>>>>
>>>>>>
Re: How do you use a custom region?
Posted by Andrew McDonald <am...@ccri.com>.
I've created the JIRA https://issues.apache.org/jira/browse/NIFI-5893
I couldn't find one if you had already made it so it could be a dup.
Regards,Andrew
On 11/30/18 11:58, Sivaprasanna wrote:
> I understand where the problem is. The reason why we replaced
> /this.client.setEndpoint(urlstr);/ with
> /this.client.setEndpoint(urlstr, this.client.getServiceName(),
> this.region.getName()); /was that when working VPC enabled services,
> the endpoint URL wasn't properly parsed. Having said that,
> /this.region.getName()/ shouldn't be there. I'll work on a solution.
> Thanks for bringing this up.
>
> -
> Sivaprasanna
>
> On Fri, Nov 30, 2018 at 9:49 PM Andrew McDonald <amcdonald@ccri.com
> <ma...@ccri.com>> wrote:
>
> Setting line 289 in AbstractAWSProcessor.java to
>
> this.client.setEndpoint(urlstr);
>
> worked for me.
>
> Regards,Andrew
>
> On 11/30/18 09:12, Andrew McDonald wrote:
>>
>> This is on an air-gapped system so I'll type in the essential part
>>
>> Failed to receive messages from Amazon SQS due to
>> com.amazonaws.services.sqs.model.AmazonSQSException: Credential
>> should be scoped to a valid region, not 'us-east-1'. (Service:
>> AmazonSQS; Status Code: 403; Error Code: SignatureDoesNotMatch;
>> Request ID: xxxxx-xxxx-xxxx-xxxx-xxxxxxxx).
>>
>> I believe that line 289 in AbstractAWSProcessor.java is setting
>> the wrong regionId
>>
>> this.client.setEndpoint(urlstr, this.client.getServiceName(),
>> this.region.getName());
>>
>> The amazon docs for the third parameter (regionId) state:
>>
>> * @param regionId * The ID of the region in which this service
>> resides AND the * overriding region for signing purposes.
>>
>> This would explain the SignatureDoesNotMatch error.
>>
>> The properties for the sqs processor work for nifi 1.4.0.
>>
>> Regards, Andrew
>>
>>
>> On 11/29/18 23:22, Sivaprasanna wrote:
>>> Andrew, Can you please share the complete error message and also
>>> show us how the properties look for the GetSQS processor?
>>>
>>> Thanks.
>>>
>>> On Fri, Nov 30, 2018 at 3:45 AM Andrew McDonald
>>> <amcdonald@ccri.com <ma...@ccri.com>> wrote:
>>>
>>> So I've installed nifi 1.8.0 and endpoint override didn't help
>>>
>>> The GetSQS is throwing and exception
>>>
>>> com.amazonaws.services.sqs.model.AmazonSQSException:
>>> Credential should be scoped to a valid region, not
>>> 'us-east-1'.
>>>
>>> I'm guessing that the override endpoint is not overriding
>>> the region b/c 'us-east-1' is not my region.
>>>
>>> Regards, Andrew
>>>
>>> On 11/29/18 02:09, Sivaprasanna wrote:
>>>> Yeah. This was added in 1.8.0 for SQS. However, the reason
>>>> why a custom enum was added on the NiFi side[1] was to have
>>>> a proper readable region instead of just the region code
>>>> i.e., Asia Pacific (Singapore) instead of ap-southeast-1.
>>>> However, I raised a request later to the AWS Java SDK team
>>>> to have a readable name. It was added recently, if I
>>>> remember correctly. So ideally this enum has to be removed
>>>> and the one on the official AWS SDK has to be leveraged
>>>> completely. I have created a Jira[2] and started working on
>>>> it. I'll raise a PR soon.
>>>>
>>>> [1] https://issues.apache.org/jira/browse/NIFI-5129
>>>> [2] https://issues.apache.org/jira/browse/NIFI-5850
>>>>
>>>> Thanks,
>>>>
>>>> On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald
>>>> <amcdonald@ccri.com <ma...@ccri.com>> wrote:
>>>>
>>>> zenfenan added this later to 1.8.0, yay!
>>>>
>>>> On 11/28/18 14:23, Andrew McDonald wrote:
>>>>>
>>>>>
>>>>> This workaround doesn't work for sqs because it
>>>>> doesn't have endpoint override URL property
>>>>>
>>>>>
>>>>> On 11/28/18 12:16, Michael Moser wrote:
>>>>>> Greetings! This JIRA ticket [1] describes the
>>>>>> recommended work around for AWS regions that aren't
>>>>>> in the list.
>>>>>>
>>>>>> -- Mike
>>>>>>
>>>>>> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan
>>>>>> <jmlogan@buffalo.edu <ma...@buffalo.edu>> wrote:
>>>>>>
>>>>>> Andrew,
>>>>>>
>>>>>> I know there's a few regions not in the list. I'm
>>>>>> not sure which region you're targeting, but at
>>>>>> least for the case of one of the new regions, I
>>>>>> submitted a PR for this. I haven't dug into it
>>>>>> deeply, but it seems like a better way to do this
>>>>>> might be to remove the enum entirely and get the
>>>>>> region list via the AWS API, or allow a free-form
>>>>>> entry.
>>>>>>
>>>>>> https://github.com/apache/nifi/pull/3187
>>>>>>
>>>>>>
>>>>>> Jon
>>>>>>
>>>>>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald
>>>>>> <amcdonald@ccri.com <ma...@ccri.com>>
>>>>>> wrote:
>>>>>>
>>>>>> I'm trying to upgrade from 1.4.0 to 1.7.1 but
>>>>>> the s3 processors can not
>>>>>> be initialized.
>>>>>>
>>>>>> Nifi 1.7.1 uses 1.11.319 and is throwing an
>>>>>> IllegalArgumentException: no
>>>>>> region provided
>>>>>>
>>>>>> The region I'm using isn't in the enum, so is
>>>>>> it possible to use a
>>>>>> custom region?
>>>>>>
>>>>>> Regards,Andrew
>>>>>>
>>>>>>
Re: How do you use a custom region?
Posted by Sivaprasanna <si...@gmail.com>.
I understand where the problem is. The reason why we replaced
*this.client.setEndpoint(urlstr);* with *this.client.setEndpoint(urlstr,
this.client.getServiceName(), this.region.getName()); *was that when
working VPC enabled services, the endpoint URL wasn't properly parsed.
Having said that, *this.region.getName()* shouldn't be there. I'll work on
a solution. Thanks for bringing this up.
-
Sivaprasanna
On Fri, Nov 30, 2018 at 9:49 PM Andrew McDonald <am...@ccri.com> wrote:
> Setting line 289 in AbstractAWSProcessor.java to
>
> this.client.setEndpoint(urlstr);
>
> worked for me.
>
> Regards,Andrew
> On 11/30/18 09:12, Andrew McDonald wrote:
>
> This is on an air-gapped system so I'll type in the essential part
>
> Failed to receive messages from Amazon SQS due to
> com.amazonaws.services.sqs.model.AmazonSQSException: Credential should be
> scoped to a valid region, not 'us-east-1'. (Service: AmazonSQS; Status
> Code: 403; Error Code: SignatureDoesNotMatch; Request ID:
> xxxxx-xxxx-xxxx-xxxx-xxxxxxxx).
>
> I believe that line 289 in AbstractAWSProcessor.java is setting the wrong
> regionId
>
> this.client.setEndpoint(urlstr, this.client.getServiceName(), this.region.getName());
>
> The amazon docs for the third parameter (regionId) state:
>
> * @param regionId* The ID of the region in which this service resides AND the* overriding region for signing purposes.
>
> This would explain the SignatureDoesNotMatch error.
>
> The properties for the sqs processor work for nifi 1.4.0.
>
> Regards, Andrew
>
>
> On 11/29/18 23:22, Sivaprasanna wrote:
>
> Andrew, Can you please share the complete error message and also show us
> how the properties look for the GetSQS processor?
>
> Thanks.
>
> On Fri, Nov 30, 2018 at 3:45 AM Andrew McDonald <am...@ccri.com>
> wrote:
>
>> So I've installed nifi 1.8.0 and endpoint override didn't help
>>
>> The GetSQS is throwing and exception
>>
>> com.amazonaws.services.sqs.model.AmazonSQSException: Credential should be
>> scoped to a valid region, not 'us-east-1'.
>>
>> I'm guessing that the override endpoint is not overriding the region b/c
>> 'us-east-1' is not my region.
>>
>> Regards, Andrew
>> On 11/29/18 02:09, Sivaprasanna wrote:
>>
>> Yeah. This was added in 1.8.0 for SQS. However, the reason why a custom
>> enum was added on the NiFi side[1] was to have a proper readable region
>> instead of just the region code i.e., Asia Pacific (Singapore) instead
>> of ap-southeast-1. However, I raised a request later to the AWS Java SDK
>> team to have a readable name. It was added recently, if I remember
>> correctly. So ideally this enum has to be removed and the one on the
>> official AWS SDK has to be leveraged completely. I have created a Jira[2]
>> and started working on it. I'll raise a PR soon.
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-5129
>> [2] https://issues.apache.org/jira/browse/NIFI-5850
>>
>> Thanks,
>>
>> On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald <am...@ccri.com>
>> wrote:
>>
>>> zenfenan added this later to 1.8.0, yay!
>>> On 11/28/18 14:23, Andrew McDonald wrote:
>>>
>>>
>>> This workaround doesn't work for sqs because it doesn't have endpoint
>>> override URL property
>>>
>>>
>>> On 11/28/18 12:16, Michael Moser wrote:
>>>
>>> Greetings! This JIRA ticket [1] describes the recommended work around
>>> for AWS regions that aren't in the list.
>>>
>>> -- Mike
>>>
>>> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>>
>>>
>>>
>>> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan <jm...@buffalo.edu> wrote:
>>>
>>>> Andrew,
>>>>
>>>> I know there's a few regions not in the list. I'm not sure which region
>>>> you're targeting, but at least for the case of one of the new regions, I
>>>> submitted a PR for this. I haven't dug into it deeply, but it seems like a
>>>> better way to do this might be to remove the enum entirely and get the
>>>> region list via the AWS API, or allow a free-form entry.
>>>>
>>>> https://github.com/apache/nifi/pull/3187
>>>>
>>>>
>>>> Jon
>>>>
>>>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald <am...@ccri.com>
>>>> wrote:
>>>>
>>>>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3 processors can
>>>>> not
>>>>> be initialized.
>>>>>
>>>>> Nifi 1.7.1 uses 1.11.319 and is throwing an IllegalArgumentException:
>>>>> no
>>>>> region provided
>>>>>
>>>>> The region I'm using isn't in the enum, so is it possible to use a
>>>>> custom region?
>>>>>
>>>>> Regards,Andrew
>>>>>
>>>>>
>>>>>
Re: How do you use a custom region?
Posted by Andrew McDonald <am...@ccri.com>.
Setting line 289 in AbstractAWSProcessor.java to
this.client.setEndpoint(urlstr);
worked for me.
Regards,Andrew
On 11/30/18 09:12, Andrew McDonald wrote:
>
> This is on an air-gapped system so I'll type in the essential part
>
> Failed to receive messages from Amazon SQS due to
> com.amazonaws.services.sqs.model.AmazonSQSException: Credential should
> be scoped to a valid region, not 'us-east-1'. (Service: AmazonSQS;
> Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID:
> xxxxx-xxxx-xxxx-xxxx-xxxxxxxx).
>
> I believe that line 289 in AbstractAWSProcessor.java is setting the
> wrong regionId
>
> this.client.setEndpoint(urlstr, this.client.getServiceName(),
> this.region.getName());
>
> The amazon docs for the third parameter (regionId) state:
>
> * @param regionId * The ID of the region in which this service resides
> AND the * overriding region for signing purposes.
>
> This would explain the SignatureDoesNotMatch error.
>
> The properties for the sqs processor work for nifi 1.4.0.
>
> Regards, Andrew
>
>
> On 11/29/18 23:22, Sivaprasanna wrote:
>> Andrew, Can you please share the complete error message and also show
>> us how the properties look for the GetSQS processor?
>>
>> Thanks.
>>
>> On Fri, Nov 30, 2018 at 3:45 AM Andrew McDonald <amcdonald@ccri.com
>> <ma...@ccri.com>> wrote:
>>
>> So I've installed nifi 1.8.0 and endpoint override didn't help
>>
>> The GetSQS is throwing and exception
>>
>> com.amazonaws.services.sqs.model.AmazonSQSException:
>> Credential should be scoped to a valid region, not 'us-east-1'.
>>
>> I'm guessing that the override endpoint is not overriding the
>> region b/c 'us-east-1' is not my region.
>>
>> Regards, Andrew
>>
>> On 11/29/18 02:09, Sivaprasanna wrote:
>>> Yeah. This was added in 1.8.0 for SQS. However, the reason why a
>>> custom enum was added on the NiFi side[1] was to have a proper
>>> readable region instead of just the region code i.e., Asia
>>> Pacific (Singapore) instead of ap-southeast-1. However, I raised
>>> a request later to the AWS Java SDK team to have a readable
>>> name. It was added recently, if I remember correctly. So ideally
>>> this enum has to be removed and the one on the official AWS SDK
>>> has to be leveraged completely. I have created a Jira[2] and
>>> started working on it. I'll raise a PR soon.
>>>
>>> [1] https://issues.apache.org/jira/browse/NIFI-5129
>>> [2] https://issues.apache.org/jira/browse/NIFI-5850
>>>
>>> Thanks,
>>>
>>> On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald
>>> <amcdonald@ccri.com <ma...@ccri.com>> wrote:
>>>
>>> zenfenan added this later to 1.8.0, yay!
>>>
>>> On 11/28/18 14:23, Andrew McDonald wrote:
>>>>
>>>>
>>>> This workaround doesn't work for sqs because it doesn't
>>>> have endpoint override URL property
>>>>
>>>>
>>>> On 11/28/18 12:16, Michael Moser wrote:
>>>>> Greetings! This JIRA ticket [1] describes the recommended
>>>>> work around for AWS regions that aren't in the list.
>>>>>
>>>>> -- Mike
>>>>>
>>>>> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan
>>>>> <jmlogan@buffalo.edu <ma...@buffalo.edu>> wrote:
>>>>>
>>>>> Andrew,
>>>>>
>>>>> I know there's a few regions not in the list. I'm not
>>>>> sure which region you're targeting, but at least for
>>>>> the case of one of the new regions, I submitted a PR
>>>>> for this. I haven't dug into it deeply, but it seems
>>>>> like a better way to do this might be to remove the
>>>>> enum entirely and get the region list via the AWS API,
>>>>> or allow a free-form entry.
>>>>>
>>>>> https://github.com/apache/nifi/pull/3187
>>>>>
>>>>>
>>>>> Jon
>>>>>
>>>>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald
>>>>> <amcdonald@ccri.com <ma...@ccri.com>> wrote:
>>>>>
>>>>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the
>>>>> s3 processors can not
>>>>> be initialized.
>>>>>
>>>>> Nifi 1.7.1 uses 1.11.319 and is throwing an
>>>>> IllegalArgumentException: no
>>>>> region provided
>>>>>
>>>>> The region I'm using isn't in the enum, so is it
>>>>> possible to use a
>>>>> custom region?
>>>>>
>>>>> Regards,Andrew
>>>>>
>>>>>
Re: How do you use a custom region?
Posted by Andrew McDonald <am...@ccri.com>.
This is on an air-gapped system so I'll type in the essential part
Failed to receive messages from Amazon SQS due to
com.amazonaws.services.sqs.model.AmazonSQSException: Credential should
be scoped to a valid region, not 'us-east-1'. (Service: AmazonSQS;
Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID:
xxxxx-xxxx-xxxx-xxxx-xxxxxxxx).
I believe that line 289 in AbstractAWSProcessor.java is setting the
wrong regionId
this.client.setEndpoint(urlstr, this.client.getServiceName(),
this.region.getName());
The amazon docs for the third parameter (regionId) state:
* @param regionId * The ID of the region in which this service resides
AND the * overriding region for signing purposes.
This would explain the SignatureDoesNotMatch error.
The properties for the sqs processor work for nifi 1.4.0.
Regards, Andrew
On 11/29/18 23:22, Sivaprasanna wrote:
> Andrew, Can you please share the complete error message and also show
> us how the properties look for the GetSQS processor?
>
> Thanks.
>
> On Fri, Nov 30, 2018 at 3:45 AM Andrew McDonald <amcdonald@ccri.com
> <ma...@ccri.com>> wrote:
>
> So I've installed nifi 1.8.0 and endpoint override didn't help
>
> The GetSQS is throwing and exception
>
> com.amazonaws.services.sqs.model.AmazonSQSException:
> Credential should be scoped to a valid region, not 'us-east-1'.
>
> I'm guessing that the override endpoint is not overriding the
> region b/c 'us-east-1' is not my region.
>
> Regards, Andrew
>
> On 11/29/18 02:09, Sivaprasanna wrote:
>> Yeah. This was added in 1.8.0 for SQS. However, the reason why a
>> custom enum was added on the NiFi side[1] was to have a proper
>> readable region instead of just the region code i.e., Asia
>> Pacific (Singapore) instead of ap-southeast-1. However, I raised
>> a request later to the AWS Java SDK team to have a readable name.
>> It was added recently, if I remember correctly. So ideally this
>> enum has to be removed and the one on the official AWS SDK has to
>> be leveraged completely. I have created a Jira[2] and started
>> working on it. I'll raise a PR soon.
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-5129
>> [2] https://issues.apache.org/jira/browse/NIFI-5850
>>
>> Thanks,
>>
>> On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald
>> <amcdonald@ccri.com <ma...@ccri.com>> wrote:
>>
>> zenfenan added this later to 1.8.0, yay!
>>
>> On 11/28/18 14:23, Andrew McDonald wrote:
>>>
>>>
>>> This workaround doesn't work for sqs because it doesn't have
>>> endpoint override URL property
>>>
>>>
>>> On 11/28/18 12:16, Michael Moser wrote:
>>>> Greetings! This JIRA ticket [1] describes the recommended
>>>> work around for AWS regions that aren't in the list.
>>>>
>>>> -- Mike
>>>>
>>>> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>>>
>>>>
>>>>
>>>> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan
>>>> <jmlogan@buffalo.edu <ma...@buffalo.edu>> wrote:
>>>>
>>>> Andrew,
>>>>
>>>> I know there's a few regions not in the list. I'm not
>>>> sure which region you're targeting, but at least for
>>>> the case of one of the new regions, I submitted a PR
>>>> for this. I haven't dug into it deeply, but it seems
>>>> like a better way to do this might be to remove the
>>>> enum entirely and get the region list via the AWS API,
>>>> or allow a free-form entry.
>>>>
>>>> https://github.com/apache/nifi/pull/3187
>>>>
>>>>
>>>> Jon
>>>>
>>>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald
>>>> <amcdonald@ccri.com <ma...@ccri.com>> wrote:
>>>>
>>>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the
>>>> s3 processors can not
>>>> be initialized.
>>>>
>>>> Nifi 1.7.1 uses 1.11.319 and is throwing an
>>>> IllegalArgumentException: no
>>>> region provided
>>>>
>>>> The region I'm using isn't in the enum, so is it
>>>> possible to use a
>>>> custom region?
>>>>
>>>> Regards,Andrew
>>>>
>>>>
Re: How do you use a custom region?
Posted by Sivaprasanna <si...@gmail.com>.
Andrew, Can you please share the complete error message and also show us
how the properties look for the GetSQS processor?
Thanks.
On Fri, Nov 30, 2018 at 3:45 AM Andrew McDonald <am...@ccri.com> wrote:
> So I've installed nifi 1.8.0 and endpoint override didn't help
>
> The GetSQS is throwing and exception
>
> com.amazonaws.services.sqs.model.AmazonSQSException: Credential should be
> scoped to a valid region, not 'us-east-1'.
>
> I'm guessing that the override endpoint is not overriding the region b/c
> 'us-east-1' is not my region.
>
> Regards, Andrew
> On 11/29/18 02:09, Sivaprasanna wrote:
>
> Yeah. This was added in 1.8.0 for SQS. However, the reason why a custom
> enum was added on the NiFi side[1] was to have a proper readable region
> instead of just the region code i.e., Asia Pacific (Singapore) instead of
> ap-southeast-1. However, I raised a request later to the AWS Java SDK
> team to have a readable name. It was added recently, if I remember
> correctly. So ideally this enum has to be removed and the one on the
> official AWS SDK has to be leveraged completely. I have created a Jira[2]
> and started working on it. I'll raise a PR soon.
>
> [1] https://issues.apache.org/jira/browse/NIFI-5129
> [2] https://issues.apache.org/jira/browse/NIFI-5850
>
> Thanks,
>
> On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald <am...@ccri.com>
> wrote:
>
>> zenfenan added this later to 1.8.0, yay!
>> On 11/28/18 14:23, Andrew McDonald wrote:
>>
>>
>> This workaround doesn't work for sqs because it doesn't have endpoint
>> override URL property
>>
>>
>> On 11/28/18 12:16, Michael Moser wrote:
>>
>> Greetings! This JIRA ticket [1] describes the recommended work around
>> for AWS regions that aren't in the list.
>>
>> -- Mike
>>
>> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>
>>
>>
>> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan <jm...@buffalo.edu> wrote:
>>
>>> Andrew,
>>>
>>> I know there's a few regions not in the list. I'm not sure which region
>>> you're targeting, but at least for the case of one of the new regions, I
>>> submitted a PR for this. I haven't dug into it deeply, but it seems like a
>>> better way to do this might be to remove the enum entirely and get the
>>> region list via the AWS API, or allow a free-form entry.
>>>
>>> https://github.com/apache/nifi/pull/3187
>>>
>>>
>>> Jon
>>>
>>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald <am...@ccri.com>
>>> wrote:
>>>
>>>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3 processors can not
>>>> be initialized.
>>>>
>>>> Nifi 1.7.1 uses 1.11.319 and is throwing an IllegalArgumentException:
>>>> no
>>>> region provided
>>>>
>>>> The region I'm using isn't in the enum, so is it possible to use a
>>>> custom region?
>>>>
>>>> Regards,Andrew
>>>>
>>>>
>>>>
Re: How do you use a custom region?
Posted by Andrew McDonald <am...@ccri.com>.
So I've installed nifi 1.8.0 and endpoint override didn't help
The GetSQS is throwing and exception
com.amazonaws.services.sqs.model.AmazonSQSException: Credential
should be scoped to a valid region, not 'us-east-1'.
I'm guessing that the override endpoint is not overriding the region b/c
'us-east-1' is not my region.
Regards, Andrew
On 11/29/18 02:09, Sivaprasanna wrote:
> Yeah. This was added in 1.8.0 for SQS. However, the reason why a
> custom enum was added on the NiFi side[1] was to have a proper
> readable region instead of just the region code i.e., Asia Pacific
> (Singapore) instead of ap-southeast-1. However, I raised a request
> later to the AWS Java SDK team to have a readable name. It was added
> recently, if I remember correctly. So ideally this enum has to be
> removed and the one on the official AWS SDK has to be leveraged
> completely. I have created a Jira[2] and started working on it. I'll
> raise a PR soon.
>
> [1] https://issues.apache.org/jira/browse/NIFI-5129
> [2] https://issues.apache.org/jira/browse/NIFI-5850
>
> Thanks,
>
> On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald <amcdonald@ccri.com
> <ma...@ccri.com>> wrote:
>
> zenfenan added this later to 1.8.0, yay!
>
> On 11/28/18 14:23, Andrew McDonald wrote:
>>
>>
>> This workaround doesn't work for sqs because it doesn't have
>> endpoint override URL property
>>
>>
>> On 11/28/18 12:16, Michael Moser wrote:
>>> Greetings! This JIRA ticket [1] describes the recommended work
>>> around for AWS regions that aren't in the list.
>>>
>>> -- Mike
>>>
>>> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>>
>>>
>>>
>>> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan <jmlogan@buffalo.edu
>>> <ma...@buffalo.edu>> wrote:
>>>
>>> Andrew,
>>>
>>> I know there's a few regions not in the list. I'm not sure
>>> which region you're targeting, but at least for the case of
>>> one of the new regions, I submitted a PR for this. I haven't
>>> dug into it deeply, but it seems like a better way to do
>>> this might be to remove the enum entirely and get the region
>>> list via the AWS API, or allow a free-form entry.
>>>
>>> https://github.com/apache/nifi/pull/3187
>>>
>>>
>>> Jon
>>>
>>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald
>>> <amcdonald@ccri.com <ma...@ccri.com>> wrote:
>>>
>>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3
>>> processors can not
>>> be initialized.
>>>
>>> Nifi 1.7.1 uses 1.11.319 and is throwing an
>>> IllegalArgumentException: no
>>> region provided
>>>
>>> The region I'm using isn't in the enum, so is it
>>> possible to use a
>>> custom region?
>>>
>>> Regards,Andrew
>>>
>>>
Re: How do you use a custom region?
Posted by Sivaprasanna <si...@gmail.com>.
Yeah. This was added in 1.8.0 for SQS. However, the reason why a custom
enum was added on the NiFi side[1] was to have a proper readable region
instead of just the region code i.e., Asia Pacific (Singapore) instead of ap
-southeast-1. However, I raised a request later to the AWS Java SDK team to
have a readable name. It was added recently, if I remember correctly. So
ideally this enum has to be removed and the one on the official AWS SDK has
to be leveraged completely. I have created a Jira[2] and started working on
it. I'll raise a PR soon.
[1] https://issues.apache.org/jira/browse/NIFI-5129
[2] https://issues.apache.org/jira/browse/NIFI-5850
Thanks,
On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald <am...@ccri.com> wrote:
> zenfenan added this later to 1.8.0, yay!
> On 11/28/18 14:23, Andrew McDonald wrote:
>
>
> This workaround doesn't work for sqs because it doesn't have endpoint
> override URL property
>
>
> On 11/28/18 12:16, Michael Moser wrote:
>
> Greetings! This JIRA ticket [1] describes the recommended work around for
> AWS regions that aren't in the list.
>
> -- Mike
>
> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>
>
>
> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan <jm...@buffalo.edu> wrote:
>
>> Andrew,
>>
>> I know there's a few regions not in the list. I'm not sure which region
>> you're targeting, but at least for the case of one of the new regions, I
>> submitted a PR for this. I haven't dug into it deeply, but it seems like a
>> better way to do this might be to remove the enum entirely and get the
>> region list via the AWS API, or allow a free-form entry.
>>
>> https://github.com/apache/nifi/pull/3187
>>
>>
>> Jon
>>
>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald <am...@ccri.com>
>> wrote:
>>
>>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3 processors can not
>>> be initialized.
>>>
>>> Nifi 1.7.1 uses 1.11.319 and is throwing an IllegalArgumentException: no
>>> region provided
>>>
>>> The region I'm using isn't in the enum, so is it possible to use a
>>> custom region?
>>>
>>> Regards,Andrew
>>>
>>>
>>>
Re: How do you use a custom region?
Posted by Andrew McDonald <am...@ccri.com>.
zenfenan added this later to 1.8.0, yay!
On 11/28/18 14:23, Andrew McDonald wrote:
>
>
> This workaround doesn't work for sqs because it doesn't have endpoint
> override URL property
>
>
> On 11/28/18 12:16, Michael Moser wrote:
>> Greetings! This JIRA ticket [1] describes the recommended work
>> around for AWS regions that aren't in the list.
>>
>> -- Mike
>>
>> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>
>>
>>
>> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan <jmlogan@buffalo.edu
>> <ma...@buffalo.edu>> wrote:
>>
>> Andrew,
>>
>> I know there's a few regions not in the list. I'm not sure which
>> region you're targeting, but at least for the case of one of the
>> new regions, I submitted a PR for this. I haven't dug into it
>> deeply, but it seems like a better way to do this might be to
>> remove the enum entirely and get the region list via the AWS API,
>> or allow a free-form entry.
>>
>> https://github.com/apache/nifi/pull/3187
>>
>>
>> Jon
>>
>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald
>> <amcdonald@ccri.com <ma...@ccri.com>> wrote:
>>
>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3
>> processors can not
>> be initialized.
>>
>> Nifi 1.7.1 uses 1.11.319 and is throwing an
>> IllegalArgumentException: no
>> region provided
>>
>> The region I'm using isn't in the enum, so is it possible to
>> use a
>> custom region?
>>
>> Regards,Andrew
>>
>>
Re: How do you use a custom region?
Posted by Andrew McDonald <am...@ccri.com>.
This workaround doesn't work for sqs because it doesn't have endpoint
override URL property
On 11/28/18 12:16, Michael Moser wrote:
> Greetings! This JIRA ticket [1] describes the recommended work around
> for AWS regions that aren't in the list.
>
> -- Mike
>
> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>
>
>
> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan <jmlogan@buffalo.edu
> <ma...@buffalo.edu>> wrote:
>
> Andrew,
>
> I know there's a few regions not in the list. I'm not sure which
> region you're targeting, but at least for the case of one of the
> new regions, I submitted a PR for this. I haven't dug into it
> deeply, but it seems like a better way to do this might be to
> remove the enum entirely and get the region list via the AWS API,
> or allow a free-form entry.
>
> https://github.com/apache/nifi/pull/3187
>
>
> Jon
>
> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald
> <amcdonald@ccri.com <ma...@ccri.com>> wrote:
>
> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3
> processors can not
> be initialized.
>
> Nifi 1.7.1 uses 1.11.319 and is throwing an
> IllegalArgumentException: no
> region provided
>
> The region I'm using isn't in the enum, so is it possible to
> use a
> custom region?
>
> Regards,Andrew
>
>
Re: How do you use a custom region?
Posted by Michael Moser <mo...@gmail.com>.
Greetings! This JIRA ticket [1] describes the recommended work around for
AWS regions that aren't in the list.
-- Mike
[1] - https://issues.apache.org/jira/browse/NIFI-4523
On Wed, Nov 28, 2018 at 11:44 AM Jon Logan <jm...@buffalo.edu> wrote:
> Andrew,
>
> I know there's a few regions not in the list. I'm not sure which region
> you're targeting, but at least for the case of one of the new regions, I
> submitted a PR for this. I haven't dug into it deeply, but it seems like a
> better way to do this might be to remove the enum entirely and get the
> region list via the AWS API, or allow a free-form entry.
>
> https://github.com/apache/nifi/pull/3187
>
>
> Jon
>
> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald <am...@ccri.com>
> wrote:
>
>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3 processors can not
>> be initialized.
>>
>> Nifi 1.7.1 uses 1.11.319 and is throwing an IllegalArgumentException: no
>> region provided
>>
>> The region I'm using isn't in the enum, so is it possible to use a
>> custom region?
>>
>> Regards,Andrew
>>
>>
>>
Re: How do you use a custom region?
Posted by Jon Logan <jm...@buffalo.edu>.
Andrew,
I know there's a few regions not in the list. I'm not sure which region
you're targeting, but at least for the case of one of the new regions, I
submitted a PR for this. I haven't dug into it deeply, but it seems like a
better way to do this might be to remove the enum entirely and get the
region list via the AWS API, or allow a free-form entry.
https://github.com/apache/nifi/pull/3187
Jon
On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald <am...@ccri.com> wrote:
> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3 processors can not
> be initialized.
>
> Nifi 1.7.1 uses 1.11.319 and is throwing an IllegalArgumentException: no
> region provided
>
> The region I'm using isn't in the enum, so is it possible to use a
> custom region?
>
> Regards,Andrew
>
>
>