You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by "Gresock, Joseph" <jo...@lmco.com> on 2016/07/13 12:09:15 UTC
GetSQS variable queue
Hi folks,
I have a production and a staging SQS queue, and I was hoping I could have a single GetSQS processor configured with EL using a system property from bootstrap.conf. Is it possible to allow GetSQS Queue URL to support EL, or would this break the design of the processor?
If this is not possible, are there any suggestions on configuring a queue name per environment? In the past, I've scripted string replacements in the flow.tar prior to deployment, but I was hoping to use something more configuration-based.
Thanks,
Joe Gresock
Lockheed Martin Software Engineer Stf
AMPERE LIZARDSPOCK TTO
443-294-2661
joseph.gresock@lmco.com
Re: EXTERNAL: Re: GetSQS variable queue
Posted by Mark Payne <ma...@hotmail.com>.
Joe,
The only difference would be when you call context.getProperty(QUEUE_URL).evaluateAttributeExpressions() you don't pass any FlowFile to evaluateAttributeExpressions(). This will evaluate only the variables in system/env properties and variable registry.
Thanks
-Mark
> On Jul 13, 2016, at 10:14 AM, Gresock, Joseph <jo...@lmco.com> wrote:
>
> Thanks Mark. I've created a ticket and picked it up, but I have another question. So far I've only ever evaluated EL on a flow file, but in GetSQS there is no flow file when the queue URL is read from the property. What is the recommended way to evaluate an EL expression using just the variables from the variable registry?
>
> Thanks,
>
> Joe Gresock
> Lockheed Martin Software Engineer Stf
> AMPERE LIZARDSPOCK TTO
> 443-294-2661
> joseph.gresock@lmco.com
>
> ________________________________________
> From: Mark Payne [markap14@hotmail.com]
> Sent: Wednesday, July 13, 2016 8:40 AM
> To: dev@nifi.apache.org
> Subject: EXTERNAL: Re: GetSQS variable queue
>
> Hey Joe -
>
> I don't think there is any specific reason that GetSQS could not support Expression Language
> (limited to environment variables, system variables). It would require a code change to the processor,
> but I don't believe it would break the design.
>
> Also of note, in 0.7.0 we have an initial implementation of a Variable Registry, which would allow
> you to configure in your nifi.properties file a list of Variable Registry properties files, specifically
> for this type of purpose. I.e., in nifi.properties, you could set the "nifi.variable.registry.properties"
> property to say "aws-config.properties" and then you can have a production "aws-config.properties"
> file that has the variables that you want to use and a staging "aws-config.properties" that has
> the variables that you want to use. Hopefully I explained this clearly - It's just a bit cleaner than
> having to add system properties to the bootstrap.conf file.
>
> Thanks
> -Mark
>
>
>> On Jul 13, 2016, at 8:09 AM, Gresock, Joseph <jo...@lmco.com> wrote:
>>
>> Hi folks,
>>
>> I have a production and a staging SQS queue, and I was hoping I could have a single GetSQS processor configured with EL using a system property from bootstrap.conf. Is it possible to allow GetSQS Queue URL to support EL, or would this break the design of the processor?
>>
>> If this is not possible, are there any suggestions on configuring a queue name per environment? In the past, I've scripted string replacements in the flow.tar prior to deployment, but I was hoping to use something more configuration-based.
>>
>> Thanks,
>>
>> Joe Gresock
>> Lockheed Martin Software Engineer Stf
>> AMPERE LIZARDSPOCK TTO
>> 443-294-2661
>> joseph.gresock@lmco.com
>
RE: EXTERNAL: Re: GetSQS variable queue
Posted by "Gresock, Joseph" <jo...@lmco.com>.
Thanks Mark. I've created a ticket and picked it up, but I have another question. So far I've only ever evaluated EL on a flow file, but in GetSQS there is no flow file when the queue URL is read from the property. What is the recommended way to evaluate an EL expression using just the variables from the variable registry?
Thanks,
Joe Gresock
Lockheed Martin Software Engineer Stf
AMPERE LIZARDSPOCK TTO
443-294-2661
joseph.gresock@lmco.com
________________________________________
From: Mark Payne [markap14@hotmail.com]
Sent: Wednesday, July 13, 2016 8:40 AM
To: dev@nifi.apache.org
Subject: EXTERNAL: Re: GetSQS variable queue
Hey Joe -
I don't think there is any specific reason that GetSQS could not support Expression Language
(limited to environment variables, system variables). It would require a code change to the processor,
but I don't believe it would break the design.
Also of note, in 0.7.0 we have an initial implementation of a Variable Registry, which would allow
you to configure in your nifi.properties file a list of Variable Registry properties files, specifically
for this type of purpose. I.e., in nifi.properties, you could set the "nifi.variable.registry.properties"
property to say "aws-config.properties" and then you can have a production "aws-config.properties"
file that has the variables that you want to use and a staging "aws-config.properties" that has
the variables that you want to use. Hopefully I explained this clearly - It's just a bit cleaner than
having to add system properties to the bootstrap.conf file.
Thanks
-Mark
> On Jul 13, 2016, at 8:09 AM, Gresock, Joseph <jo...@lmco.com> wrote:
>
> Hi folks,
>
> I have a production and a staging SQS queue, and I was hoping I could have a single GetSQS processor configured with EL using a system property from bootstrap.conf. Is it possible to allow GetSQS Queue URL to support EL, or would this break the design of the processor?
>
> If this is not possible, are there any suggestions on configuring a queue name per environment? In the past, I've scripted string replacements in the flow.tar prior to deployment, but I was hoping to use something more configuration-based.
>
> Thanks,
>
> Joe Gresock
> Lockheed Martin Software Engineer Stf
> AMPERE LIZARDSPOCK TTO
> 443-294-2661
> joseph.gresock@lmco.com
Re: GetSQS variable queue
Posted by Mark Payne <ma...@hotmail.com>.
Hey Joe -
I don't think there is any specific reason that GetSQS could not support Expression Language
(limited to environment variables, system variables). It would require a code change to the processor,
but I don't believe it would break the design.
Also of note, in 0.7.0 we have an initial implementation of a Variable Registry, which would allow
you to configure in your nifi.properties file a list of Variable Registry properties files, specifically
for this type of purpose. I.e., in nifi.properties, you could set the "nifi.variable.registry.properties"
property to say "aws-config.properties" and then you can have a production "aws-config.properties"
file that has the variables that you want to use and a staging "aws-config.properties" that has
the variables that you want to use. Hopefully I explained this clearly - It's just a bit cleaner than
having to add system properties to the bootstrap.conf file.
Thanks
-Mark
> On Jul 13, 2016, at 8:09 AM, Gresock, Joseph <jo...@lmco.com> wrote:
>
> Hi folks,
>
> I have a production and a staging SQS queue, and I was hoping I could have a single GetSQS processor configured with EL using a system property from bootstrap.conf. Is it possible to allow GetSQS Queue URL to support EL, or would this break the design of the processor?
>
> If this is not possible, are there any suggestions on configuring a queue name per environment? In the past, I've scripted string replacements in the flow.tar prior to deployment, but I was hoping to use something more configuration-based.
>
> Thanks,
>
> Joe Gresock
> Lockheed Martin Software Engineer Stf
> AMPERE LIZARDSPOCK TTO
> 443-294-2661
> joseph.gresock@lmco.com