You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Vijay Chhipa <vc...@apple.com> on 2019/05/14 17:48:55 UTC
Getting the value of a variable defined in a processor group
Hello
I am writing a new NiFi processor. In the source code I would like to use a variable defined in the NiFi processor group using the UI.
Is there a way to get the map of all the variables available to this processor?
I am on NiFi 1.9.1
Thanks
Re: Getting the value of a variable defined in a processor group
Posted by Vijay Chhipa <vc...@apple.com>.
Bryan, Andy
Thanks for the reply,
I could use the PropertyDescriptor and have the user enter a value using EL.
My use case is this:
I am constructing a topic name, to post data to.
Part of the topic name, I would the user to enter in a field. But the other part I would like to pick up from a variable without having the user worry about it.
Not a blocker issue, but would be nice.
Thanks again.
> On May 14, 2019, at 1:17 PM, Bryan Bende <bb...@gmail.com> wrote:
>
> The intent is to get a variable through a property of a component
> using expression language. So you define a PropertyDescriptor that
> supports EL like:
>
> public static final PropertyDescriptor DIRECTORY = new
> PropertyDescriptor.Builder()
> .name("Directory")
> .description("The HDFS directory from which files should be read")
> .required(true)
> .addValidator(StandardValidators.ATTRIBUTE_EXPRESSION_LANGUAGE_VALIDATOR)
> .expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
> .build();
>
> Then you get the value and evaluate the expressions:
>
> context.getProperty(DIRECTORY).evaluateAttributeExpressions(flow).getValue()
>
> This will automatically substitute in any variables, so if the user
> entered ${directory} and there is a variable "directory", it will be
> substituted in. The order of precedence when evaluating expression
> language is the following:
>
> 1) flow file attributes
> 2) variable registry at process group level
> 3) variable registry properties files
> 4) system properties
> 5) environment variables
>
> So if there was a flow file attribute named "directory" it would take
> precedence over a process group variable named "directory".
>
> Some properties don't support EL from flow file attributes and only
> have expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
> and then don't pass in a flow file when evaluating:
>
> context.getProperty(DIRECTORY).evaluateAttributeExpressions().getValue()
>
> Is there a good use case for getting a variable value without having a
> PropertyDescriptor?
>
> On Tue, May 14, 2019 at 2:05 PM Andy LoPresto <al...@apache.org> wrote:
>>
>> Vijay,
>>
>> I don’t believe there is a good way to get a complete list of all variables within processor code. If you want to explore this further, you can take a look at StandardProcessContext [1], which processors have access to in their #onTrigger() method, and MutableVariableRegistry [2], which is the only concrete implementation of the ComponentVariableRegistry interface which exposes a variable map (the StandardComponentVariableRegistry returns an empty map when accessed). The issue is that the context does not expose the ProcessorNode field, which is the intermediary to the variable registry. I believe this is by design. There may be a valid use case to provide an accessor method on the context to return a variable map, or at least the entry set.
>>
>> I’d like to hear from Matt Gilman, Bryan Bende, and/or Mark Payne with their thoughts, since much of this code was written by them.
>>
>> [1] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/processor/StandardProcessContext.java#L87
>> [2] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/registry/variable/MutableVariableRegistry.java#L27
>>
>>
>> Andy LoPresto
>> alopresto@apache.org
>> alopresto.apache@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>>
>> On May 14, 2019, at 10:48 AM, Vijay Chhipa <vc...@apple.com> wrote:
>>
>> Hello
>>
>> I am writing a new NiFi processor. In the source code I would like to use a variable defined in the NiFi processor group using the UI.
>> Is there a way to get the map of all the variables available to this processor?
>>
>>
>> I am on NiFi 1.9.1
>>
>>
>> Thanks
>>
>>
>>
>>
Re: Getting the value of a variable defined in a processor group
Posted by Bryan Bende <bb...@gmail.com>.
The intent is to get a variable through a property of a component
using expression language. So you define a PropertyDescriptor that
supports EL like:
public static final PropertyDescriptor DIRECTORY = new
PropertyDescriptor.Builder()
.name("Directory")
.description("The HDFS directory from which files should be read")
.required(true)
.addValidator(StandardValidators.ATTRIBUTE_EXPRESSION_LANGUAGE_VALIDATOR)
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
.build();
Then you get the value and evaluate the expressions:
context.getProperty(DIRECTORY).evaluateAttributeExpressions(flow).getValue()
This will automatically substitute in any variables, so if the user
entered ${directory} and there is a variable "directory", it will be
substituted in. The order of precedence when evaluating expression
language is the following:
1) flow file attributes
2) variable registry at process group level
3) variable registry properties files
4) system properties
5) environment variables
So if there was a flow file attribute named "directory" it would take
precedence over a process group variable named "directory".
Some properties don't support EL from flow file attributes and only
have expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
and then don't pass in a flow file when evaluating:
context.getProperty(DIRECTORY).evaluateAttributeExpressions().getValue()
Is there a good use case for getting a variable value without having a
PropertyDescriptor?
On Tue, May 14, 2019 at 2:05 PM Andy LoPresto <al...@apache.org> wrote:
>
> Vijay,
>
> I don’t believe there is a good way to get a complete list of all variables within processor code. If you want to explore this further, you can take a look at StandardProcessContext [1], which processors have access to in their #onTrigger() method, and MutableVariableRegistry [2], which is the only concrete implementation of the ComponentVariableRegistry interface which exposes a variable map (the StandardComponentVariableRegistry returns an empty map when accessed). The issue is that the context does not expose the ProcessorNode field, which is the intermediary to the variable registry. I believe this is by design. There may be a valid use case to provide an accessor method on the context to return a variable map, or at least the entry set.
>
> I’d like to hear from Matt Gilman, Bryan Bende, and/or Mark Payne with their thoughts, since much of this code was written by them.
>
> [1] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/processor/StandardProcessContext.java#L87
> [2] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/registry/variable/MutableVariableRegistry.java#L27
>
>
> Andy LoPresto
> alopresto@apache.org
> alopresto.apache@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On May 14, 2019, at 10:48 AM, Vijay Chhipa <vc...@apple.com> wrote:
>
> Hello
>
> I am writing a new NiFi processor. In the source code I would like to use a variable defined in the NiFi processor group using the UI.
> Is there a way to get the map of all the variables available to this processor?
>
>
> I am on NiFi 1.9.1
>
>
> Thanks
>
>
>
>
Re: Getting the value of a variable defined in a processor group
Posted by Andy LoPresto <al...@apache.org>.
Vijay,
I don’t believe there is a good way to get a complete list of all variables within processor code. If you want to explore this further, you can take a look at StandardProcessContext [1], which processors have access to in their #onTrigger() method, and MutableVariableRegistry [2], which is the only concrete implementation of the ComponentVariableRegistry interface which exposes a variable map (the StandardComponentVariableRegistry returns an empty map when accessed). The issue is that the context does not expose the ProcessorNode field, which is the intermediary to the variable registry. I believe this is by design. There may be a valid use case to provide an accessor method on the context to return a variable map, or at least the entry set.
I’d like to hear from Matt Gilman, Bryan Bende, and/or Mark Payne with their thoughts, since much of this code was written by them.
[1] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/processor/StandardProcessContext.java#L87
[2] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/registry/variable/MutableVariableRegistry.java#L27 <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/registry/variable/MutableVariableRegistry.java#L27>
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On May 14, 2019, at 10:48 AM, Vijay Chhipa <vc...@apple.com> wrote:
>
> Hello
>
> I am writing a new NiFi processor. In the source code I would like to use a variable defined in the NiFi processor group using the UI.
> Is there a way to get the map of all the variables available to this processor?
>
>
> I am on NiFi 1.9.1
>
>
> Thanks
>
>
>