You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by "Peter Wicks (pwicks)" <pw...@micron.com> on 2016/08/15 19:22:15 UTC
Externally update processor State
I'm looking for a way to externally update a processors state, preferably through an API call and then at some point the existing State viewer with the clear button. From digging through the code base I couldn't find an exposed API that would allow for state modification; only a state clearing.
Was there a design decision behind this, or is this a feature waiting to be added?
Regards,
Peter
Re: Externally update processor State
Posted by Matt Burgess <ma...@gmail.com>.
Peter/Joe,
Updating an "Initial Max Value" property could set the state when the
processor is started, so you wouldn't need to clear state unless your
Initial Max Value is less than what is in state currently.
Regards,
Matt
On Mon, Aug 15, 2016 at 4:16 PM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
> Joe,
>
> So the solution would is to customize QueryDatabaseTable to support reading the initial state Max Value(s) from a property. Then, if I want to change the existing state I stop the processor, clear the state, update my initial Max Value, and start the processor again?
>
> --Peter
>
> -----Original Message-----
> From: Joe Witt [mailto:joe.witt@gmail.com]
> Sent: Monday, August 15, 2016 2:06 PM
> To: dev@nifi.apache.org
> Subject: Re: Externally update processor State
>
> Peter,
>
> Ok understood. I feel like the QueryDatabaseTable processor should have a property for exactly this. And there is already a clear and well managed model and REST API for operating on those. Does that sound like a fair alternative?
>
> Thanks
> JOe
>
> On Mon, Aug 15, 2016 at 1:03 PM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
>> Joe,
>>
>> The specific use cases at the moment both surround QueryDatabaseTable and its Max Value Column tracking.
>>
>> First, I'd like to be able to set an initial value for a max value column; so essentially an initial state value. The idea would be to support situations where we want to start loading data only from the last 'x' time period.
>>
>> Secondly, if something happened downstream and we need to roll back the clock I want our operations team to be able to stop the processor, change the timestamp to the necessary value, and then start the processor back up as part of their resolution process.
>>
>> Regards,
>> Peter
>>
>> -----Original Message-----
>> From: Joe Witt [mailto:joe.witt@gmail.com]
>> Sent: Monday, August 15, 2016 1:51 PM
>> To: dev@nifi.apache.org
>> Subject: Re: Externally update processor State
>>
>> Peter,
>>
>> The state management function the is built-in is to allow processors to set/interact with their own managed state. We did not intend for that to be something someone external to the processor could interact with directly. Rather, that is what properties are expected to be used for.
>>
>> So far the couple of requests like this I've seen are where there is a sort of human in the process flow going on - like a workflow manager.
>> Can you describe the details of your use case/ideal user experience a bit further?
>>
>> Thanks
>> Joe
>>
>> On Mon, Aug 15, 2016 at 12:22 PM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
>>> I'm looking for a way to externally update a processors state, preferably through an API call and then at some point the existing State viewer with the clear button. From digging through the code base I couldn't find an exposed API that would allow for state modification; only a state clearing.
>>> Was there a design decision behind this, or is this a feature waiting to be added?
>>>
>>> Regards,
>>> Peter
RE: Externally update processor State
Posted by "Peter Wicks (pwicks)" <pw...@micron.com>.
Joe,
So the solution would is to customize QueryDatabaseTable to support reading the initial state Max Value(s) from a property. Then, if I want to change the existing state I stop the processor, clear the state, update my initial Max Value, and start the processor again?
--Peter
-----Original Message-----
From: Joe Witt [mailto:joe.witt@gmail.com]
Sent: Monday, August 15, 2016 2:06 PM
To: dev@nifi.apache.org
Subject: Re: Externally update processor State
Peter,
Ok understood. I feel like the QueryDatabaseTable processor should have a property for exactly this. And there is already a clear and well managed model and REST API for operating on those. Does that sound like a fair alternative?
Thanks
JOe
On Mon, Aug 15, 2016 at 1:03 PM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
> Joe,
>
> The specific use cases at the moment both surround QueryDatabaseTable and its Max Value Column tracking.
>
> First, I'd like to be able to set an initial value for a max value column; so essentially an initial state value. The idea would be to support situations where we want to start loading data only from the last 'x' time period.
>
> Secondly, if something happened downstream and we need to roll back the clock I want our operations team to be able to stop the processor, change the timestamp to the necessary value, and then start the processor back up as part of their resolution process.
>
> Regards,
> Peter
>
> -----Original Message-----
> From: Joe Witt [mailto:joe.witt@gmail.com]
> Sent: Monday, August 15, 2016 1:51 PM
> To: dev@nifi.apache.org
> Subject: Re: Externally update processor State
>
> Peter,
>
> The state management function the is built-in is to allow processors to set/interact with their own managed state. We did not intend for that to be something someone external to the processor could interact with directly. Rather, that is what properties are expected to be used for.
>
> So far the couple of requests like this I've seen are where there is a sort of human in the process flow going on - like a workflow manager.
> Can you describe the details of your use case/ideal user experience a bit further?
>
> Thanks
> Joe
>
> On Mon, Aug 15, 2016 at 12:22 PM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
>> I'm looking for a way to externally update a processors state, preferably through an API call and then at some point the existing State viewer with the clear button. From digging through the code base I couldn't find an exposed API that would allow for state modification; only a state clearing.
>> Was there a design decision behind this, or is this a feature waiting to be added?
>>
>> Regards,
>> Peter
Re: Externally update processor State
Posted by Joe Witt <jo...@gmail.com>.
Peter,
Ok understood. I feel like the QueryDatabaseTable processor should
have a property for exactly this. And there is already a clear and
well managed model and REST API for operating on those. Does that
sound like a fair alternative?
Thanks
JOe
On Mon, Aug 15, 2016 at 1:03 PM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
> Joe,
>
> The specific use cases at the moment both surround QueryDatabaseTable and its Max Value Column tracking.
>
> First, I'd like to be able to set an initial value for a max value column; so essentially an initial state value. The idea would be to support situations where we want to start loading data only from the last 'x' time period.
>
> Secondly, if something happened downstream and we need to roll back the clock I want our operations team to be able to stop the processor, change the timestamp to the necessary value, and then start the processor back up as part of their resolution process.
>
> Regards,
> Peter
>
> -----Original Message-----
> From: Joe Witt [mailto:joe.witt@gmail.com]
> Sent: Monday, August 15, 2016 1:51 PM
> To: dev@nifi.apache.org
> Subject: Re: Externally update processor State
>
> Peter,
>
> The state management function the is built-in is to allow processors to set/interact with their own managed state. We did not intend for that to be something someone external to the processor could interact with directly. Rather, that is what properties are expected to be used for.
>
> So far the couple of requests like this I've seen are where there is a sort of human in the process flow going on - like a workflow manager.
> Can you describe the details of your use case/ideal user experience a bit further?
>
> Thanks
> Joe
>
> On Mon, Aug 15, 2016 at 12:22 PM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
>> I'm looking for a way to externally update a processors state, preferably through an API call and then at some point the existing State viewer with the clear button. From digging through the code base I couldn't find an exposed API that would allow for state modification; only a state clearing.
>> Was there a design decision behind this, or is this a feature waiting to be added?
>>
>> Regards,
>> Peter
RE: Externally update processor State
Posted by "Peter Wicks (pwicks)" <pw...@micron.com>.
Joe,
The specific use cases at the moment both surround QueryDatabaseTable and its Max Value Column tracking.
First, I'd like to be able to set an initial value for a max value column; so essentially an initial state value. The idea would be to support situations where we want to start loading data only from the last 'x' time period.
Secondly, if something happened downstream and we need to roll back the clock I want our operations team to be able to stop the processor, change the timestamp to the necessary value, and then start the processor back up as part of their resolution process.
Regards,
Peter
-----Original Message-----
From: Joe Witt [mailto:joe.witt@gmail.com]
Sent: Monday, August 15, 2016 1:51 PM
To: dev@nifi.apache.org
Subject: Re: Externally update processor State
Peter,
The state management function the is built-in is to allow processors to set/interact with their own managed state. We did not intend for that to be something someone external to the processor could interact with directly. Rather, that is what properties are expected to be used for.
So far the couple of requests like this I've seen are where there is a sort of human in the process flow going on - like a workflow manager.
Can you describe the details of your use case/ideal user experience a bit further?
Thanks
Joe
On Mon, Aug 15, 2016 at 12:22 PM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
> I'm looking for a way to externally update a processors state, preferably through an API call and then at some point the existing State viewer with the clear button. From digging through the code base I couldn't find an exposed API that would allow for state modification; only a state clearing.
> Was there a design decision behind this, or is this a feature waiting to be added?
>
> Regards,
> Peter
Re: Externally update processor State
Posted by Joe Witt <jo...@gmail.com>.
Peter,
The state management function the is built-in is to allow processors
to set/interact with their own managed state. We did not intend for
that to be something someone external to the processor could interact
with directly. Rather, that is what properties are expected to be
used for.
So far the couple of requests like this I've seen are where there is a
sort of human in the process flow going on - like a workflow manager.
Can you describe the details of your use case/ideal user experience a
bit further?
Thanks
Joe
On Mon, Aug 15, 2016 at 12:22 PM, Peter Wicks (pwicks)
<pw...@micron.com> wrote:
> I'm looking for a way to externally update a processors state, preferably through an API call and then at some point the existing State viewer with the clear button. From digging through the code base I couldn't find an exposed API that would allow for state modification; only a state clearing.
> Was there a design decision behind this, or is this a feature waiting to be added?
>
> Regards,
> Peter