State Management - Set initial State or update State through GUI or API

["Peter Wicks (pwicks)" <pw...@micron.com>]

I'm using the QueryDatabaseTable processor.  Frequently I would like to set an initial state value for my columns max value, or I'd like to change the value stored in State Management for that column, this comes up in operational scenarios ("reload all data after this exact timestamp, that's when everything went bad").

But in looking through the code, and even the rest API, I don't see any options to set/modify a processes state.  Am I missing something? Is there a way to set initial values like I want?

Regards,
  Peter

Re: State Management - Set initial State or update State through GUI or API

[Mel J <me...@gmail.com>]

Hi,

Just checking if there was a resolution to this . I also have a similar use
case to implement "I would like to
set an initial state value for my columns max value, or I'd like to change
the value stored in State Management for that column, this comes up in
operational scenarios ("reload all data after this exact timestamp, that's
when everything went bad").
 But in looking through the code, and even the rest API, I don't see any
 options to set/modify a processes state.  Am I missing something? Is there
 a way to set initial values like I want? "

Thanks



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/State-Management-Set-initial-State-or-update-State-through-GUI-or-API-tp12909p16595.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: State Management - Set initial State or update State through GUI or API

[Joe Witt <jo...@gmail.com>]

Hello,

The idea is definitely a good one.  I'd propose that rather than
making state management editable (other than clearing/reseting it)
that a given processor should expose a property or properties to allow
a user to express their needs in terms of starting point.  If we were
to allow altering state then that too becomes an interface we have to
honor and it wouldn't offer much beyond the processor properties
themselves which have a well defined lifecycle, documentation,
interface, etc..  With that in mind I see state management as is to be
a valuable tool but really one that allows one to 'look behind the
wall' to see the internal state of a processor but not to interact
with it other than saying "reset" which can mean whatever reset might
mean for that processor.

So for this QueryDatabase processor perhaps a JIRA to add support for
a property or couple of properties that allows the original intent of
this thread to be captured.

Does that sound reasonable?

Thanks
Joe

On Thu, Jul 28, 2016 at 4:28 PM, Bryan Bende <bb...@gmail.com> wrote:
> We do allow users to view and clear the state for a given processor, which
> would set you back at the beginning.
>
> I'm wondering if there would be any downside to letting the users set the
> value of one of the keys in the state map on the view state screen.
> Seems like a cool feature to be able to start back at a specific value,
> rather than have to start at the beginning, but it also opens up someone
> entering something that causes bad behavior for the processor.
>
>
> On Thu, Jul 28, 2016 at 3:55 PM, Matt Burgess <ma...@gmail.com> wrote:
>
>> I believe that was a design decision to ensure for clusters and such that
>> state could not be modified on the fly (and thus possibly become
>> inconsistent or alter what would be a normal provenance trace). Processors
>> internally are responsible for maintaining the state they wish to keep, I
>> don't think it's possible to tweak it in order to change the behavior of
>> the processors.
>>
>> Having said that, there are much more knowledgeable folks than me re:
>> state management, hopefully they can clarify/correct my answer :)
>>
>> Regards,
>> Matt
>>
>> > On Jul 28, 2016, at 3:21 PM, Peter Wicks (pwicks) <pw...@micron.com>
>> wrote:
>> >
>> > I'm using the QueryDatabaseTable processor.  Frequently I would like to
>> set an initial state value for my columns max value, or I'd like to change
>> the value stored in State Management for that column, this comes up in
>> operational scenarios ("reload all data after this exact timestamp, that's
>> when everything went bad").
>> >
>> > But in looking through the code, and even the rest API, I don't see any
>> options to set/modify a processes state.  Am I missing something? Is there
>> a way to set initial values like I want?
>> >
>> > Regards,
>> >  Peter
>>

Re: State Management - Set initial State or update State through GUI or API

[Bryan Bende <bb...@gmail.com>]

We do allow users to view and clear the state for a given processor, which
would set you back at the beginning.

I'm wondering if there would be any downside to letting the users set the
value of one of the keys in the state map on the view state screen.
Seems like a cool feature to be able to start back at a specific value,
rather than have to start at the beginning, but it also opens up someone
entering something that causes bad behavior for the processor.


On Thu, Jul 28, 2016 at 3:55 PM, Matt Burgess <ma...@gmail.com> wrote:

> I believe that was a design decision to ensure for clusters and such that
> state could not be modified on the fly (and thus possibly become
> inconsistent or alter what would be a normal provenance trace). Processors
> internally are responsible for maintaining the state they wish to keep, I
> don't think it's possible to tweak it in order to change the behavior of
> the processors.
>
> Having said that, there are much more knowledgeable folks than me re:
> state management, hopefully they can clarify/correct my answer :)
>
> Regards,
> Matt
>
> > On Jul 28, 2016, at 3:21 PM, Peter Wicks (pwicks) <pw...@micron.com>
> wrote:
> >
> > I'm using the QueryDatabaseTable processor.  Frequently I would like to
> set an initial state value for my columns max value, or I'd like to change
> the value stored in State Management for that column, this comes up in
> operational scenarios ("reload all data after this exact timestamp, that's
> when everything went bad").
> >
> > But in looking through the code, and even the rest API, I don't see any
> options to set/modify a processes state.  Am I missing something? Is there
> a way to set initial values like I want?
> >
> > Regards,
> >  Peter
>

Re: State Management - Set initial State or update State through GUI or API

[Matt Burgess <ma...@gmail.com>]

I believe that was a design decision to ensure for clusters and such that state could not be modified on the fly (and thus possibly become inconsistent or alter what would be a normal provenance trace). Processors internally are responsible for maintaining the state they wish to keep, I don't think it's possible to tweak it in order to change the behavior of the processors.

Having said that, there are much more knowledgeable folks than me re: state management, hopefully they can clarify/correct my answer :)

Regards,
Matt

> On Jul 28, 2016, at 3:21 PM, Peter Wicks (pwicks) <pw...@micron.com> wrote:
> 
> I'm using the QueryDatabaseTable processor.  Frequently I would like to set an initial state value for my columns max value, or I'd like to change the value stored in State Management for that column, this comes up in operational scenarios ("reload all data after this exact timestamp, that's when everything went bad").
> 
> But in looking through the code, and even the rest API, I don't see any options to set/modify a processes state.  Am I missing something? Is there a way to set initial values like I want?
> 
> Regards,
>  Peter