Creating new sessions...

[Russell Bateman <ru...@windofkeltia.com>]

The ability to create a new session is conferred by extending 
AbstractSessionFactoryProcessor(in lieu of AbstractProcessorwhich 
further restricts Processor, among other things, removing the session 
factory) giving one the option of calling 
sessionFactory.createSession(). This begs the question of why or when 
would I wish to create a new session rather than operate within the one 
I'm given (by AbstractProcessor)?

Re: Creating new sessions...

[Matt Burgess <ma...@apache.org>]

Russell,

There are a couple of use cases I can think of where you'd want
AbstractSessionFactoryProcessor over AbstractProcessor:

1) You only want to update the State Manager after a successful commit
has taken place. Because AbstractProcessor.onTrigger() calls commit
for you, you can't add code after that to update the state map.

2) I am working on a processor that needs to keep the same session
across multiple calls to onTrigger, basically its a transactional
thing and I don't know when it will end. If something bad happens in
the meantime, I need to roll back that original session.

3) What Mark said :)

Regards,
Matt

On Fri, Mar 17, 2017 at 12:45 PM, Russell Bateman <ru...@windofkeltia.com> wrote:
> The ability to create a new session is conferred by extending
> AbstractSessionFactoryProcessor(in lieu of AbstractProcessorwhich further
> restricts Processor, among other things, removing the session factory)
> giving one the option of calling sessionFactory.createSession(). This begs
> the question of why or when would I wish to create a new session rather than
> operate within the one I'm given (by AbstractProcessor)?
>

Re: Creating new sessions...

[Russell Bateman <ru...@windofkeltia.com>]

Mark,

Yes, this helps me to understand. I had scoured the docs and not really 
seen a suggestion to this effect. I have inherited a number of processor 
that make use of this whereas I have only written processors that don't 
have need of notions of bins or batches.

Thanks (and to Matt also),

Russ

On 03/17/2017 10:54 AM, Mark Payne wrote:
> Russell,
>
> MergeContent is a good example of when this would be done.
>
> It creates a ProcessSession for each "bin" of FlowFiles that it holds onto. This allows it to build up several FlowFiles
> in one bin and then process that entire bin, committing the Process Session that is associated with it. Doing this does not
> affect any of the FlowFiles in another bin, though, as they belong to a different Process Session. If they all belonged to the
> same Process Session, then we would not be able to commit the session whenever one bin is full. Instead, we would have
> to wait until all bins are full, because in order to commit a Process Session, all of its FlowFiles must be accounted for
> (transferred or a relationship or removed).
>
> So in a more general sense, you would want this any time that you have multiple FlowFiles where you need them to belong
> to different session so that you can commit/rollback a session without accounting for all FlowFiles that you've seen.
>
> The AbstractSessionFactoryProcessor is also sometimes used when we want to avoid committing the session when returning
> from onTrigger. For example, if there is some asynchronous process going on.
>
> Does that make sense?
>
> Thanks
> -Mark
>
>
>> On Mar 17, 2017, at 12:45 PM, Russell Bateman <ru...@windofkeltia.com> wrote:
>>
>> The ability to create a new session is conferred by extending AbstractSessionFactoryProcessor(in lieu of AbstractProcessorwhich further restricts Processor, among other things, removing the session factory) giving one the option of calling sessionFactory.createSession(). This begs the question of why or when would I wish to create a new session rather than operate within the one I'm given (by AbstractProcessor)?
>>

Re: Creating new sessions...

[Mark Payne <ma...@hotmail.com>]

Russell,

MergeContent is a good example of when this would be done.

It creates a ProcessSession for each "bin" of FlowFiles that it holds onto. This allows it to build up several FlowFiles
in one bin and then process that entire bin, committing the Process Session that is associated with it. Doing this does not
affect any of the FlowFiles in another bin, though, as they belong to a different Process Session. If they all belonged to the
same Process Session, then we would not be able to commit the session whenever one bin is full. Instead, we would have
to wait until all bins are full, because in order to commit a Process Session, all of its FlowFiles must be accounted for
(transferred or a relationship or removed).

So in a more general sense, you would want this any time that you have multiple FlowFiles where you need them to belong
to different session so that you can commit/rollback a session without accounting for all FlowFiles that you've seen.

The AbstractSessionFactoryProcessor is also sometimes used when we want to avoid committing the session when returning
from onTrigger. For example, if there is some asynchronous process going on.

Does that make sense?

Thanks
-Mark


> On Mar 17, 2017, at 12:45 PM, Russell Bateman <ru...@windofkeltia.com> wrote:
> 
> The ability to create a new session is conferred by extending AbstractSessionFactoryProcessor(in lieu of AbstractProcessorwhich further restricts Processor, among other things, removing the session factory) giving one the option of calling sessionFactory.createSession(). This begs the question of why or when would I wish to create a new session rather than operate within the one I'm given (by AbstractProcessor)?
>