You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@nifi.apache.org by Tom Stewart <st...@yahoo.com> on 2016/04/13 17:59:02 UTC

'On primary node' ListSFTP not working for new cluster

I built a NiFi cluster and some test flows and things seem to be working fine. My three nodes show in the cluster view and are all connected with one marked PRIMARY. I cannot get 'On primary node' working with several processors I have tried. My current one is a simple flow consisting of ListSFTP and LogAttribute. If I set my ListSFTP to Timer Driven with Run schedule of 60 sec, it works fine. However then it runs on all of my nodes. I changed it to On Primary Node and while I see the log in the my primary node where it claims to start:
nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler Thread-3-SendThread(los90hdf4.novalocal:2181)] org.apache.zookeeper.ClientCnxn Reading reply sessionid:0x354103c2b860000, packet:: clientPath:null serverPath:null finished:false header:: 2,4 replyHeader:: 2,55834574850,0 request:: '/nifi/components/4603bfe7-6d98-4ad4-99f2-2a740034ae03,F response:: #10000,s{51539607588,51539607597,1460559569213,1460560385643,8,0,0,0,5,0,51539607588}
nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler Thread-1] o.a.nifi.processors.standard.ListSFTP ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] Returning CLUSTER State: StandardStateMap[version=8, values={}]
nifi-app.log:2016-04-13 10:53:55,599 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] to run with 1 threads

However, I do not see any errors or accesses on my SFTP server where it is actually ever attempting to connect. I can flip the processor back to Timer Driven and my SFTP server starts seeing requests. But when I toggle back to On Primary Node it doesn't appear to be executing. I think the other processor I tried this with was GetHTTP with similar experience.

Curious if there are any debug steps or setting recommendations that are useful to check if it appears that "On Primary Node" doesn't work for a cluster.

Re: 'On primary node' ListSFTP not working for new cluster

Posted by Joe Witt <jo...@gmail.com>.

Tom,

Ok that is pretty interesting and we'd want to get to the bottom of
it.  If you happen to see that state again could you please run
./bin/nifi.sh dump and send the logs/nifi-bootstrap.log that results.

Thanks
Joe

On Wed, Apr 13, 2016 at 2:36 PM, Tom Stewart <st...@yahoo.com> wrote:
> Yes I have three nodes on the NiFi cluster screen - two are CONNECTED and
> one is CONNECTED,PRIMARY. All have up to date heartbeats. When I run
> GenerateFlowFile with Timer Driven it executes on all three of my nodes.
>
> However, I just switched the PRIMARY to another node via election and now it
> is working. I moved it back to the one it was on previously and it is
> working there too.
>
> So it appears re-electing seemed to correct whatever state I was in that was
> preventing "On Primary Node" functionality from working. Thanks for the
> help!
>
>
> ________________________________
> From: Mark Payne <ma...@hotmail.com>
> To: users@nifi.apache.org; Tom Stewart <st...@yahoo.com>
> Sent: Wednesday, April 13, 2016 1:16 PM
>
> Subject: Re: 'On primary node' ListSFTP not working for new cluster
>
> Tom,
>
> It sounds like you do not have any node elected primary at all. If you click
> the cluster icon in the top-right corner,
> it should show all of the nodes in your cluster. Next to the nodes should be
> a ribbon that you can click to elect
> a new primary node... Though it should also show which node is currently the
> primary. Can you check if it shows
> a primary node? And if so, can you verify that the primary node is actually
> doing anything? I.e., if you start GenerateFlowFile
> on all nodes, can you see that it is indeed running on the primary node, in
> addition to the others?
>
> Thanks
> -Mark
>
>
> On Apr 13, 2016, at 2:01 PM, Tom Stewart <st...@yahoo.com> wrote:
>
> On Primary Node - the Tasks/Time stays at zero for the 5 minute interval
> that is displaying for me. When I flip it to Timer Driven it does increment
> as expected. I have my Run Schedule at "60 sec".
>
> The View State shows two keys (listing,timestamp/processed.timestamp), but
> the Value is not changing. Both show Scope=Cluster. I think this is residual
> data from when I had it set to "Timer Driven" and it actually processed some
> files.
>
> I neglected to mention the version - this is 0.60.
> I tried an even simpler flow with just GenerateFlowFile and LogAttribute and
> it does the same for me.
>
>
> ________________________________
> From: Mark Payne <ma...@hotmail.com>
> To: users@nifi.apache.org; Tom Stewart <st...@yahoo.com>
> Sent: Wednesday, April 13, 2016 11:42 AM
> Subject: Re: 'On primary node' ListSFTP not working for new cluster
>
> Tom,
>
> When you are running on Primary Node, do you see the "Tasks/Time" on the
> processor showing that tasks are running?
>
> If you right-click on the Processor and choose "View State", does it show
> anything in the table?
>
> Thanks
> -Mark
>
> On Apr 13, 2016, at 11:59 AM, Tom Stewart <st...@yahoo.com> wrote:
>
> I built a NiFi cluster and some test flows and things seem to be working
> fine. My three nodes show in the cluster view and are all connected with one
> marked PRIMARY. I cannot get 'On primary node' working with several
> processors I have tried. My current one is a simple flow consisting of
> ListSFTP and LogAttribute. If I set my ListSFTP to Timer Driven with Run
> schedule of 60 sec, it works fine. However then it runs on all of my nodes.
> I changed it to On Primary Node and while I see the log in the my primary
> node where it claims to start:
>
> nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler
> Thread-3-SendThread(los90hdf4.novalocal:2181)]
> org.apache.zookeeper.ClientCnxn Reading reply sessionid:0x354103c2b860000,
> packet:: clientPath:null serverPath:null finished:false header:: 2,4
> replyHeader:: 2,55834574850,0  request::
> '/nifi/components/4603bfe7-6d98-4ad4-99f2-2a740034ae03,F  response::
> #10000,s{51539607588,51539607597,1460559569213,1460560385643,8,0,0,0,5,0,51539607588}
> nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler
> Thread-1] o.a.nifi.processors.standard.ListSFTP
> ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] Returning CLUSTER State:
> StandardStateMap[version=8, values={}]
> nifi-app.log:2016-04-13 10:53:55,599 INFO [StandardProcessScheduler
> Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled
> ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] to run with 1 threads
>
> However, I do not see any errors or accesses on my SFTP server where it is
> actually ever attempting to connect. I can flip the processor back to Timer
> Driven and my SFTP server starts seeing requests. But when I toggle back to
> On Primary Node it doesn't appear to be executing. I think the other
> processor I tried this with was GetHTTP with similar experience.
>
> Curious if there are any debug steps or setting recommendations that are
> useful to check if it appears that "On Primary Node" doesn't work for a
> cluster.
>
>
>
>
>
>
>
>
>

Re: 'On primary node' ListSFTP not working for new cluster

Posted by Tom Stewart <st...@yahoo.com>.

Yes I have three nodes on the NiFi cluster screen - two are CONNECTED and one is CONNECTED,PRIMARY. All have up to date heartbeats. When I run GenerateFlowFile with Timer Driven it executes on all three of my nodes.

However, I just switched the PRIMARY to another node via election and now it is working. I moved it back to the one it was on previously and it is working there too.

So it appears re-electing seemed to correct whatever state I was in that was preventing "On Primary Node" functionality from working. Thanks for the help!

From: Mark Payne <ma...@hotmail.com>
To: users@nifi.apache.org; Tom Stewart <st...@yahoo.com>
Sent: Wednesday, April 13, 2016 1:16 PM
Subject: Re: 'On primary node' ListSFTP not working for new cluster

Tom,
It sounds like you do not have any node elected primary at all. If you click the cluster icon in the top-right corner,it should show all of the nodes in your cluster. Next to the nodes should be a ribbon that you can click to electa new primary node... Though it should also show which node is currently the primary. Can you check if it showsa primary node? And if so, can you verify that the primary node is actually doing anything? I.e., if you start GenerateFlowFileon all nodes, can you see that it is indeed running on the primary node, in addition to the others?
Thanks-Mark

On Apr 13, 2016, at 2:01 PM, Tom Stewart <st...@yahoo.com> wrote:
On Primary Node - the Tasks/Time stays at zero for the 5 minute interval that is displaying for me. When I flip it to Timer Driven it does increment as expected. I have my Run Schedule at "60 sec".

The View State shows two keys (listing,timestamp/processed.timestamp), but the Value is not changing. Both show Scope=Cluster. I think this is residual data from when I had it set to "Timer Driven" and it actually processed some files.

I neglected to mention the version - this is 0.60.
I tried an even simpler flow with just GenerateFlowFile and LogAttribute and it does the same for me.

From: Mark Payne <ma...@hotmail.com>
To: users@nifi.apache.org; Tom Stewart <st...@yahoo.com>
Sent: Wednesday, April 13, 2016 11:42 AM
Subject: Re: 'On primary node' ListSFTP not working for new cluster

Tom,
When you are running on Primary Node, do you see the "Tasks/Time" on the processor showing that tasks are running?
If you right-click on the Processor and choose "View State", does it show anything in the table?
Thanks-Mark

On Apr 13, 2016, at 11:59 AM, Tom Stewart <st...@yahoo.com> wrote:
I built a NiFi cluster and some test flows and things seem to be working fine. My three nodes show in the cluster view and are all connected with one marked PRIMARY. I cannot get 'On primary node' working with several processors I have tried. My current one is a simple flow consisting of ListSFTP and LogAttribute. If I set my ListSFTP to Timer Driven with Run schedule of 60 sec, it works fine. However then it runs on all of my nodes. I changed it to On Primary Node and while I see the log in the my primary node where it claims to start:
nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler Thread-3-SendThread(los90hdf4.novalocal:2181)] org.apache.zookeeper.ClientCnxn Reading reply sessionid:0x354103c2b860000, packet:: clientPath:null serverPath:null finished:false header:: 2,4 replyHeader:: 2,55834574850,0 request:: '/nifi/components/4603bfe7-6d98-4ad4-99f2-2a740034ae03,F response:: #10000,s{51539607588,51539607597,1460559569213,1460560385643,8,0,0,0,5,0,51539607588}
nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler Thread-1] o.a.nifi.processors.standard.ListSFTP ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] Returning CLUSTER State: StandardStateMap[version=8, values={}]
nifi-app.log:2016-04-13 10:53:55,599 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] to run with 1 threads

Curious if there are any debug steps or setting recommendations that are useful to check if it appears that "On Primary Node" doesn't work for a cluster.

Re: 'On primary node' ListSFTP not working for new cluster

Posted by Mark Payne <ma...@hotmail.com>.

Tom,

It sounds like you do not have any node elected primary at all. If you click the cluster icon in the top-right corner,
it should show all of the nodes in your cluster. Next to the nodes should be a ribbon that you can click to elect
a new primary node... Though it should also show which node is currently the primary. Can you check if it shows
a primary node? And if so, can you verify that the primary node is actually doing anything? I.e., if you start GenerateFlowFile
on all nodes, can you see that it is indeed running on the primary node, in addition to the others?

Thanks
-Mark


> On Apr 13, 2016, at 2:01 PM, Tom Stewart <st...@yahoo.com> wrote:
> 
> On Primary Node - the Tasks/Time stays at zero for the 5 minute interval that is displaying for me. When I flip it to Timer Driven it does increment as expected. I have my Run Schedule at "60 sec". 
> 
> The View State shows two keys (listing,timestamp/processed.timestamp), but the Value is not changing. Both show Scope=Cluster. I think this is residual data from when I had it set to "Timer Driven" and it actually processed some files. 
> 
> I neglected to mention the version - this is 0.60. 
> I tried an even simpler flow with just GenerateFlowFile and LogAttribute and it does the same for me. 
> 
> 
> From: Mark Payne <ma...@hotmail.com>
> To: users@nifi.apache.org; Tom Stewart <st...@yahoo.com> 
> Sent: Wednesday, April 13, 2016 11:42 AM
> Subject: Re: 'On primary node' ListSFTP not working for new cluster
> 
> Tom,
> 
> When you are running on Primary Node, do you see the "Tasks/Time" on the processor showing that tasks are running?
> 
> If you right-click on the Processor and choose "View State", does it show anything in the table?
> 
> Thanks
> -Mark
> 
>> On Apr 13, 2016, at 11:59 AM, Tom Stewart <stewartthomasj@yahoo.com <ma...@yahoo.com>> wrote:
>> 
>> I built a NiFi cluster and some test flows and things seem to be working fine. My three nodes show in the cluster view and are all connected with one marked PRIMARY. I cannot get 'On primary node' working with several processors I have tried. My current one is a simple flow consisting of ListSFTP and LogAttribute. If I set my ListSFTP to Timer Driven with Run schedule of 60 sec, it works fine. However then it runs on all of my nodes. I changed it to On Primary Node and while I see the log in the my primary node where it claims to start:
>> 
>> nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler Thread-3-SendThread(los90hdf4.novalocal:2181)] org.apache.zookeeper.ClientCnxn Reading reply sessionid:0x354103c2b860000, packet:: clientPath:null serverPath:null finished:false header:: 2,4  replyHeader:: 2,55834574850,0  request:: '/nifi/components/4603bfe7-6d98-4ad4-99f2-2a740034ae03,F  response:: #10000,s{51539607588,51539607597,1460559569213,1460560385643,8,0,0,0,5,0,51539607588}
>> nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler Thread-1] o.a.nifi.processors.standard.ListSFTP ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] Returning CLUSTER State: StandardStateMap[version=8, values={}]
>> nifi-app.log:2016-04-13 10:53:55,599 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] to run with 1 threads
>> 
>> However, I do not see any errors or accesses on my SFTP server where it is actually ever attempting to connect. I can flip the processor back to Timer Driven and my SFTP server starts seeing requests. But when I toggle back to On Primary Node it doesn't appear to be executing. I think the other processor I tried this with was GetHTTP with similar experience. 
>> 
>> Curious if there are any debug steps or setting recommendations that are useful to check if it appears that "On Primary Node" doesn't work for a cluster. 
>> 
>> 
>> 
> 
> 
>

Re: 'On primary node' ListSFTP not working for new cluster

Posted by Tom Stewart <st...@yahoo.com>.

On Primary Node - the Tasks/Time stays at zero for the 5 minute interval that is displaying for me. When I flip it to Timer Driven it does increment as expected. I have my Run Schedule at "60 sec".

I neglected to mention the version - this is 0.60.
I tried an even simpler flow with just GenerateFlowFile and LogAttribute and it does the same for me.

Curious if there are any debug steps or setting recommendations that are useful to check if it appears that "On Primary Node" doesn't work for a cluster.

Re: 'On primary node' ListSFTP not working for new cluster

Posted by Mark Payne <ma...@hotmail.com>.

Tom,

When you are running on Primary Node, do you see the "Tasks/Time" on the processor showing that tasks are running?

If you right-click on the Processor and choose "View State", does it show anything in the table?

Thanks
-Mark

> On Apr 13, 2016, at 11:59 AM, Tom Stewart <st...@yahoo.com> wrote:
> 
> I built a NiFi cluster and some test flows and things seem to be working fine. My three nodes show in the cluster view and are all connected with one marked PRIMARY. I cannot get 'On primary node' working with several processors I have tried. My current one is a simple flow consisting of ListSFTP and LogAttribute. If I set my ListSFTP to Timer Driven with Run schedule of 60 sec, it works fine. However then it runs on all of my nodes. I changed it to On Primary Node and while I see the log in the my primary node where it claims to start:
> 
> nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler Thread-3-SendThread(los90hdf4.novalocal:2181)] org.apache.zookeeper.ClientCnxn Reading reply sessionid:0x354103c2b860000, packet:: clientPath:null serverPath:null finished:false header:: 2,4  replyHeader:: 2,55834574850,0  request:: '/nifi/components/4603bfe7-6d98-4ad4-99f2-2a740034ae03,F  response:: #10000,s{51539607588,51539607597,1460559569213,1460560385643,8,0,0,0,5,0,51539607588}
> nifi-app.log:2016-04-13 10:53:55,598 DEBUG [StandardProcessScheduler Thread-1] o.a.nifi.processors.standard.ListSFTP ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] Returning CLUSTER State: StandardStateMap[version=8, values={}]
> nifi-app.log:2016-04-13 10:53:55,599 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled ListSFTP[id=4603bfe7-6d98-4ad4-99f2-2a740034ae03] to run with 1 threads
> 
> However, I do not see any errors or accesses on my SFTP server where it is actually ever attempting to connect. I can flip the processor back to Timer Driven and my SFTP server starts seeing requests. But when I toggle back to On Primary Node it doesn't appear to be executing. I think the other processor I tried this with was GetHTTP with similar experience. 
> 
> Curious if there are any debug steps or setting recommendations that are useful to check if it appears that "On Primary Node" doesn't work for a cluster. 
> 
> 
>