You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Maxim Vyazovsky <il...@gmail.com> on 2018/04/18 18:12:47 UTC
LIstAzureBlobStorage stops producing flow files
We use dockerized NiFi 1.6.0 in cluster mode (2 nodes) with external
zookeeper. LIstAzureBlobStorage processor is configured to run on primary
node. While new files appear on Azure Blob Storage, after some time
LIstAzureBlobStorage stops to generate flow files, cluster state for this
component constantly returns same value:
nifi_1 | 2018-04-18 15:18:13,333 DEBUG [Timer-Driven Process
Thread-7] o.a.n.p.a.storage.ListAzureBlobStorage
ListAzureBlobStorage[id=c9823e0d-b2af-161f-92d5-5860f1c39ea6] Returning
CLUSTER State: StandardStateMap[version=818,
values={processed.timestamp=1524062282000, listing.timestamp=1524062283000,
id.0=
https://STORAGE_ACCOUNT_NAME.blob.core.windows.net/backup/20180418/xxx/Alarm/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.json.gz,
id.1=
https://STORAGE_ACCOUNT_NAME.blob.core.windows.net/backup/20180418/yyy/Alarm/yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy.json.gz
}]
nifi_1 | 2018-04-18 15:18:13,333 DEBUG [Timer-Driven Process
Thread-7] o.a.n.p.a.storage.ListAzureBlobStorage
ListAzureBlobStorage[id=c9823e0d-b2af-161f-92d5-5860f1c39ea6] has chosen to
yield its resources; will not be scheduled to run again for 1000
milliseconds
nifi_1 | 2018-04-18 15:18:14,334 DEBUG [Timer-Driven Process
Thread-2] o.a.n.p.a.storage.ListAzureBlobStorage
ListAzureBlobStorage[id=c9823e0d-b2af-161f-92d5-5860f1c39ea6] Returning
CLUSTER State: StandardStateMap[version=818,
values={processed.timestamp=1524062282000, listing.timestamp=1524062283000,
id.0=
https://STORAGE_ACCOUNT_NAME.blob.core.windows.net/backup/20180418/xxx/Alarm/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.json.gz,
id.1=
https://STORAGE_ACCOUNT_NAME.blob.core.windows.net/backup/20180418/yyy/Alarm/yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy.json.gz
}]
nifi_1 | 2018-04-18 15:18:14,334 DEBUG [Timer-Driven Process
Thread-2] o.a.n.p.a.storage.ListAzureBlobStorage
ListAzureBlobStorage[id=c9823e0d-b2af-161f-92d5-5860f1c39ea6] has chosen to
yield its resources; will not be scheduled to run again for 1000
milliseconds
I thought it might be related to quit one of the node from the cluster and
starting of reelecting process, but I've tried to manually stop Primary
Node and everything went smoothly with reelection of new Primary and
continuation of getting new blob flow files even after returning of the
stopped node back to the cluster.
Let me know if you need additional details to help us resolve the issue.
--
Thanks,
Max