You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Vibhath Ileperuma <vi...@gmail.com> on 2021/04/01 12:48:05 UTC

Understanding few points regarding a NIFI flow

Hi All.

I'm developing a NIFI flow to fetch csv files from a S3 bucket and write
those data to Apache kudu. I need to understand a few things regarding this
flow.

   1. Does ListS3 processor list down S3 objects in alphabetical order? Or
   is there any other specific order it uses to list down objects
   2. I need to pass the csv directory in the S3 bucket, from an external
   python script. But I notice that ListS3 does not allow
   incoming relationships. Is there a way to  change the csv directory
   dynamically? (Once all the csv files in a folder is fetched, processor
   should list down  files in another directory which is provided by the
   external python script)
   3. Once all the csv files in a directory are fetched and written to
   kudu, I need to notify the external python script that this much of rows
   are fetched and this muh of rows are written. Is it possible to do this? It
   is possible to use even an external database like Postgre.

*Best Regards,*

*Vibhath.*