Dependent processing of files compressed files.

[Rich Kroll <ri...@modernizingmedicine.com>]

Hello all,
We are prototyping an NiFi based bulk load use case and have run into a
challenge that I hope one of you can help with.  We have the following file
layout:

file.tar.gz
/user1/user.xml
/user1/attachments/
/user1/attachments/file1.jpg
/user1/attachments/file2.jpg
/user2/user2.xml
....


What we are trying to accomplish is to decompress the file and for each xml
file, invoke a remote HTTP to create a new user and after successful
creation post each file within the attachments folder to another RESTful
HTTP endpoint using the identifier of the newly created user.

Picking up, decompressing, and creating each user by using a
RouteOnAttribute and InvokeHTTP is working well, but I can't seem to wrap
my head around a way to get access to the files in the nested folder
associated to the current user.

Can anyone provide guidance on how to best structure a flow for this type
of use case?

Thanks!

Rich

-- 
Rich Kroll
Director of Software Engineering



[image: [ Modernizing Medicine ]] <http://www.modmed.com>
[image: [ Facebook ]] <http://www.facebook.com/modernizingmedicine> [image:
[ LinkedIn ]] <http://www.linkedin.com/company/modernizing-medicine/> [image:
[ YouTube ]] <http://www.youtube.com/user/modernizingmedicine> [image: [
Twitter ]] <https://twitter.com/modmed_EMA> [image: [ Blog ]]
<http://www.modmed.com/BlogBeyondEMR> [image: [ Instagram ]]
<http://instagram.com/modernizing_medicine>

Re: Dependent processing of files compressed files.

[Aldrin Piri <al...@gmail.com>]

Rich,

The first thing that comes to mind is extracting an attribute, in this
case, "user" and accessing that via expression language [1], EL, to select
those files.  The successful response of InvokeHTTP could then take that
output and use it to drive a GetFile specifying the path for that user via
EL.

Let us know if this gets you a little further along your task at hand.

[1]
https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html

On Mon, Feb 22, 2016 at 2:22 PM, Rich Kroll <
rich.kroll@modernizingmedicine.com> wrote:

> Hello all,
> We are prototyping an NiFi based bulk load use case and have run into a
> challenge that I hope one of you can help with.  We have the following file
> layout:
>
> file.tar.gz
> /user1/user.xml
> /user1/attachments/
> /user1/attachments/file1.jpg
> /user1/attachments/file2.jpg
> /user2/user2.xml
> ....
>
>
> What we are trying to accomplish is to decompress the file and for each
> xml file, invoke a remote HTTP to create a new user and after successful
> creation post each file within the attachments folder to another RESTful
> HTTP endpoint using the identifier of the newly created user.
>
> Picking up, decompressing, and creating each user by using a
> RouteOnAttribute and InvokeHTTP is working well, but I can't seem to wrap
> my head around a way to get access to the files in the nested folder
> associated to the current user.
>
> Can anyone provide guidance on how to best structure a flow for this type
> of use case?
>
> Thanks!
>
> Rich
>
> --
> Rich Kroll
> Director of Software Engineering
>
>
>
> [image: [ Modernizing Medicine ]] <http://www.modmed.com>
> [image: [ Facebook ]] <http://www.facebook.com/modernizingmedicine> [image:
> [ LinkedIn ]] <http://www.linkedin.com/company/modernizing-medicine/> [image:
> [ YouTube ]] <http://www.youtube.com/user/modernizingmedicine> [image: [
> Twitter ]] <https://twitter.com/modmed_EMA> [image: [ Blog ]]
> <http://www.modmed.com/BlogBeyondEMR> [image: [ Instagram ]]
> <http://instagram.com/modernizing_medicine>
>
>