You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by James McMahon <js...@gmail.com> on 2023/12/31 17:37:33 UTC
Extract from jars and nars
I have a NiFi flow that handles many jar and nar archive files as incoming
flowfiles. I am trying to figure out a way I can extract files from these
archives - for example, in most cases one incoming jar has a number of
files in its archive. So one flowfile should yield N output flowfiles if
there are N files in the archive.
I do not have /usr/bin/jar on my system. I have read, though, that unzip
can be employed to extract from jars, and I have that. So I am trying to
use that.
How can I configure an ExecuteStreamCommand processor to take an incoming
flowfile as stdin, and output each member of the archive as one of N output
flowfiles to stdout? Ideally I want to avoid having to output my streaming
flowfile to a temporary physical directory; I want to perform the
extraction entirely in stream.
I have used ExecuteStreamCommand before but can't recall how to get it to
work for this use case.
Thanks for any help.
Re: Extract from jars and nars
Posted by James McMahon <js...@gmail.com>.
It does indeed work perfectly, thanks very much Matt. Digging into the
errors I was seeing, it turned out that the jar and nar archive files were
being mangled by me in a prior flow step while trying to parse file
signatures out of the file headers. I mangled the headers and so corrupted
my jars. When I fixed that problem and tried what you said, it worked very
well. Thank you.
Cheers,
Jim
On Sun, Dec 31, 2023 at 6:49 PM Matt Burgess <ma...@apache.org> wrote:
> Jim,
>
> When you say you want to "avoid having to output them to a temp
> directory", does that include the content repo? If not you can use
> UnpackContent with a Packaging Type of zip. I tried on both JARs and
> NARs and it works.
>
> Regards,
> Matt
>
> On Sun, Dec 31, 2023 at 12:37 PM James McMahon <js...@gmail.com>
> wrote:
> >
> > I have a NiFi flow that handles many jar and nar archive files as
> incoming flowfiles. I am trying to figure out a way I can extract files
> from these archives - for example, in most cases one incoming jar has a
> number of files in its archive. So one flowfile should yield N output
> flowfiles if there are N files in the archive.
> >
> > I do not have /usr/bin/jar on my system. I have read, though, that unzip
> can be employed to extract from jars, and I have that. So I am trying to
> use that.
> >
> > How can I configure an ExecuteStreamCommand processor to take an
> incoming flowfile as stdin, and output each member of the archive as one of
> N output flowfiles to stdout? Ideally I want to avoid having to output my
> streaming flowfile to a temporary physical directory; I want to perform
> the extraction entirely in stream.
> >
> > I have used ExecuteStreamCommand before but can't recall how to get it
> to work for this use case.
> >
> > Thanks for any help.
>
Re: Extract from jars and nars
Posted by Matt Burgess <ma...@apache.org>.
Jim,
When you say you want to "avoid having to output them to a temp
directory", does that include the content repo? If not you can use
UnpackContent with a Packaging Type of zip. I tried on both JARs and
NARs and it works.
Regards,
Matt
On Sun, Dec 31, 2023 at 12:37 PM James McMahon <js...@gmail.com> wrote:
>
> I have a NiFi flow that handles many jar and nar archive files as incoming flowfiles. I am trying to figure out a way I can extract files from these archives - for example, in most cases one incoming jar has a number of files in its archive. So one flowfile should yield N output flowfiles if there are N files in the archive.
>
> I do not have /usr/bin/jar on my system. I have read, though, that unzip can be employed to extract from jars, and I have that. So I am trying to use that.
>
> How can I configure an ExecuteStreamCommand processor to take an incoming flowfile as stdin, and output each member of the archive as one of N output flowfiles to stdout? Ideally I want to avoid having to output my streaming flowfile to a temporary physical directory; I want to perform the extraction entirely in stream.
>
> I have used ExecuteStreamCommand before but can't recall how to get it to work for this use case.
>
> Thanks for any help.