You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by plj <pl...@mitre.org> on 2015/08/17 20:05:17 UTC
RE: Process to create multiple files
Ok that makes sense. What processor would I extend/implement to allow me to send out multiple files from the process that unpacks the NITF bundle? Any examples?
Thank,
Pat
From: Joe Witt [via Apache NiFi (incubating) Developer List] [mailto:ml-node+s39713n2513h65@n7.nabble.com]
Sent: Monday, August 17, 2015 1:56 PM
To: Jones, Patrick L. <pl...@mitre.org>
Subject: Re: Process to create multiple files
Hello Pat,
Yeah makes sense. You would need a custom processor to support the
NITF format [1]
There does appear to be a Java library to deal with it but its
licensing isn't ASL v2 compatible so we're not likely to be able to
play along anytime soon. But that should be a good way to get started
building what you need.
The processor would unpack the items out of the NITF bundle and then
could send out the full size images if that is how it works. Those
full size images can then be resized to thumbnails. Tons of ways to
play this one and it is a very common style of use case.
Thanks
Joe
[1] https://en.wikipedia.org/wiki/National_Imagery_Transmission_Format
[2] https://github.com/codice/imaging-nitf
On Mon, Aug 17, 2015 at 1:45 PM, plj <[hidden email]</user/SendEmail.jtp?type=node&node=2513&i=0>> wrote:
> Howdy,
>
> Thanks for the reply. I think my situation is different than you suggest. I have an image file in NITF format. That file may have multiple images and multiple bands inside that one file. The software that I have that reads that file does create a thumbnail for each image and each band inside that file. I don't know how many thumbnails I need until I read the files metadata.
>
>
> Thank
>
> Pat
>
> From: Dan Bress [via Apache NiFi (incubating) Developer List] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=2513&i=1>]
> Sent: Monday, August 17, 2015 1:40 PM
> To: Jones, Patrick L. <[hidden email]</user/SendEmail.jtp?type=node&node=2513&i=2>>
> Subject: Re: Process to create multiple files
>
> plj,
> I would not recommend having this processor create multiple thumbnails. What I would recommend is the following:
>
> Create a new processor called "CreateThumbnail" or "RescaleImage"
>
> Then have a configuration on the processor that says what size the output image should be(e.g. 128x128, or 1/X of original size).
>
> Your new processor will read in the incoming image, and rescale it down to the user specified size and pass it forward.
>
> Now if you want to create a 128x128 64x64 and 32x32 sized images you would do the following.
>
> (GetFile)->(RescaleImage configured to 128x128)->(PutFile)
> |->(RescaleImage configured to 64x64)->(PutFile)
> \->(RescaleImage configured to 32x32)->(PutFile)
>
> Where GetFile has 3 success relationships, each going to a different RescaleImage processor.
>
> I think it makes more sense to have one processor create one file, then you can use the flow to visually configure how many copies of the file you want to make. This should make this processor simpler and more reusable.
>
>
> Dan Bress
> Software Engineer
> ONYX Consulting Services
>
> ________________________________________
> From: plj <[hidden email]</user/SendEmail.jtp?type=node&node=2511&i=0>>
> Sent: Monday, August 17, 2015 1:27 PM
> To: [hidden email]</user/SendEmail.jtp?type=node&node=2511&i=1>
> Subject: Process to create multiple files
>
> Howdy,
>
> I'm new to NiFi so please bear with me. What I want to accomplish is:
> read an image file
> process the file to create one or more thumbnails from the image.
> Send the resulting thumbnails along the flow
>
> So I can use GetFile to read the file and then send it along. I think I
> need to write a custom java processor that will process the image file and
> then send each of the thumbnail files (say .jpg for now) on to the next
> thing in the flow (say PutFile for example).
>
> Are there suggestions on what I should implement or extend to create my
> custom processor? It will take in one file and output multiple files.
> Would extending "PutFile" so that it processed and then puts each thumbnail
> on the flow be a good strategy? Other ideas?
>
> Thank you,
>
>
>
>
>
> --
> View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
>
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2511.html
> To unsubscribe from Process to create multiple files, click here<
> NAML<http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
>
> --
> View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2512.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
________________________________
If you reply to this email, your message will be added to the discussion below:
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2513.html
To unsubscribe from Process to create multiple files, click here<http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2510&code=cGxqQG1pdHJlLm9yZ3wyNTEwfDU1MTMzODA1>.
NAML<http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2516.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
Re: Process to create multiple files
Posted by Joe Percivall <jo...@yahoo.com.INVALID>.
For a very simple example you could check out the DuplicateFlowFile standard processor.
Joe- - - - - - Joseph Percivalllinkedin.com/in/Percivall
c: (410) 693-8032e: joepercivall@yahoo.com
On Monday, August 17, 2015 2:36 PM, plj <pl...@mitre.org> wrote:
Ok that makes sense. What processor would I extend/implement to allow me to send out multiple files from the process that unpacks the NITF bundle? Any examples?
Thank,
Pat
From: Joe Witt [via Apache NiFi (incubating) Developer List] [mailto:ml-node+s39713n2513h65@n7.nabble.com]
Sent: Monday, August 17, 2015 1:56 PM
To: Jones, Patrick L. <pl...@mitre.org>
Subject: Re: Process to create multiple files
Hello Pat,
Yeah makes sense. You would need a custom processor to support the
NITF format [1]
There does appear to be a Java library to deal with it but its
licensing isn't ASL v2 compatible so we're not likely to be able to
play along anytime soon. But that should be a good way to get started
building what you need.
The processor would unpack the items out of the NITF bundle and then
could send out the full size images if that is how it works. Those
full size images can then be resized to thumbnails. Tons of ways to
play this one and it is a very common style of use case.
Thanks
Joe
[1] https://en.wikipedia.org/wiki/National_Imagery_Transmission_Format
[2] https://github.com/codice/imaging-nitf
On Mon, Aug 17, 2015 at 1:45 PM, plj <[hidden email]</user/SendEmail.jtp?type=node&node=2513&i=0>> wrote:
> Howdy,
>
> Thanks for the reply. I think my situation is different than you suggest. I have an image file in NITF format. That file may have multiple images and multiple bands inside that one file. The software that I have that reads that file does create a thumbnail for each image and each band inside that file. I don't know how many thumbnails I need until I read the files metadata.
>
>
> Thank
>
> Pat
>
> From: Dan Bress [via Apache NiFi (incubating) Developer List] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=2513&i=1>]
> Sent: Monday, August 17, 2015 1:40 PM
> To: Jones, Patrick L. <[hidden email]</user/SendEmail.jtp?type=node&node=2513&i=2>>
> Subject: Re: Process to create multiple files
>
> plj,
> I would not recommend having this processor create multiple thumbnails. What I would recommend is the following:
>
> Create a new processor called "CreateThumbnail" or "RescaleImage"
>
> Then have a configuration on the processor that says what size the output image should be(e.g. 128x128, or 1/X of original size).
>
> Your new processor will read in the incoming image, and rescale it down to the user specified size and pass it forward.
>
> Now if you want to create a 128x128 64x64 and 32x32 sized images you would do the following.
>
> (GetFile)->(RescaleImage configured to 128x128)->(PutFile)
> |->(RescaleImage configured to 64x64)->(PutFile)
> \->(RescaleImage configured to 32x32)->(PutFile)
>
> Where GetFile has 3 success relationships, each going to a different RescaleImage processor.
>
> I think it makes more sense to have one processor create one file, then you can use the flow to visually configure how many copies of the file you want to make. This should make this processor simpler and more reusable.
>
>
> Dan Bress
> Software Engineer
> ONYX Consulting Services
>
> ________________________________________
> From: plj <[hidden email]</user/SendEmail.jtp?type=node&node=2511&i=0>>
> Sent: Monday, August 17, 2015 1:27 PM
> To: [hidden email]</user/SendEmail.jtp?type=node&node=2511&i=1>
> Subject: Process to create multiple files
>
> Howdy,
>
> I'm new to NiFi so please bear with me. What I want to accomplish is:
> read an image file
> process the file to create one or more thumbnails from the image.
> Send the resulting thumbnails along the flow
>
> So I can use GetFile to read the file and then send it along. I think I
> need to write a custom java processor that will process the image file and
> then send each of the thumbnail files (say .jpg for now) on to the next
> thing in the flow (say PutFile for example).
>
> Are there suggestions on what I should implement or extend to create my
> custom processor? It will take in one file and output multiple files.
> Would extending "PutFile" so that it processed and then puts each thumbnail
> on the flow be a good strategy? Other ideas?
>
> Thank you,
>
>
>
>
>
> --
> View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
>
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2511.html
> To unsubscribe from Process to create multiple files, click here<
> NAML<http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
>
> --
> View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2512.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
________________________________
If you reply to this email, your message will be added to the discussion below:
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2513.html
To unsubscribe from Process to create multiple files, click here<http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2510&code=cGxqQG1pdHJlLm9yZ3wyNTEwfDU1MTMzODA1>.
NAML<http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context: http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2516.html
Sent from the Apache NiFi (incubating) Developer List mailing list archive at Nabble.com.
Re: Process to create multiple files
Posted by Bryan Bende <bb...@gmail.com>.
Pat,
You don't need to extend anything special to produce multiple files. You
can create as many FlowFiles as you want and transfer them to a
relationship, such as...
FlowFile childFlowFile = session.create(original);
// create more
List<FlowFile> flowFiles = new ArrayList<FlowFile>();
flowFiles.add(childFLowFile);
session.transfer(flowFIles, YOUR_RELATIONSHIP);
Good examples might be SplitJson or SplitXML.
-Bryan
On Mon, Aug 17, 2015 at 2:05 PM, plj <pl...@mitre.org> wrote:
> Ok that makes sense. What processor would I extend/implement to allow me
> to send out multiple files from the process that unpacks the NITF bundle?
> Any examples?
>
> Thank,
>
> Pat
>
>
> From: Joe Witt [via Apache NiFi (incubating) Developer List] [mailto:
> ml-node+s39713n2513h65@n7.nabble.com]
> Sent: Monday, August 17, 2015 1:56 PM
> To: Jones, Patrick L. <pl...@mitre.org>
> Subject: Re: Process to create multiple files
>
> Hello Pat,
>
> Yeah makes sense. You would need a custom processor to support the
> NITF format [1]
>
> There does appear to be a Java library to deal with it but its
> licensing isn't ASL v2 compatible so we're not likely to be able to
> play along anytime soon. But that should be a good way to get started
> building what you need.
>
> The processor would unpack the items out of the NITF bundle and then
> could send out the full size images if that is how it works. Those
> full size images can then be resized to thumbnails. Tons of ways to
> play this one and it is a very common style of use case.
>
> Thanks
> Joe
>
> [1] https://en.wikipedia.org/wiki/National_Imagery_Transmission_Format
> [2] https://github.com/codice/imaging-nitf
>
> On Mon, Aug 17, 2015 at 1:45 PM, plj <[hidden
> email]</user/SendEmail.jtp?type=node&node=2513&i=0>> wrote:
>
> > Howdy,
> >
> > Thanks for the reply. I think my situation is different
> than you suggest. I have an image file in NITF format. That file may have
> multiple images and multiple bands inside that one file. The software that
> I have that reads that file does create a thumbnail for each image and each
> band inside that file. I don't know how many thumbnails I need until I
> read the files metadata.
> >
> >
> > Thank
> >
> > Pat
> >
> > From: Dan Bress [via Apache NiFi (incubating) Developer List] [mailto:[hidden
> email]</user/SendEmail.jtp?type=node&node=2513&i=1>]
> > Sent: Monday, August 17, 2015 1:40 PM
> > To: Jones, Patrick L. <[hidden
> email]</user/SendEmail.jtp?type=node&node=2513&i=2>>
> > Subject: Re: Process to create multiple files
> >
> > plj,
> > I would not recommend having this processor create multiple
> thumbnails. What I would recommend is the following:
> >
> > Create a new processor called "CreateThumbnail" or "RescaleImage"
> >
> > Then have a configuration on the processor that says what size the
> output image should be(e.g. 128x128, or 1/X of original size).
> >
> > Your new processor will read in the incoming image, and rescale it
> down to the user specified size and pass it forward.
> >
> > Now if you want to create a 128x128 64x64 and 32x32 sized images
> you would do the following.
> >
> > (GetFile)->(RescaleImage configured to 128x128)->(PutFile)
> > |->(RescaleImage configured to 64x64)->(PutFile)
> > \->(RescaleImage configured to 32x32)->(PutFile)
> >
> > Where GetFile has 3 success relationships, each going to a different
> RescaleImage processor.
> >
> > I think it makes more sense to have one processor create one file, then
> you can use the flow to visually configure how many copies of the file you
> want to make. This should make this processor simpler and more reusable.
> >
> >
> > Dan Bress
> > Software Engineer
> > ONYX Consulting Services
> >
> > ________________________________________
> > From: plj <[hidden email]</user/SendEmail.jtp?type=node&node=2511&i=0>>
> > Sent: Monday, August 17, 2015 1:27 PM
> > To: [hidden email]</user/SendEmail.jtp?type=node&node=2511&i=1>
> > Subject: Process to create multiple files
> >
> > Howdy,
> >
> > I'm new to NiFi so please bear with me. What I want to accomplish is:
> > read an image file
> > process the file to create one or more thumbnails from the image.
> > Send the resulting thumbnails along the flow
> >
> > So I can use GetFile to read the file and then send it along. I
> think I
> > need to write a custom java processor that will process the image file
> and
> > then send each of the thumbnail files (say .jpg for now) on to the next
> > thing in the flow (say PutFile for example).
> >
> > Are there suggestions on what I should implement or extend to create
> my
> > custom processor? It will take in one file and output multiple files.
> > Would extending "PutFile" so that it processed and then puts each
> thumbnail
> > on the flow be a good strategy? Other ideas?
> >
> > Thank you,
> >
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510.html
> > Sent from the Apache NiFi (incubating) Developer List mailing list
> archive at Nabble.com.
> >
> > ________________________________
> > If you reply to this email, your message will be added to the discussion
> below:
> >
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2511.html
> > To unsubscribe from Process to create multiple files, click here<
> > NAML<
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2512.html
> > Sent from the Apache NiFi (incubating) Developer List mailing list
> archive at Nabble.com.
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2513.html
> To unsubscribe from Process to create multiple files, click here<
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2510&code=cGxqQG1pdHJlLm9yZ3wyNTEwfDU1MTMzODA1
> >.
> NAML<
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Process-to-create-multiple-files-tp2510p2516.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive
> at Nabble.com.
>