You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Mark Nuttall <mk...@gmail.com> on 2017/10/01 00:13:54 UTC

Re: REST DSL process multipart/form-data binary file?

it is just a file. any example of processing a file should work.  you will
be able to save it somewhere and then you will have to call some processor
to read/process it.

On Fri, Sep 29, 2017 at 7:47 PM, Mark <el...@gmail.com> wrote:

> I'm trying to figure out how to process a binary file that is sent to my
> Camel REST service via POST.  I have not found any good examples on how
> this can be done using Camel.  Does anyone have any experiences they could
> share or links to more documentation?
>
> I'm currently using the jetty component for the restconfiguration.
>
> Thanks,
> Mark
>

Re: REST DSL process multipart/form-data binary file?

Posted by Mark <el...@gmail.com>.
Figured it out.  The Postman extension for Chrome sucks.  The Advanced REST
Client works properly and sends data to my server with no problem.  I still
think something is broke with the version of Jetty that comes with Camel
2.16.2, but netty4-http works like a champ.

Thanks for helping me out.


On Mon, Oct 2, 2017 at 7:09 AM, Mark <el...@gmail.com> wrote:

> I'm sending the data right now using the Postman app in Chrome.
>
> My data comes across looking like this:
>
> ------WebKitFormBoundaryNZwE5GrXaeBuK9I9
>
> Content-Disposition: form-data; name="fileUpload"; filename="nhc.kmz"
> Content-Type: application/vnd.google-earth.kmz
>
> [binary data]
>
> ------WebKitFormBoundaryNZwE5GrXaeBuK9I9
> Content-Disposition: form-data; name="header1"
>
> value1
>
> ------WebKitFormBoundaryNZwE5GrXaeBuK9I9--
>
>
>
> On Mon, Oct 2, 2017 at 5:41 AM, Roman Vottner <ro...@gmx.at> wrote:
>
>> How do you send data to your application? Have you specified a
>> Content-Disposition header for the respective parts?
>>
>> We analysed a raw request via wireshark and it does look something like
>> this:
>>
>> GET /api/someResource HTTP/1.1
>> Host: www.somewhere.com <http://www.somewhere.com/>
>> …
>> Content-Type: multipart/form-data; boundary=---------------------
>> ---d9c950f09cd11bba
>>
>> --------------------------d9c950f09cd11bba
>> Content-Disposition: form-data; name="dat1"; filename=„SomeFile.zip"
>> Content-Type: application/octet-stream
>>
>> …
>>
>> --------------------------d9c950f09cd11bba
>> Content-Disposition: form-data; name="dat2"; filename=„SomeOtherFile.zip"
>> Content-Type: application/octet-stream
>>
>> …
>>
>>
>> We use Camel 2.19.0 and the Undertow version Camel
>> (camel-undertow-starter) and/or Spring-Boot (spring-boot-starter-undertow)
>> includes their dependencies (undertow 1.4.13.FINAL).
>>
>>
>> > Am 02.10.2017 um 04:25 schrieb Mark Webb [via Camel] <
>> ml+s465427n5814065h66@n5.nabble.com>:
>> >
>> > Thanks for the example.  I cannot get your code working though.  I get
>> the
>> > following:
>> >
>> > [qtp1422312468-17] HandleUpload                   INFO  Upload received
>> > [ qtp1422312468-17] HandleUpload                   INFO  Exchange body:
>> > org.apache.camel.converter.stream.InputStreamCache@1cf48789
>> > [qtp1422312468-17] HandleUpload                   INFO  Attachment
>> size: 0
>> > [qtp1422312468-17] HandleUpload                   INFO  Attachment
>> object
>> > size: 0
>> > [qtp1422312468-17] HandleUpload                   WARN  No attachments
>> > found!
>> >
>> > I am testing with the Postman app in the Chrome Browser and using Jetty.
>> > I'm using Camel 2.16.2, since we're deploying in ServiceMix.   I tried
>> > testing with Camel 2.19.3 and observed the same problem so I don't
>> think a
>> > newer version of Camel fixes things.  What environment are you running
>> your
>> > code inside?
>> >
>> >
>> >
>> >
>> >
>> > On Sat, Sep 30, 2017 at 9:13 PM, Roman Vottner <[hidden email]
>> <x-msg://5/user/SendEmail.jtp?type=node&node=5814065&i=0>> wrote:
>> >
>> > > Our Camel REST DSL configuration looks like this:
>> > >
>> > >     onException(Exception.class)
>> > >         .handled(true)
>> > >         .logExhausted(false)
>> > >         .log(LoggingLevel.ERROR, ">> In default/message exception
>> handler.
>> > > ${exception}, message: ${exception.message}, stacktrace:
>> > > ${exception.stacktrace}")
>> > >         .onRedelivery((Exchange exchange) -> LOG.debug(">> Default
>> > > exception handler"))
>> > >         .bean(PrepareErrorResponse.class).id("ErrorResponse")
>> > >         .end();
>> > >
>> > >     restConfiguration()
>> > >         .component("undertow")
>> > >         .port("{{server.port}}")
>> > >         .contextPath("/api")
>> > >         .endpointProperty("matchOnUriPrefix", "true")
>> > >         .endpointProperty("sendServerVersion", "false")
>> > >         .endpointProperty("chunked", "true");
>> > >
>> > > where the actual route handling the multipart-request does look
>> something
>> > > along the line:
>> > >
>> > >     rest("/someEndpoint")
>> > >         .post("/{id}")
>> > >         .consumes(MediaType.MULTIPART_FORM_DATA_VALUE)
>> > >         .route().routeId("upload-multipart-route")
>> > >           // Spring Security authentication check via Authorization
>> header
>> > >           .bean(SpringSecurityContextLoader.class)
>> > >               .policy(authorizationPolicy)
>> > >           .log("Uploading to resource id ${header.id}")
>> > >           .log(LoggingLevel.INFO, LOG, CONFIDENTIAL,"Headers:
>> ${headers}")
>> > >           .bean(HandleUpload.class)
>> > >           .process((Exchange exchange) -> {
>> > >             exchange.getIn().setHeader(Exchange.HTTP_RESPONSE_CODE,
>> 200);
>> > >             exchange.getIn().setHeader(Exchange.CONTENT_TYPE,
>> > > MediaType.APPLICATION_JSON_UTF8_VALUE);
>> > >             ...
>> > >           })
>> > >     .endRest();
>> > >
>> > > and the HandleUpload class looking something like this:
>> > >
>> > > public class HandleUpload {
>> > >
>> > >   private static final Logger LOG = LoggerFactory.getLogger(Method
>> > > Handles.lookup().lookupClass());
>> > >
>> > >   @Autowired
>> > >   private SomeRepository someRepository;
>> > >
>> > >   @Handler
>> > >   public void processUpload(@Attachments Map<String, DataHandler>
>> > > attachments,
>> > >                             @Header("Content-Type") String
>> contentType,
>> > >                             @Header("id") String id,
>> > >                             Exchange exchange)
>> > >       throws Exception {
>> > >     LOG.info("Upload received");
>> > >
>> > >     LOG.info("Exchange body: " + exchange.getIn().getBody());
>> > >     LOG.info("Attachment size: " + (attachments == null ? 0 :
>> > > attachments.size()));
>> > >     LOG.info("Attachment object size: " +
>> (exchange.getIn().getAttachmentObjects()
>> > > == null ? 0
>> > > : exchange.getIn().getAttachmentObjects().size()));
>> > >     if (exchange.getIn().getAttachmentNames() != null) {
>> > >       for (String name : exchange.getIn().getAttachmentNames()) {
>> > >         LOG.info("Attachment name: " + name);
>> > >       }
>> > >     }
>> > >
>> > >     if (contentType == null || !contentType.startsWith(MediaT
>> > > ype.MULTIPART_FORM_DATA_VALUE)) {
>> > >       LOG.warn("Unsupported media type!");
>> > >       throw new UnSupportedUploadMediaTypeException("Content-Type
>> has to
>> > > be 'multipart/form-data'");
>> > >     }
>> > >
>> > >     if (attachments.size() == 0) {
>> > >       LOG.warn("No attachments found!");
>> > >     } else {
>> > >       for (String key : attachments.keys()) {
>> > >         LOG.info("Filename: " + key);
>> > >
>> > >         String uploadKey = id + "_" + new
>> Date().toInstant().toEpochMilli()
>> > > + "_" +
>> > > attachments.get(key).getDataSource().getName();
>> > >
>> > >         // stream data directly to a file to save memory footprint
>> > >         File targetFile = new File(uploadKey);
>> > >         try (OutputStream outStream = new FileOutputStream(targetFile,
>> > > false)) {
>> > >             attachments.get(key).writeTo(outStream);
>> > >         }
>> > >
>> > >         ...
>> > >       }
>> > >     }
>> > >
>> > >     ...
>> > >   }
>> > > }
>> > >
>> > > Although we currently test Undertow, switching it with Jetty
>> shouldn't be
>> > > an issue.
>> > >
>> > > HTH,
>> > > roman
>> > >
>> > >
>> > >
>> > > Am 01.10.2017 um 02:18 schrieb Mark:
>> > >
>> > >> I understand that, problem is that I can't figure out how to
>> configure the
>> > >> Camel Route to properly receive/parse the data.  If I was receiving
>> > >> JSON/KML, this would be easy using the functionality in Camel.
>> Binary
>> > >> files seem to be totally different.
>> > >>
>> > >>
>> > >> On Sat, Sep 30, 2017 at 8:13 PM, Mark Nuttall <[hidden email]
>> <x-msg://5/user/SendEmail.jtp?type=node&node=5814065&i=1>>
>> > >> wrote:
>> > >>
>> > >> it is just a file. any example of processing a file should work.
>> you will
>> > >>> be able to save it somewhere and then you will have to call some
>> > >>> processor
>> > >>> to read/process it.
>> > >>>
>> > >>> On Fri, Sep 29, 2017 at 7:47 PM, Mark <[hidden email]
>> <x-msg://5/user/SendEmail.jtp?type=node&node=5814065&i=2>> wrote:
>> > >>>
>> > >>> I'm trying to figure out how to process a binary file that is sent
>> to my
>> > >>>> Camel REST service via POST.  I have not found any good examples
>> on how
>> > >>>> this can be done using Camel.  Does anyone have any experiences
>> they
>> > >>>>
>> > >>> could
>> > >>>
>> > >>>> share or links to more documentation?
>> > >>>>
>> > >>>> I'm currently using the jetty component for the restconfiguration.
>> > >>>>
>> > >>>> Thanks,
>> > >>>> Mark
>> > >>>>
>> > >>>>
>> > >
>> >
>> >
>> > If you reply to this email, your message will be added to the
>> discussion below:
>> > http://camel.465427.n5.nabble.com/REST-DSL-process-multipart
>> -form-data-binary-file-tp5814059p5814065.html <
>> http://camel.465427.n5.nabble.com/REST-DSL-process-multipar
>> t-form-data-binary-file-tp5814059p5814065.html>
>> > To unsubscribe from Camel - Users, click here <
>> http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?
>> macro=unsubscribe_by_code&node=465428&code=cm92b0BnbXguY
>> XR8NDY1NDI4fC04OTkzMjQ3MzM=>.
>> > NAML <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?
>> macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&
>> base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.
>> template.NabbleNamespace-nabble.view.web.template.NodeNamesp
>> ace&breadcrumbs=notify_subscribers%21nabble%3Aemail.
>> naml-instant_emails%21nabble%3Aemail.naml-send_instant_
>> email%21nabble%3Aemail.naml>
>>
>
>

Re: REST DSL process multipart/form-data binary file?

Posted by Mark <el...@gmail.com>.
I'm sending the data right now using the Postman app in Chrome.

My data comes across looking like this:

------WebKitFormBoundaryNZwE5GrXaeBuK9I9

Content-Disposition: form-data; name="fileUpload"; filename="nhc.kmz"
Content-Type: application/vnd.google-earth.kmz

[binary data]

------WebKitFormBoundaryNZwE5GrXaeBuK9I9
Content-Disposition: form-data; name="header1"

value1

------WebKitFormBoundaryNZwE5GrXaeBuK9I9--



On Mon, Oct 2, 2017 at 5:41 AM, Roman Vottner <ro...@gmx.at> wrote:

> How do you send data to your application? Have you specified a
> Content-Disposition header for the respective parts?
>
> We analysed a raw request via wireshark and it does look something like
> this:
>
> GET /api/someResource HTTP/1.1
> Host: www.somewhere.com <http://www.somewhere.com/>
> …
> Content-Type: multipart/form-data; boundary=---------------------
> ---d9c950f09cd11bba
>
> --------------------------d9c950f09cd11bba
> Content-Disposition: form-data; name="dat1"; filename=„SomeFile.zip"
> Content-Type: application/octet-stream
>
> …
>
> --------------------------d9c950f09cd11bba
> Content-Disposition: form-data; name="dat2"; filename=„SomeOtherFile.zip"
> Content-Type: application/octet-stream
>
> …
>
>
> We use Camel 2.19.0 and the Undertow version Camel
> (camel-undertow-starter) and/or Spring-Boot (spring-boot-starter-undertow)
> includes their dependencies (undertow 1.4.13.FINAL).
>
>
> > Am 02.10.2017 um 04:25 schrieb Mark Webb [via Camel] <
> ml+s465427n5814065h66@n5.nabble.com>:
> >
> > Thanks for the example.  I cannot get your code working though.  I get
> the
> > following:
> >
> > [qtp1422312468-17] HandleUpload                   INFO  Upload received
> > [ qtp1422312468-17] HandleUpload                   INFO  Exchange body:
> > org.apache.camel.converter.stream.InputStreamCache@1cf48789
> > [qtp1422312468-17] HandleUpload                   INFO  Attachment size:
> 0
> > [qtp1422312468-17] HandleUpload                   INFO  Attachment object
> > size: 0
> > [qtp1422312468-17] HandleUpload                   WARN  No attachments
> > found!
> >
> > I am testing with the Postman app in the Chrome Browser and using Jetty.
> > I'm using Camel 2.16.2, since we're deploying in ServiceMix.   I tried
> > testing with Camel 2.19.3 and observed the same problem so I don't think
> a
> > newer version of Camel fixes things.  What environment are you running
> your
> > code inside?
> >
> >
> >
> >
> >
> > On Sat, Sep 30, 2017 at 9:13 PM, Roman Vottner <[hidden email]
> <x-msg://5/user/SendEmail.jtp?type=node&node=5814065&i=0>> wrote:
> >
> > > Our Camel REST DSL configuration looks like this:
> > >
> > >     onException(Exception.class)
> > >         .handled(true)
> > >         .logExhausted(false)
> > >         .log(LoggingLevel.ERROR, ">> In default/message exception
> handler.
> > > ${exception}, message: ${exception.message}, stacktrace:
> > > ${exception.stacktrace}")
> > >         .onRedelivery((Exchange exchange) -> LOG.debug(">> Default
> > > exception handler"))
> > >         .bean(PrepareErrorResponse.class).id("ErrorResponse")
> > >         .end();
> > >
> > >     restConfiguration()
> > >         .component("undertow")
> > >         .port("{{server.port}}")
> > >         .contextPath("/api")
> > >         .endpointProperty("matchOnUriPrefix", "true")
> > >         .endpointProperty("sendServerVersion", "false")
> > >         .endpointProperty("chunked", "true");
> > >
> > > where the actual route handling the multipart-request does look
> something
> > > along the line:
> > >
> > >     rest("/someEndpoint")
> > >         .post("/{id}")
> > >         .consumes(MediaType.MULTIPART_FORM_DATA_VALUE)
> > >         .route().routeId("upload-multipart-route")
> > >           // Spring Security authentication check via Authorization
> header
> > >           .bean(SpringSecurityContextLoader.class)
> > >               .policy(authorizationPolicy)
> > >           .log("Uploading to resource id ${header.id}")
> > >           .log(LoggingLevel.INFO, LOG, CONFIDENTIAL,"Headers:
> ${headers}")
> > >           .bean(HandleUpload.class)
> > >           .process((Exchange exchange) -> {
> > >             exchange.getIn().setHeader(Exchange.HTTP_RESPONSE_CODE,
> 200);
> > >             exchange.getIn().setHeader(Exchange.CONTENT_TYPE,
> > > MediaType.APPLICATION_JSON_UTF8_VALUE);
> > >             ...
> > >           })
> > >     .endRest();
> > >
> > > and the HandleUpload class looking something like this:
> > >
> > > public class HandleUpload {
> > >
> > >   private static final Logger LOG = LoggerFactory.getLogger(Method
> > > Handles.lookup().lookupClass());
> > >
> > >   @Autowired
> > >   private SomeRepository someRepository;
> > >
> > >   @Handler
> > >   public void processUpload(@Attachments Map<String, DataHandler>
> > > attachments,
> > >                             @Header("Content-Type") String contentType,
> > >                             @Header("id") String id,
> > >                             Exchange exchange)
> > >       throws Exception {
> > >     LOG.info("Upload received");
> > >
> > >     LOG.info("Exchange body: " + exchange.getIn().getBody());
> > >     LOG.info("Attachment size: " + (attachments == null ? 0 :
> > > attachments.size()));
> > >     LOG.info("Attachment object size: " + (exchange.getIn().
> getAttachmentObjects()
> > > == null ? 0
> > > : exchange.getIn().getAttachmentObjects().size()));
> > >     if (exchange.getIn().getAttachmentNames() != null) {
> > >       for (String name : exchange.getIn().getAttachmentNames()) {
> > >         LOG.info("Attachment name: " + name);
> > >       }
> > >     }
> > >
> > >     if (contentType == null || !contentType.startsWith(MediaT
> > > ype.MULTIPART_FORM_DATA_VALUE)) {
> > >       LOG.warn("Unsupported media type!");
> > >       throw new UnSupportedUploadMediaTypeException("Content-Type has
> to
> > > be 'multipart/form-data'");
> > >     }
> > >
> > >     if (attachments.size() == 0) {
> > >       LOG.warn("No attachments found!");
> > >     } else {
> > >       for (String key : attachments.keys()) {
> > >         LOG.info("Filename: " + key);
> > >
> > >         String uploadKey = id + "_" + new Date().toInstant().
> toEpochMilli()
> > > + "_" +
> > > attachments.get(key).getDataSource().getName();
> > >
> > >         // stream data directly to a file to save memory footprint
> > >         File targetFile = new File(uploadKey);
> > >         try (OutputStream outStream = new FileOutputStream(targetFile,
> > > false)) {
> > >             attachments.get(key).writeTo(outStream);
> > >         }
> > >
> > >         ...
> > >       }
> > >     }
> > >
> > >     ...
> > >   }
> > > }
> > >
> > > Although we currently test Undertow, switching it with Jetty shouldn't
> be
> > > an issue.
> > >
> > > HTH,
> > > roman
> > >
> > >
> > >
> > > Am 01.10.2017 um 02:18 schrieb Mark:
> > >
> > >> I understand that, problem is that I can't figure out how to
> configure the
> > >> Camel Route to properly receive/parse the data.  If I was receiving
> > >> JSON/KML, this would be easy using the functionality in Camel.  Binary
> > >> files seem to be totally different.
> > >>
> > >>
> > >> On Sat, Sep 30, 2017 at 8:13 PM, Mark Nuttall <[hidden email]
> <x-msg://5/user/SendEmail.jtp?type=node&node=5814065&i=1>>
> > >> wrote:
> > >>
> > >> it is just a file. any example of processing a file should work.  you
> will
> > >>> be able to save it somewhere and then you will have to call some
> > >>> processor
> > >>> to read/process it.
> > >>>
> > >>> On Fri, Sep 29, 2017 at 7:47 PM, Mark <[hidden email]
> <x-msg://5/user/SendEmail.jtp?type=node&node=5814065&i=2>> wrote:
> > >>>
> > >>> I'm trying to figure out how to process a binary file that is sent
> to my
> > >>>> Camel REST service via POST.  I have not found any good examples on
> how
> > >>>> this can be done using Camel.  Does anyone have any experiences they
> > >>>>
> > >>> could
> > >>>
> > >>>> share or links to more documentation?
> > >>>>
> > >>>> I'm currently using the jetty component for the restconfiguration.
> > >>>>
> > >>>> Thanks,
> > >>>> Mark
> > >>>>
> > >>>>
> > >
> >
> >
> > If you reply to this email, your message will be added to the discussion
> below:
> > http://camel.465427.n5.nabble.com/REST-DSL-process-
> multipart-form-data-binary-file-tp5814059p5814065.html <
> http://camel.465427.n5.nabble.com/REST-DSL-process-
> multipart-form-data-binary-file-tp5814059p5814065.html>
> > To unsubscribe from Camel - Users, click here <http://camel.465427.n5.
> nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=
> 465428&code=cm92b0BnbXguYXR8NDY1NDI4fC04OTkzMjQ3MzM=>.
> > NAML <http://camel.465427.n5.nabble.com/template/
> NamlServlet.jtp?macro=macro_viewer&id=instant_html%
> 21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> BasicNamespace-nabble.view.web.template.NabbleNamespace-
> nabble.view.web.template.NodeNamespace&breadcrumbs=
> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>

Re: REST DSL process multipart/form-data binary file?

Posted by Roman Vottner <ro...@gmx.at>.
How do you send data to your application? Have you specified a Content-Disposition header for the respective parts?

We analysed a raw request via wireshark and it does look something like this:

GET /api/someResource HTTP/1.1
Host: www.somewhere.com <http://www.somewhere.com/>
…
Content-Type: multipart/form-data; boundary=------------------------d9c950f09cd11bba

--------------------------d9c950f09cd11bba
Content-Disposition: form-data; name="dat1"; filename=„SomeFile.zip"
Content-Type: application/octet-stream

…

--------------------------d9c950f09cd11bba
Content-Disposition: form-data; name="dat2"; filename=„SomeOtherFile.zip"
Content-Type: application/octet-stream

…


We use Camel 2.19.0 and the Undertow version Camel (camel-undertow-starter) and/or Spring-Boot (spring-boot-starter-undertow) includes their dependencies (undertow 1.4.13.FINAL). 


> Am 02.10.2017 um 04:25 schrieb Mark Webb [via Camel] <ml...@n5.nabble.com>:
> 
> Thanks for the example.  I cannot get your code working though.  I get the 
> following: 
> 
> [qtp1422312468-17] HandleUpload                   INFO  Upload received 
> [ qtp1422312468-17] HandleUpload                   INFO  Exchange body: 
> org.apache.camel.converter.stream.InputStreamCache@1cf48789 
> [qtp1422312468-17] HandleUpload                   INFO  Attachment size: 0 
> [qtp1422312468-17] HandleUpload                   INFO  Attachment object 
> size: 0 
> [qtp1422312468-17] HandleUpload                   WARN  No attachments 
> found! 
> 
> I am testing with the Postman app in the Chrome Browser and using Jetty. 
> I'm using Camel 2.16.2, since we're deploying in ServiceMix.   I tried 
> testing with Camel 2.19.3 and observed the same problem so I don't think a 
> newer version of Camel fixes things.  What environment are you running your 
> code inside? 
> 
> 
> 
> 
> 
> On Sat, Sep 30, 2017 at 9:13 PM, Roman Vottner <[hidden email] <x-msg://5/user/SendEmail.jtp?type=node&node=5814065&i=0>> wrote: 
> 
> > Our Camel REST DSL configuration looks like this: 
> > 
> >     onException(Exception.class) 
> >         .handled(true) 
> >         .logExhausted(false) 
> >         .log(LoggingLevel.ERROR, ">> In default/message exception handler. 
> > ${exception}, message: ${exception.message}, stacktrace: 
> > ${exception.stacktrace}") 
> >         .onRedelivery((Exchange exchange) -> LOG.debug(">> Default 
> > exception handler")) 
> >         .bean(PrepareErrorResponse.class).id("ErrorResponse") 
> >         .end(); 
> > 
> >     restConfiguration() 
> >         .component("undertow") 
> >         .port("{{server.port}}") 
> >         .contextPath("/api") 
> >         .endpointProperty("matchOnUriPrefix", "true") 
> >         .endpointProperty("sendServerVersion", "false") 
> >         .endpointProperty("chunked", "true"); 
> > 
> > where the actual route handling the multipart-request does look something 
> > along the line: 
> > 
> >     rest("/someEndpoint") 
> >         .post("/{id}") 
> >         .consumes(MediaType.MULTIPART_FORM_DATA_VALUE) 
> >         .route().routeId("upload-multipart-route") 
> >           // Spring Security authentication check via Authorization header 
> >           .bean(SpringSecurityContextLoader.class) 
> >               .policy(authorizationPolicy) 
> >           .log("Uploading to resource id ${header.id}") 
> >           .log(LoggingLevel.INFO, LOG, CONFIDENTIAL,"Headers: ${headers}") 
> >           .bean(HandleUpload.class) 
> >           .process((Exchange exchange) -> { 
> >             exchange.getIn().setHeader(Exchange.HTTP_RESPONSE_CODE, 200); 
> >             exchange.getIn().setHeader(Exchange.CONTENT_TYPE, 
> > MediaType.APPLICATION_JSON_UTF8_VALUE); 
> >             ... 
> >           }) 
> >     .endRest(); 
> > 
> > and the HandleUpload class looking something like this: 
> > 
> > public class HandleUpload { 
> > 
> >   private static final Logger LOG = LoggerFactory.getLogger(Method 
> > Handles.lookup().lookupClass()); 
> > 
> >   @Autowired 
> >   private SomeRepository someRepository; 
> > 
> >   @Handler 
> >   public void processUpload(@Attachments Map<String, DataHandler> 
> > attachments, 
> >                             @Header("Content-Type") String contentType, 
> >                             @Header("id") String id, 
> >                             Exchange exchange) 
> >       throws Exception { 
> >     LOG.info("Upload received"); 
> > 
> >     LOG.info("Exchange body: " + exchange.getIn().getBody()); 
> >     LOG.info("Attachment size: " + (attachments == null ? 0 : 
> > attachments.size())); 
> >     LOG.info("Attachment object size: " + (exchange.getIn().getAttachmentObjects() 
> > == null ? 0 
> > : exchange.getIn().getAttachmentObjects().size())); 
> >     if (exchange.getIn().getAttachmentNames() != null) { 
> >       for (String name : exchange.getIn().getAttachmentNames()) { 
> >         LOG.info("Attachment name: " + name); 
> >       } 
> >     } 
> > 
> >     if (contentType == null || !contentType.startsWith(MediaT 
> > ype.MULTIPART_FORM_DATA_VALUE)) { 
> >       LOG.warn("Unsupported media type!"); 
> >       throw new UnSupportedUploadMediaTypeException("Content-Type has to 
> > be 'multipart/form-data'"); 
> >     } 
> > 
> >     if (attachments.size() == 0) { 
> >       LOG.warn("No attachments found!"); 
> >     } else { 
> >       for (String key : attachments.keys()) { 
> >         LOG.info("Filename: " + key); 
> > 
> >         String uploadKey = id + "_" + new Date().toInstant().toEpochMilli() 
> > + "_" + 
> > attachments.get(key).getDataSource().getName(); 
> > 
> >         // stream data directly to a file to save memory footprint 
> >         File targetFile = new File(uploadKey); 
> >         try (OutputStream outStream = new FileOutputStream(targetFile, 
> > false)) { 
> >             attachments.get(key).writeTo(outStream); 
> >         } 
> > 
> >         ... 
> >       } 
> >     } 
> > 
> >     ... 
> >   } 
> > } 
> > 
> > Although we currently test Undertow, switching it with Jetty shouldn't be 
> > an issue. 
> > 
> > HTH, 
> > roman 
> > 
> > 
> > 
> > Am 01.10.2017 um 02:18 schrieb Mark: 
> > 
> >> I understand that, problem is that I can't figure out how to configure the 
> >> Camel Route to properly receive/parse the data.  If I was receiving 
> >> JSON/KML, this would be easy using the functionality in Camel.  Binary 
> >> files seem to be totally different. 
> >> 
> >> 
> >> On Sat, Sep 30, 2017 at 8:13 PM, Mark Nuttall <[hidden email] <x-msg://5/user/SendEmail.jtp?type=node&node=5814065&i=1>> 
> >> wrote: 
> >> 
> >> it is just a file. any example of processing a file should work.  you will 
> >>> be able to save it somewhere and then you will have to call some 
> >>> processor 
> >>> to read/process it. 
> >>> 
> >>> On Fri, Sep 29, 2017 at 7:47 PM, Mark <[hidden email] <x-msg://5/user/SendEmail.jtp?type=node&node=5814065&i=2>> wrote: 
> >>> 
> >>> I'm trying to figure out how to process a binary file that is sent to my 
> >>>> Camel REST service via POST.  I have not found any good examples on how 
> >>>> this can be done using Camel.  Does anyone have any experiences they 
> >>>> 
> >>> could 
> >>> 
> >>>> share or links to more documentation? 
> >>>> 
> >>>> I'm currently using the jetty component for the restconfiguration. 
> >>>> 
> >>>> Thanks, 
> >>>> Mark 
> >>>> 
> >>>> 
> > 
> 
> 
> If you reply to this email, your message will be added to the discussion below:
> http://camel.465427.n5.nabble.com/REST-DSL-process-multipart-form-data-binary-file-tp5814059p5814065.html <http://camel.465427.n5.nabble.com/REST-DSL-process-multipart-form-data-binary-file-tp5814059p5814065.html>
> To unsubscribe from Camel - Users, click here <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=cm92b0BnbXguYXR8NDY1NDI4fC04OTkzMjQ3MzM=>.
> NAML <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>

Re: REST DSL process multipart/form-data binary file?

Posted by Mark <el...@gmail.com>.
Thanks for the example.  I cannot get your code working though.  I get the
following:

[qtp1422312468-17] HandleUpload                   INFO  Upload received
[ qtp1422312468-17] HandleUpload                   INFO  Exchange body:
org.apache.camel.converter.stream.InputStreamCache@1cf48789
[qtp1422312468-17] HandleUpload                   INFO  Attachment size: 0
[qtp1422312468-17] HandleUpload                   INFO  Attachment object
size: 0
[qtp1422312468-17] HandleUpload                   WARN  No attachments
found!

I am testing with the Postman app in the Chrome Browser and using Jetty.
I'm using Camel 2.16.2, since we're deploying in ServiceMix.   I tried
testing with Camel 2.19.3 and observed the same problem so I don't think a
newer version of Camel fixes things.  What environment are you running your
code inside?





On Sat, Sep 30, 2017 at 9:13 PM, Roman Vottner <ro...@gmx.at> wrote:

> Our Camel REST DSL configuration looks like this:
>
>     onException(Exception.class)
>         .handled(true)
>         .logExhausted(false)
>         .log(LoggingLevel.ERROR, ">> In default/message exception handler.
> ${exception}, message: ${exception.message}, stacktrace:
> ${exception.stacktrace}")
>         .onRedelivery((Exchange exchange) -> LOG.debug(">> Default
> exception handler"))
>         .bean(PrepareErrorResponse.class).id("ErrorResponse")
>         .end();
>
>     restConfiguration()
>         .component("undertow")
>         .port("{{server.port}}")
>         .contextPath("/api")
>         .endpointProperty("matchOnUriPrefix", "true")
>         .endpointProperty("sendServerVersion", "false")
>         .endpointProperty("chunked", "true");
>
> where the actual route handling the multipart-request does look something
> along the line:
>
>     rest("/someEndpoint")
>         .post("/{id}")
>         .consumes(MediaType.MULTIPART_FORM_DATA_VALUE)
>         .route().routeId("upload-multipart-route")
>           // Spring Security authentication check via Authorization header
>           .bean(SpringSecurityContextLoader.class)
>               .policy(authorizationPolicy)
>           .log("Uploading to resource id ${header.id}")
>           .log(LoggingLevel.INFO, LOG, CONFIDENTIAL,"Headers: ${headers}")
>           .bean(HandleUpload.class)
>           .process((Exchange exchange) -> {
>             exchange.getIn().setHeader(Exchange.HTTP_RESPONSE_CODE, 200);
>             exchange.getIn().setHeader(Exchange.CONTENT_TYPE,
> MediaType.APPLICATION_JSON_UTF8_VALUE);
>             ...
>           })
>     .endRest();
>
> and the HandleUpload class looking something like this:
>
> public class HandleUpload {
>
>   private static final Logger LOG = LoggerFactory.getLogger(Method
> Handles.lookup().lookupClass());
>
>   @Autowired
>   private SomeRepository someRepository;
>
>   @Handler
>   public void processUpload(@Attachments Map<String, DataHandler>
> attachments,
>                             @Header("Content-Type") String contentType,
>                             @Header("id") String id,
>                             Exchange exchange)
>       throws Exception {
>     LOG.info("Upload received");
>
>     LOG.info("Exchange body: " + exchange.getIn().getBody());
>     LOG.info("Attachment size: " + (attachments == null ? 0 :
> attachments.size()));
>     LOG.info("Attachment object size: " + (exchange.getIn().getAttachmentObjects()
> == null ? 0
> : exchange.getIn().getAttachmentObjects().size()));
>     if (exchange.getIn().getAttachmentNames() != null) {
>       for (String name : exchange.getIn().getAttachmentNames()) {
>         LOG.info("Attachment name: " + name);
>       }
>     }
>
>     if (contentType == null || !contentType.startsWith(MediaT
> ype.MULTIPART_FORM_DATA_VALUE)) {
>       LOG.warn("Unsupported media type!");
>       throw new UnSupportedUploadMediaTypeException("Content-Type has to
> be 'multipart/form-data'");
>     }
>
>     if (attachments.size() == 0) {
>       LOG.warn("No attachments found!");
>     } else {
>       for (String key : attachments.keys()) {
>         LOG.info("Filename: " + key);
>
>         String uploadKey = id + "_" + new Date().toInstant().toEpochMilli()
> + "_" +
> attachments.get(key).getDataSource().getName();
>
>         // stream data directly to a file to save memory footprint
>         File targetFile = new File(uploadKey);
>         try (OutputStream outStream = new FileOutputStream(targetFile,
> false)) {
>             attachments.get(key).writeTo(outStream);
>         }
>
>         ...
>       }
>     }
>
>     ...
>   }
> }
>
> Although we currently test Undertow, switching it with Jetty shouldn't be
> an issue.
>
> HTH,
> roman
>
>
>
> Am 01.10.2017 um 02:18 schrieb Mark:
>
>> I understand that, problem is that I can't figure out how to configure the
>> Camel Route to properly receive/parse the data.  If I was receiving
>> JSON/KML, this would be easy using the functionality in Camel.  Binary
>> files seem to be totally different.
>>
>>
>> On Sat, Sep 30, 2017 at 8:13 PM, Mark Nuttall <mk...@gmail.com>
>> wrote:
>>
>> it is just a file. any example of processing a file should work.  you will
>>> be able to save it somewhere and then you will have to call some
>>> processor
>>> to read/process it.
>>>
>>> On Fri, Sep 29, 2017 at 7:47 PM, Mark <el...@gmail.com> wrote:
>>>
>>> I'm trying to figure out how to process a binary file that is sent to my
>>>> Camel REST service via POST.  I have not found any good examples on how
>>>> this can be done using Camel.  Does anyone have any experiences they
>>>>
>>> could
>>>
>>>> share or links to more documentation?
>>>>
>>>> I'm currently using the jetty component for the restconfiguration.
>>>>
>>>> Thanks,
>>>> Mark
>>>>
>>>>
>

Re: REST DSL process multipart/form-data binary file?

Posted by Roman Vottner <ro...@gmx.at>.
Our Camel REST DSL configuration looks like this:

     onException(Exception.class)
         .handled(true)
         .logExhausted(false)
         .log(LoggingLevel.ERROR, ">> In default/message exception 
handler. ${exception}, message: ${exception.message}, stacktrace: 
${exception.stacktrace}")
         .onRedelivery((Exchange exchange) -> LOG.debug(">> Default 
exception handler"))
         .bean(PrepareErrorResponse.class).id("ErrorResponse")
         .end();

     restConfiguration()
         .component("undertow")
         .port("{{server.port}}")
         .contextPath("/api")
         .endpointProperty("matchOnUriPrefix", "true")
         .endpointProperty("sendServerVersion", "false")
         .endpointProperty("chunked", "true");

where the actual route handling the multipart-request does look 
something along the line:

     rest("/someEndpoint")
         .post("/{id}")
         .consumes(MediaType.MULTIPART_FORM_DATA_VALUE)
         .route().routeId("upload-multipart-route")
           // Spring Security authentication check via Authorization header
           .bean(SpringSecurityContextLoader.class)
               .policy(authorizationPolicy)
           .log("Uploading to resource id ${header.id}")
           .log(LoggingLevel.INFO, LOG, CONFIDENTIAL,"Headers: ${headers}")
           .bean(HandleUpload.class)
           .process((Exchange exchange) -> {
             exchange.getIn().setHeader(Exchange.HTTP_RESPONSE_CODE, 200);
             exchange.getIn().setHeader(Exchange.CONTENT_TYPE, 
MediaType.APPLICATION_JSON_UTF8_VALUE);
             ...
           })
     .endRest();

and the HandleUpload class looking something like this:

public class HandleUpload {

   private static final Logger LOG = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());

   @Autowired
   private SomeRepository someRepository;

   @Handler
   public void processUpload(@Attachments Map<String, DataHandler> 
attachments,
                             @Header("Content-Type") String contentType,
                             @Header("id") String id,
                             Exchange exchange)
       throws Exception {
     LOG.info("Upload received");

     LOG.info("Exchange body: " + exchange.getIn().getBody());
     LOG.info("Attachment size: " + (attachments == null ? 0 : 
attachments.size()));
     LOG.info("Attachment object size: " + 
(exchange.getIn().getAttachmentObjects() == null ? 0
: exchange.getIn().getAttachmentObjects().size()));
     if (exchange.getIn().getAttachmentNames() != null) {
       for (String name : exchange.getIn().getAttachmentNames()) {
         LOG.info("Attachment name: " + name);
       }
     }

     if (contentType == null || 
!contentType.startsWith(MediaType.MULTIPART_FORM_DATA_VALUE)) {
       LOG.warn("Unsupported media type!");
       throw new UnSupportedUploadMediaTypeException("Content-Type has 
to be 'multipart/form-data'");
     }

     if (attachments.size() == 0) {
       LOG.warn("No attachments found!");
     } else {
       for (String key : attachments.keys()) {
         LOG.info("Filename: " + key);

         String uploadKey = id + "_" + new 
Date().toInstant().toEpochMilli() + "_" +
attachments.get(key).getDataSource().getName();

         // stream data directly to a file to save memory footprint
         File targetFile = new File(uploadKey);
         try (OutputStream outStream = new FileOutputStream(targetFile, 
false)) {
             attachments.get(key).writeTo(outStream);
         }

         ...
       }
     }

     ...
   }
}

Although we currently test Undertow, switching it with Jetty shouldn't 
be an issue.

HTH,
roman


Am 01.10.2017 um 02:18 schrieb Mark:
> I understand that, problem is that I can't figure out how to configure the
> Camel Route to properly receive/parse the data.  If I was receiving
> JSON/KML, this would be easy using the functionality in Camel.  Binary
> files seem to be totally different.
>
>
> On Sat, Sep 30, 2017 at 8:13 PM, Mark Nuttall <mk...@gmail.com> wrote:
>
>> it is just a file. any example of processing a file should work.  you will
>> be able to save it somewhere and then you will have to call some processor
>> to read/process it.
>>
>> On Fri, Sep 29, 2017 at 7:47 PM, Mark <el...@gmail.com> wrote:
>>
>>> I'm trying to figure out how to process a binary file that is sent to my
>>> Camel REST service via POST.  I have not found any good examples on how
>>> this can be done using Camel.  Does anyone have any experiences they
>> could
>>> share or links to more documentation?
>>>
>>> I'm currently using the jetty component for the restconfiguration.
>>>
>>> Thanks,
>>> Mark
>>>


Re: REST DSL process multipart/form-data binary file?

Posted by Mark <el...@gmail.com>.
I understand that, problem is that I can't figure out how to configure the
Camel Route to properly receive/parse the data.  If I was receiving
JSON/KML, this would be easy using the functionality in Camel.  Binary
files seem to be totally different.


On Sat, Sep 30, 2017 at 8:13 PM, Mark Nuttall <mk...@gmail.com> wrote:

> it is just a file. any example of processing a file should work.  you will
> be able to save it somewhere and then you will have to call some processor
> to read/process it.
>
> On Fri, Sep 29, 2017 at 7:47 PM, Mark <el...@gmail.com> wrote:
>
> > I'm trying to figure out how to process a binary file that is sent to my
> > Camel REST service via POST.  I have not found any good examples on how
> > this can be done using Camel.  Does anyone have any experiences they
> could
> > share or links to more documentation?
> >
> > I'm currently using the jetty component for the restconfiguration.
> >
> > Thanks,
> > Mark
> >
>