You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Norman Walsh <nd...@nwalsh.com> on 2019/06/01 17:58:03 UTC
Constructing multipart/mixed requests
Hi all,
Apologies if I’ve been down this road before on this list. The
HttpClient 4.5 library has changed the way multipart works (from some
earlier 4.x where I had it working).
Near as I can tell from looking at the multipart examples in 4.5,
they’re all geared towards file uploading. I’m not interested in file
uploading, I want to construct a payload for a web service that’s
expecting a multipart/mixed request: I want complete control over the
parts, the headers associated with those parts, their content types,
etc.
I stumbled from HttpClient to MIME4J, prehaps on the advice of someone
from this list. I got a little further that way, at least in as much
as I now believe I’ve constructed a mime4j.message that is logically
what I want to send.
But I cannot see how to get from that back to an HttpEntity that I can
set as the entity for an HttpClient request.
Does anyone have an example of sending a fully general multipart/mixed
example? Or is anyone familiar enough with both HttpClient and MIME4J
to point me to an explanation of how to turn one of those into a
request Entity?
(Multipart is only one possibility so I’d really, really rather not
have to have two entirely different code paths where I use HttpClient
for some requests and use direct serialization of MIME4J payloads over
a URLConnection for the other.)
Help and advice most humbly solicited.
Be seeing you,
norm
--
Norman Walsh <nd...@nwalsh.com> | Why do writers write? Because it isn't
http://nwalsh.com/ | there.--Thomas Berger
Re: Constructing multipart/mixed requests
Posted by Adam Retter <ad...@exist-db.org>.
Thanks, I thought that GitHub was just a mirror for an Apache repo somewhere!
I have opened a PR here -
https://github.com/apache/httpcomponents-client/pull/151
On Sun, 2 Jun 2019 at 15:43, Oleg Kalnichevski <ol...@ok2consulting.com> wrote:
>
> On June 2, 2019 4:30:44 PM GMT+02:00, Adam Retter <ad...@exist-db.org> wrote:
> >Oleg,
> >
> >Attached is a first draft for a patch against 5.0 to make the
> >Multipart stuff more applicable for those people who don't want
> >multipart/form-data. What would be the next stage to progress this?
> >
>
> Could you please submit this change-set as a PR at Github?
>
> Oleg
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
--
Adam Retter
eXist Core Developer
{ United Kingdom }
adam@exist-db.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Constructing multipart/mixed requests
Posted by Oleg Kalnichevski <ol...@ok2consulting.com>.
On June 2, 2019 4:30:44 PM GMT+02:00, Adam Retter <ad...@exist-db.org> wrote:
>Oleg,
>
>Attached is a first draft for a patch against 5.0 to make the
>Multipart stuff more applicable for those people who don't want
>multipart/form-data. What would be the next stage to progress this?
>
Could you please submit this change-set as a PR at Github?
Oleg
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Constructing multipart/mixed requests
Posted by Adam Retter <ad...@exist-db.org>.
Oleg,
Attached is a first draft for a patch against 5.0 to make the
Multipart stuff more applicable for those people who don't want
multipart/form-data. What would be the next stage to progress this?
On Sun, 2 Jun 2019 at 11:23, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> On Sun, 2019-06-02 at 09:47 +0100, Adam Retter wrote:
> > > > On my system that produces a HTTP Request like:
> > > >
> > > > POST / HTTP/1.1
> > > > Content-Length: 456
> > > > Content-Type: multipart/form-data;
> > > > boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
> > >
> > > I can change this content-type by calling .setContentType() on the
> > > entity builder,
> >
> > Ah yes! I forgot to include that, sorry.
> >
> >
> > > however:
> > >
> > > > Content-Disposition: form-data; name="part1"
> > >
> > > Every part gets this content-disposition which is clearly bogus for
> > > non-form multipart messages.
> >
> > I missed that somehow when I was checking the output. The closest I
> > think you could get is to override the Content-Disposition in each
> > part by using:
> >
> > addField("Content-Disposition", "inline")
> >
> > Whether you should use `inline` or `attachment` I cannot say, as I
> > don't know enough about your use-case.
> >
> > Another option, which would be cleaner would be to to derive your own
> > builder class from org.apache.http.entity.mime.FormBodyPartBuilder.
> > Having studied the code of that class, it looks to me like it already
> > does 95% of what you need, and you could just modify your version of
> > the `build` method, to not do the Content-Disposition stuff.
> >
> > Whether that class stays stable over time I cannot say, I agree with
> > you that it looks like the HTTP Client is missing an easy way to
> > cleanly do multipart. It would seem to me that a
> > org.apache.http.entity.mime.MultiPartBuilder would make sense, from
> > which org.apache.http.entity.mime.FormBodyPartBuilder is subclassed.
> >
> > Cheers Adam.
> >
>
> Adam, Norman, et al.
>
> 12 years ago the plan was to use Apache Mime4j once its APIs got
> frozen. The existing multipart code was initially intended as a throw-
> away stop-gap fix.
>
> It looks increasingly likely Mime4j 1.0 will never get released in our
> life span, but it is still infinitely more useful and flexible than
> what HttpClient has to offer.
>
> Having said all that, there is still time to get onboard and fix
> whatever you deem in need of fixing in 5.0. Time is running out,
> though, as we are looking at freezing 5.0 APIs soon.
>
> By the way, MIME spec refers to MIME headers as `header fields`, so the
> choice of method names was not completely random.
>
> Cheers
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
--
Adam Retter
eXist Core Developer
{ United Kingdom }
adam@exist-db.org
Re: Constructing multipart/mixed requests
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2019-06-02 at 09:47 +0100, Adam Retter wrote:
> > > On my system that produces a HTTP Request like:
> > >
> > > POST / HTTP/1.1
> > > Content-Length: 456
> > > Content-Type: multipart/form-data;
> > > boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
> >
> > I can change this content-type by calling .setContentType() on the
> > entity builder,
>
> Ah yes! I forgot to include that, sorry.
>
>
> > however:
> >
> > > Content-Disposition: form-data; name="part1"
> >
> > Every part gets this content-disposition which is clearly bogus for
> > non-form multipart messages.
>
> I missed that somehow when I was checking the output. The closest I
> think you could get is to override the Content-Disposition in each
> part by using:
>
> addField("Content-Disposition", "inline")
>
> Whether you should use `inline` or `attachment` I cannot say, as I
> don't know enough about your use-case.
>
> Another option, which would be cleaner would be to to derive your own
> builder class from org.apache.http.entity.mime.FormBodyPartBuilder.
> Having studied the code of that class, it looks to me like it already
> does 95% of what you need, and you could just modify your version of
> the `build` method, to not do the Content-Disposition stuff.
>
> Whether that class stays stable over time I cannot say, I agree with
> you that it looks like the HTTP Client is missing an easy way to
> cleanly do multipart. It would seem to me that a
> org.apache.http.entity.mime.MultiPartBuilder would make sense, from
> which org.apache.http.entity.mime.FormBodyPartBuilder is subclassed.
>
> Cheers Adam.
>
Adam, Norman, et al.
12 years ago the plan was to use Apache Mime4j once its APIs got
frozen. The existing multipart code was initially intended as a throw-
away stop-gap fix.
It looks increasingly likely Mime4j 1.0 will never get released in our
life span, but it is still infinitely more useful and flexible than
what HttpClient has to offer.
Having said all that, there is still time to get onboard and fix
whatever you deem in need of fixing in 5.0. Time is running out,
though, as we are looking at freezing 5.0 APIs soon.
By the way, MIME spec refers to MIME headers as `header fields`, so the
choice of method names was not completely random.
Cheers
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Constructing multipart/mixed requests
Posted by Adam Retter <ad...@exist-db.org>.
> > On my system that produces a HTTP Request like:
> >
> > POST / HTTP/1.1
> > Content-Length: 456
> > Content-Type: multipart/form-data; boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
>
> I can change this content-type by calling .setContentType() on the entity builder,
Ah yes! I forgot to include that, sorry.
> however:
>
> > Content-Disposition: form-data; name="part1"
>
> Every part gets this content-disposition which is clearly bogus for
> non-form multipart messages.
I missed that somehow when I was checking the output. The closest I
think you could get is to override the Content-Disposition in each
part by using:
addField("Content-Disposition", "inline")
Whether you should use `inline` or `attachment` I cannot say, as I
don't know enough about your use-case.
Another option, which would be cleaner would be to to derive your own
builder class from org.apache.http.entity.mime.FormBodyPartBuilder.
Having studied the code of that class, it looks to me like it already
does 95% of what you need, and you could just modify your version of
the `build` method, to not do the Content-Disposition stuff.
Whether that class stays stable over time I cannot say, I agree with
you that it looks like the HTTP Client is missing an easy way to
cleanly do multipart. It would seem to me that a
org.apache.http.entity.mime.MultiPartBuilder would make sense, from
which org.apache.http.entity.mime.FormBodyPartBuilder is subclassed.
Cheers Adam.
--
Adam Retter
eXist Core Developer
{ United Kingdom }
adam@exist-db.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Constructing multipart/mixed requests
Posted by Norman Walsh <nd...@nwalsh.com>.
Adam Retter <ad...@exist-db.org> writes:
> Hi Norm,
Hi Adam. Thanks for the example; you’ve threaded the needle better than I did.
Maybe it’s good enough, but it’s still not exactly what I was hoping for.
> final FormBodyPart part1 = FormBodyPartBuilder.create()
> .setName("part1")
The requirement that parts have a name is a form requirement, not a
multipart requirement.
> On my system that produces a HTTP Request like:
>
> POST / HTTP/1.1
> Content-Length: 456
> Content-Type: multipart/form-data; boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
I can change this content-type by calling .setContentType() on the entity builder,
however:
> Content-Disposition: form-data; name="part1"
Every part gets this content-disposition which is clearly bogus for
non-form multipart messages.
Bogus but perhaps harmless. I’m going to run with it and see what
happens. I’ll fail all of the XProc multipart tests of course, but…
Be seeing you,
norm
--
Norman Walsh <nd...@nwalsh.com> | So, are you working on finding that bug
http://nwalsh.com/ | now, or are you leaving it until later?
| Yes.
Re: Constructing multipart/mixed requests
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sat, 2019-06-01 at 23:27 +0100, Adam Retter wrote:
> Hi Norm,
>
> I think this might be what you are looking for:
>
> package norm1;
>
> import org.apache.http.HttpEntity;
> import org.apache.http.client.methods.HttpPost;
> import org.apache.http.entity.ContentType;
> import org.apache.http.entity.mime.FormBodyPart;
> import org.apache.http.entity.mime.FormBodyPartBuilder;
> import org.apache.http.entity.mime.MultipartEntityBuilder;
> import org.apache.http.entity.mime.content.StringBody;
> import org.apache.http.impl.client.CloseableHttpClient;
> import org.apache.http.impl.client.HttpClientBuilder;
>
> import java.io.IOException;
>
> public class MultipartExample {
>
> public static void main(final String args[]) throws IOException {
>
> try(final CloseableHttpClient client =
> HttpClientBuilder.create().build()) {
>
> final FormBodyPart part1 = FormBodyPartBuilder.create()
> .setName("part1")
> .addField("X-HELLO", "adam")
> .addField("X-HELLO", "norm")
> .setBody(new StringBody("<some-body/>",
> ContentType.TEXT_XML))
> .build();
>
> final FormBodyPart part2 = FormBodyPartBuilder.create()
> .setName("part2")
> .addField("X-BYE", "adam")
> .addField("X-BYE", "norm")
> .setBody(new StringBody("{\"some\": \"json\"}",
> ContentType.APPLICATION_JSON))
> .build();
>
> final HttpEntity entity = MultipartEntityBuilder.create()
> .addPart(part1)
> .addPart(part2)
> .build();
>
>
> final HttpPost post = new HttpPost("http://localhost:8080
> ");
> post.setEntity(entity);
>
> client.execute(post);
> }
> }
> }
>
>
> On my system that produces a HTTP Request like:
>
> POST / HTTP/1.1
> Content-Length: 456
> Content-Type: multipart/form-data;
> boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
> Host: localhost:8080
> Connection: Keep-Alive
> User-Agent: Apache-HttpClient/4.5.8 (Java/1.8.0_202)
> Accept-Encoding: gzip,deflate
>
> --ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
> X-HELLO: adam
> X-HELLO: norm
> Content-Disposition: form-data; name="part1"
> Content-Type: text/xml; charset=ISO-8859-1
> Content-Transfer-Encoding: 8bit
>
> <some-body/>
> --ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
> X-BYE: adam
> X-BYE: norm
> Content-Disposition: form-data; name="part2"
> Content-Type: application/json; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> {"some": "json"}
> --ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb--
>
>
> Personally I find the naming of some of these things
> inaccurate/misleading... i.e. the class `FormBodyPart`, and the
> method
> `addField` (which appears to actually add a header). Regardless it
> seems to "just about work".
>
> Hope that helps?
>
> Kind Regards. Adam.
>
Here's something similar based on Apache Mime4j
---
StorageBodyFactory bodyFactory = new StorageBodyFactory();
Message message = Message.Builder.of()
.setFrom("John Doe <jd...@machine.example>")
.setTo("Mary Smith <ma...@example.net>")
.setSubject("An image for you")
.setDate(new Date())
.generateMessageId(InetAddress.getLocalHost().getCanonicalHostName())
.setBody(MultipartBuilder.create("mixed")
.use(bodyFactory)
.setPreamble("This is a multi-part message in MIME format.")
.addBodyPart(BodyPartBuilder.create()
.use(bodyFactory)
.setBody("Why so serious?", Charsets.UTF_8)
.setContentTransferEncoding("quoted-printable")
.build())
.addBodyPart(BodyPartBuilder.create()
.use(bodyFactory)
.setBody(new byte[] { 1, 2, 3}, "application/octet-stream")
.setContentTransferEncoding("base64")
.setContentDisposition("attachment", "smiley.png")
.build())
.build())
.build();
EntityTemplate entityTemplate = new EntityTemplate(outstream -> {
try {
MessageWriter writer = new DefaultMessageWriter();
writer.writeMessage(message, outstream);
} finally {
message.dispose();
}
});
entityTemplate.setContentType(message.getMimeType());
---
Oleg
> On Sat, 1 Jun 2019 at 18:58, Norman Walsh <nd...@nwalsh.com> wrote:
> >
> > Hi all,
> >
> > Apologies if I’ve been down this road before on this list. The
> > HttpClient 4.5 library has changed the way multipart works (from
> > some
> > earlier 4.x where I had it working).
> >
> > Near as I can tell from looking at the multipart examples in 4.5,
> > they’re all geared towards file uploading. I’m not interested in
> > file
> > uploading, I want to construct a payload for a web service that’s
> > expecting a multipart/mixed request: I want complete control over
> > the
> > parts, the headers associated with those parts, their content
> > types,
> > etc.
> >
> > I stumbled from HttpClient to MIME4J, prehaps on the advice of
> > someone
> > from this list. I got a little further that way, at least in as
> > much
> > as I now believe I’ve constructed a mime4j.message that is
> > logically
> > what I want to send.
> >
> > But I cannot see how to get from that back to an HttpEntity that I
> > can
> > set as the entity for an HttpClient request.
> >
> > Does anyone have an example of sending a fully general
> > multipart/mixed
> > example? Or is anyone familiar enough with both HttpClient and
> > MIME4J
> > to point me to an explanation of how to turn one of those into a
> > request Entity?
> >
> > (Multipart is only one possibility so I’d really, really rather not
> > have to have two entirely different code paths where I use
> > HttpClient
> > for some requests and use direct serialization of MIME4J payloads
> > over
> > a URLConnection for the other.)
> >
> > Help and advice most humbly solicited.
> >
> > Be seeing you,
> > norm
> >
> > --
> > Norman Walsh <nd...@nwalsh.com> | Why do writers write? Because it
> > isn't
> > http://nwalsh.com/m/ | there.--Thomas Berger
>
>
>
> --
> Adam Retter
>
> eXist Core Developer
> { United Kingdom }
> adam@exist-db.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Constructing multipart/mixed requests
Posted by Adam Retter <ad...@exist-db.org>.
Hi Norm,
I think this might be what you are looking for:
package norm1;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.mime.FormBodyPart;
import org.apache.http.entity.mime.FormBodyPartBuilder;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.StringBody;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import java.io.IOException;
public class MultipartExample {
public static void main(final String args[]) throws IOException {
try(final CloseableHttpClient client =
HttpClientBuilder.create().build()) {
final FormBodyPart part1 = FormBodyPartBuilder.create()
.setName("part1")
.addField("X-HELLO", "adam")
.addField("X-HELLO", "norm")
.setBody(new StringBody("<some-body/>",
ContentType.TEXT_XML))
.build();
final FormBodyPart part2 = FormBodyPartBuilder.create()
.setName("part2")
.addField("X-BYE", "adam")
.addField("X-BYE", "norm")
.setBody(new StringBody("{\"some\": \"json\"}",
ContentType.APPLICATION_JSON))
.build();
final HttpEntity entity = MultipartEntityBuilder.create()
.addPart(part1)
.addPart(part2)
.build();
final HttpPost post = new HttpPost("http://localhost:8080");
post.setEntity(entity);
client.execute(post);
}
}
}
On my system that produces a HTTP Request like:
POST / HTTP/1.1
Content-Length: 456
Content-Type: multipart/form-data; boundary=ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
Host: localhost:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.5.8 (Java/1.8.0_202)
Accept-Encoding: gzip,deflate
--ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
X-HELLO: adam
X-HELLO: norm
Content-Disposition: form-data; name="part1"
Content-Type: text/xml; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
<some-body/>
--ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb
X-BYE: adam
X-BYE: norm
Content-Disposition: form-data; name="part2"
Content-Type: application/json; charset=UTF-8
Content-Transfer-Encoding: 8bit
{"some": "json"}
--ZQwFLuoO6V1SFdqg2lI6DaVwlvKIZjj2Pb--
Personally I find the naming of some of these things
inaccurate/misleading... i.e. the class `FormBodyPart`, and the method
`addField` (which appears to actually add a header). Regardless it
seems to "just about work".
Hope that helps?
Kind Regards. Adam.
On Sat, 1 Jun 2019 at 18:58, Norman Walsh <nd...@nwalsh.com> wrote:
>
> Hi all,
>
> Apologies if I’ve been down this road before on this list. The
> HttpClient 4.5 library has changed the way multipart works (from some
> earlier 4.x where I had it working).
>
> Near as I can tell from looking at the multipart examples in 4.5,
> they’re all geared towards file uploading. I’m not interested in file
> uploading, I want to construct a payload for a web service that’s
> expecting a multipart/mixed request: I want complete control over the
> parts, the headers associated with those parts, their content types,
> etc.
>
> I stumbled from HttpClient to MIME4J, prehaps on the advice of someone
> from this list. I got a little further that way, at least in as much
> as I now believe I’ve constructed a mime4j.message that is logically
> what I want to send.
>
> But I cannot see how to get from that back to an HttpEntity that I can
> set as the entity for an HttpClient request.
>
> Does anyone have an example of sending a fully general multipart/mixed
> example? Or is anyone familiar enough with both HttpClient and MIME4J
> to point me to an explanation of how to turn one of those into a
> request Entity?
>
> (Multipart is only one possibility so I’d really, really rather not
> have to have two entirely different code paths where I use HttpClient
> for some requests and use direct serialization of MIME4J payloads over
> a URLConnection for the other.)
>
> Help and advice most humbly solicited.
>
> Be seeing you,
> norm
>
> --
> Norman Walsh <nd...@nwalsh.com> | Why do writers write? Because it isn't
> http://nwalsh.com/ | there.--Thomas Berger
--
Adam Retter
eXist Core Developer
{ United Kingdom }
adam@exist-db.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org