You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Brian Clozel <bc...@pivotal.io> on 2017/10/19 14:11:19 UTC
TLS support: consider bundling native libs in JARs
Hi,
More and more servers are choosing to make available one or more solutions
to use TLS native stacks by shipping them as JARs:
* Netty has quite a few options there
http://netty.io/wiki/forked-tomcat-native.html
* Jetty is now shipping a conscrypt support as well
https://webtide.com/conscrypting-native-ssl-for-jetty/
I know there are other solutions for that, like changing the boot classpath
or installing native libraries directly on the host operating system. But
those solutions aren't always super easy to achieve in cloud environments;
there are also questions on this mailing list around
tomcat+tcnative+openssl versions compatibility.
Would the Tomcat community consider shipping JARs (with classifier and uber
JARs) containing the required native libraries (libtcnative + openssl +
apr)?
Bonus question: would you consider supporting boringssl or libressl?
Thanks,
--
Brian Clozel
Re: TLS support: consider bundling native libs in JARs
Posted by Brian Clozel <bc...@pivotal.io>.
Hi Mark, Christopher,
On Thu, Oct 19, 2017 at 7:32 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Mark,
>
> On 10/19/17 1:22 PM, Mark Thomas wrote:
> > On 19/10/17 16:56, Mark Thomas wrote:
> >> On 19 October 2017 15:11:19 BST, Brian Clozel
> >> <bc...@pivotal.io> wrote:
> >>> Hi,
> >>>
> >>> More and more servers are choosing to make available one or
> >>> more solutions to use TLS native stacks by shipping them as
> >>> JARs:
> >>>
> >>> * Netty has quite a few options there
> >>> http://netty.io/wiki/forked-tomcat-native.html * Jetty is now
> >>> shipping a conscrypt support as well
> >>> https://webtide.com/conscrypting-native-ssl-for-jetty/
> >>>
> >>
> >> How does shipping a native library in a JAR work? What makes it
> >> simpler than building from source?
> >
> > Found the answer to my own question. Netty unpacks the native
> > library into a temporary directory and loads it from there.
> >
> > Packaging in a JAR is simply a convenience to enable end users to
> > use their build tool of choice to pull in the library.
>
> ... and completely unnecessary for users using a packaged Tomcat,
> since part of the "installation process" (either .exe "installer" for
> Windows, or unzip/untar for anyone else) drops any packaged binaries
> in the right place already. The problem is lack of binaries.
>
> > For Tomcat, this would be useful for the embedded scenario.
>
> Yes. The problem with Tomcat self-extracting a native library would be
> that an embedded environment could have all kinds of problems with
> that. If Tomcat extracted a native library to $TMPDIR and then allowed
> the JVM to load from there, wouldn't that scare the hell out of a
> developer or end-user who wasn't expecting that kind of thing? If
> embedded users (developers) want to package Tomcat in that way, that's
> fine, but Tomcat doing it without instructions seems like a horrible
> mistake.
>
> There is room for improving support for such a thing, but having
> Tomcat provide a magic JAR file is something I'm very much -1 on doing.
>
Indeed, it's mostly useful for the embedded scenario, even more for a
12 factor app deployed on a PaaS (disclaimer: I'm a Spring Boot team
member).
In those cases, the typical application is an uber JAR that you expect
to be deployed in many environments - and you can't really expect that
infrastructure
to peek into your app and guess which version of libtcnative/openssl
it should install.
In my opinion, dealing with native libraries "manually" is harder than
using a server that uses the bootclasspath/java agent trick to support
ALPN.
Of course, that's if your primary goal is to support http/2 on JDK8.
Netty, Conscrypt and others extract libs to a temp folder, and from a
Java server application point of view, I don't see how it changes the
trust relationship you already have with the host operating system. At
least doing so would save a lot of pain to all embedded users (setting
up the native libs, choosing the right version set, reproducing the
same conditions on their laptop).
You can then run applications without worrying about the Tomcat
version and locally compiled libraries.
> > The full binary distributions could leverage the same mechanism or
> > they could do something different. That would increase the number
> > of binary builds we needed to do for a release.
>
> ... and therein lies the challenge. We intentionally stopped
> publishing binaries because it's such a PITA. We only produce binaries
> for Microsoft Windows because (a) most Windows environment don't have
> access to a compiler (sadly) and (b) there are only two artifacts to
> produce: amd32 and amd64 builds.
>
> If someone wants to volunteer to build every combination of
> architecture, OS, and web server out there, I'm sure we'd appreciate
> the contribution. Just be aware that it is a slippery slope. Next
> thing you know, James Lampert will be asking us to produce AS/400
> builds. (*ducks*)
>
> - -chris
Now I totally get how this can turn into a distribution nightmare. I
can also see the irony in asking **you** to deal with a build
nightmare when it's **us** being too lazy to compile things :-)
Worse, I'm wondering if packing openssl in a JAR could somehow link a
tomcat version with openssl CVEs...
Would supporting Conscrypt then could be an option? As an external,
optional dependency, you wouldn't have to deal with all of this -
"just" coding against their JSSE provider.
Of course, I have zero idea about the efforts required for that - I
vaguely followed what the Jetty team is currently doing (the current
4.3.8 SNAPSHOTs are already working with http/2 and Conscrypt).
This could be a good alternative for people who can't introduce
libtcnative/openssl or JDK9 in their infrastructure right now - and
could drive http/2 adoption on existing systems.
That comes of course with a price: a heavier JAR if you wish to ship
the conscrypt uberjar with your app.
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlno4boACgkQHPApP6U8
> pFh4QA//RjVkmghJ0xx3WgI/SHvN5md3ywZPKNmMrPEyeL55R4e5g2yVEx/U5UTT
> RGUICEmfhK77oQDu+YnKa95i23xGzv1CTtcE//kZeMYWu/pXaNlfF79dkmTGJqZe
> bUZHCD62Zo7WtvuUVNgDt5tQn0hjq26e26/wE1jgvXk/lG3hE4tCgtH0D58uOyxI
> 1zu76kwaw/RubOVXBLUiSFJP/HM1hRyo0ERPqH/r0BAJv1SxPZ2Ok87wKs6pDfA5
> 0+raXKUmmt/DJ0T+OEEs30C7epqDMrIKW8T8X3H9gBSNU3Gta/Sp2r1G3NMxucJc
> NZapiDRh+ZE42Tb76OCOAAQHtLcdCN8Hz34SKs1x2tq7N6HVd/M3zgKFTwh+g5xZ
> 5MlW98+uYhNIbDSqyf4CMys+5N4yBUfCIxelfduKnelxppGYMDQhadQboNqbZQjE
> NsxRbbYKobn1vMPBoNzy7G4j8NpnnYjk/Vnr52WTP4QKDaXCofM2AE5aHkyUxYge
> 7x+E6EXOKtyj4ky0uMvwhAiBGY4fUlED3I+NiWxMajI0AZVJwA833ZbW+buE0TDA
> gFaE1gwajl9EyrhWeraiSh7ZljnVc7YMdoTJt9nL8WVXPXpahzEn0EWKeiEkMvtN
> QrtNnmgrtnVdNLKAaqFAnlxJzqGK9B5Horh7noDldugAuZNdtqw=
> =tNgg
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Cheers!
--
Brian Clozel
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: TLS support: consider bundling native libs in JARs
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Mark,
On 10/19/17 1:22 PM, Mark Thomas wrote:
> On 19/10/17 16:56, Mark Thomas wrote:
>> On 19 October 2017 15:11:19 BST, Brian Clozel
>> <bc...@pivotal.io> wrote:
>>> Hi,
>>>
>>> More and more servers are choosing to make available one or
>>> more solutions to use TLS native stacks by shipping them as
>>> JARs:
>>>
>>> * Netty has quite a few options there
>>> http://netty.io/wiki/forked-tomcat-native.html * Jetty is now
>>> shipping a conscrypt support as well
>>> https://webtide.com/conscrypting-native-ssl-for-jetty/
>>>
>>
>> How does shipping a native library in a JAR work? What makes it
>> simpler than building from source?
>
> Found the answer to my own question. Netty unpacks the native
> library into a temporary directory and loads it from there.
>
> Packaging in a JAR is simply a convenience to enable end users to
> use their build tool of choice to pull in the library.
... and completely unnecessary for users using a packaged Tomcat,
since part of the "installation process" (either .exe "installer" for
Windows, or unzip/untar for anyone else) drops any packaged binaries
in the right place already. The problem is lack of binaries.
> For Tomcat, this would be useful for the embedded scenario.
Yes. The problem with Tomcat self-extracting a native library would be
that an embedded environment could have all kinds of problems with
that. If Tomcat extracted a native library to $TMPDIR and then allowed
the JVM to load from there, wouldn't that scare the hell out of a
developer or end-user who wasn't expecting that kind of thing? If
embedded users (developers) want to package Tomcat in that way, that's
fine, but Tomcat doing it without instructions seems like a horrible
mistake.
There is room for improving support for such a thing, but having
Tomcat provide a magic JAR file is something I'm very much -1 on doing.
> The full binary distributions could leverage the same mechanism or
> they could do something different. That would increase the number
> of binary builds we needed to do for a release.
... and therein lies the challenge. We intentionally stopped
publishing binaries because it's such a PITA. We only produce binaries
for Microsoft Windows because (a) most Windows environment don't have
access to a compiler (sadly) and (b) there are only two artifacts to
produce: amd32 and amd64 builds.
If someone wants to volunteer to build every combination of
architecture, OS, and web server out there, I'm sure we'd appreciate
the contribution. Just be aware that it is a slippery slope. Next
thing you know, James Lampert will be asking us to produce AS/400
builds. (*ducks*)
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlno4boACgkQHPApP6U8
pFh4QA//RjVkmghJ0xx3WgI/SHvN5md3ywZPKNmMrPEyeL55R4e5g2yVEx/U5UTT
RGUICEmfhK77oQDu+YnKa95i23xGzv1CTtcE//kZeMYWu/pXaNlfF79dkmTGJqZe
bUZHCD62Zo7WtvuUVNgDt5tQn0hjq26e26/wE1jgvXk/lG3hE4tCgtH0D58uOyxI
1zu76kwaw/RubOVXBLUiSFJP/HM1hRyo0ERPqH/r0BAJv1SxPZ2Ok87wKs6pDfA5
0+raXKUmmt/DJ0T+OEEs30C7epqDMrIKW8T8X3H9gBSNU3Gta/Sp2r1G3NMxucJc
NZapiDRh+ZE42Tb76OCOAAQHtLcdCN8Hz34SKs1x2tq7N6HVd/M3zgKFTwh+g5xZ
5MlW98+uYhNIbDSqyf4CMys+5N4yBUfCIxelfduKnelxppGYMDQhadQboNqbZQjE
NsxRbbYKobn1vMPBoNzy7G4j8NpnnYjk/Vnr52WTP4QKDaXCofM2AE5aHkyUxYge
7x+E6EXOKtyj4ky0uMvwhAiBGY4fUlED3I+NiWxMajI0AZVJwA833ZbW+buE0TDA
gFaE1gwajl9EyrhWeraiSh7ZljnVc7YMdoTJt9nL8WVXPXpahzEn0EWKeiEkMvtN
QrtNnmgrtnVdNLKAaqFAnlxJzqGK9B5Horh7noDldugAuZNdtqw=
=tNgg
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: TLS support: consider bundling native libs in JARs
Posted by Mark Thomas <ma...@apache.org>.
On 19/10/17 16:56, Mark Thomas wrote:
> On 19 October 2017 15:11:19 BST, Brian Clozel <bc...@pivotal.io> wrote:
>> Hi,
>>
>> More and more servers are choosing to make available one or more
>> solutions
>> to use TLS native stacks by shipping them as JARs:
>>
>> * Netty has quite a few options there
>> http://netty.io/wiki/forked-tomcat-native.html
>> * Jetty is now shipping a conscrypt support as well
>> https://webtide.com/conscrypting-native-ssl-for-jetty/
>>
>
> How does shipping a native library in a JAR work? What makes it simpler than building from source?
Found the answer to my own question. Netty unpacks the native library
into a temporary directory and loads it from there.
Packaging in a JAR is simply a convenience to enable end users to use
their build tool of choice to pull in the library. For Tomcat, this
would be useful for the embedded scenario. The full binary distributions
could leverage the same mechanism or they could do something different.
That would increase the number of binary builds we needed to do for a
release.
Mark
> (Past experience of providing binaries for OS other than Windows is that the number of different builds rapidly multiplies - hence source only at the moment.)
>
>> I know there are other solutions for that, like changing the boot
>> classpath
>> or installing native libraries directly on the host operating system.
>> But
>> those solutions aren't always super easy to achieve in cloud
>> environments;
>> there are also questions on this mailing list around
>> tomcat+tcnative+openssl versions compatibility.
>>
>> Would the Tomcat community consider shipping JARs (with classifier and
>> uber
>> JARs) containing the required native libraries (libtcnative + openssl +
>> apr)?
>
> In principle no reason why not but more detail on the requirements is needed.
>
>> Bonus question: would you consider supporting boringssl or libressl?
>
> Libressl is supported as of 1.2.13 apart from some features where we need functionality not available in libressl (I forget what those features were but I don't think they were significant)
>
> Boringssl should be doable as well but I don't think anyone has tried it yet.
>
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: TLS support: consider bundling native libs in JARs
Posted by Mark Thomas <ma...@apache.org>.
On 19 October 2017 15:11:19 BST, Brian Clozel <bc...@pivotal.io> wrote:
>Hi,
>
>More and more servers are choosing to make available one or more
>solutions
>to use TLS native stacks by shipping them as JARs:
>
>* Netty has quite a few options there
>http://netty.io/wiki/forked-tomcat-native.html
>* Jetty is now shipping a conscrypt support as well
>https://webtide.com/conscrypting-native-ssl-for-jetty/
>
How does shipping a native library in a JAR work? What makes it simpler than building from source?
(Past experience of providing binaries for OS other than Windows is that the number of different builds rapidly multiplies - hence source only at the moment.)
>I know there are other solutions for that, like changing the boot
>classpath
>or installing native libraries directly on the host operating system.
>But
>those solutions aren't always super easy to achieve in cloud
>environments;
>there are also questions on this mailing list around
>tomcat+tcnative+openssl versions compatibility.
>
>Would the Tomcat community consider shipping JARs (with classifier and
>uber
>JARs) containing the required native libraries (libtcnative + openssl +
>apr)?
In principle no reason why not but more detail on the requirements is needed.
>Bonus question: would you consider supporting boringssl or libressl?
Libressl is supported as of 1.2.13 apart from some features where we need functionality not available in libressl (I forget what those features were but I don't think they were significant)
Boringssl should be doable as well but I don't think anyone has tried it yet.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org