You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2018/10/12 19:18:44 UTC
Academic question about Solr's embedded web server
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
All,
I'm curious as to why Solr uses Jetty and not Tomcat.
I'm a PMC member for Tomcat and, though I won't go on a crusade to
replace Jetty with Tomcat, here, I would still like to understand the
motivations behind going with Jetty over Tomcat.
We'd like to make Tomcat such that, if you had the choice to make
again, you might pick Tomcat instead.
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvA85QACgkQHPApP6U8
pFjQnA/7BaGpYWaTqAbNXbIqLrDkylAfse4mNIT/f0d4eSzTfDtxcRm4TadJkMCN
g7OkhGv2QGF9jUlpRVhR77+7m455JaFiPwtFRe32rGlxjvIIPwQK0wUVAossqjzL
ZWvit47xVmU48j5x1E6hnbUBVT4993HVqjlZlvT7ttQgWoFEgO20RxuybACe1q4j
bGywbvqMyT4/z6X81bfevnY683vXoJ3vHRDFTcFHmcBQfqP9/p4DK6uTmDjivZP5
DPsos+m/+QP1xppK5pyyXVWVt6OBE3cqnlSn4zMH9j50mylLF1PSHvriOiBZku87
NZy3hXOnaDBjPQxg818c2p2XORLMmqqxTlazb295FB4fWh/Z98q2BAYpkaj1svTB
BK9lCp1qN9+F9npN394kcLa18e6r7YGJGKssS1xrMUo+w4ssCBomlnMFCv0a9/7+
6+yDtZP7azTIXiK+1TKSXxDMny1LxzyjuH/wIXF+tm+1CY/ecRYhn0yRYk7G/ZOR
0fbxWTXzoKU8a/S0RCv6vDOyCXxF5pWN9fqKfCV13UtyIOaN69uoRgqUJAD03FDm
az+OGeVfZk7wtYFDFW/KiDZ1XwNBXtazlYJm9foFp0oz5AeL4eIxw72VvQyqWpfK
NLoKXrV9x/UOxGEVOqqCyDylXfXPUmbCL4o/xnL++T07PrwbXv8=
=Xy48
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Academic question about Solr's embedded web server
Posted by Shawn Heisey <ap...@elyograg.org>.
On 10/13/2018 11:41 AM, Martin Gainty wrote:
> MG>that would only be true if your support didnt spend all their time
> on self-serving evasive answers
> MG>look at the 100s of JIRA requests that were torpedoed by MarkTrump
Are you TRYING to start a flamewar? I may regret asking this, but I'm
curious whether that was directed at us, or at Tomcat. If there are
Lucene/Solr issues in Jira that you think aren't getting the attention
they need, I'm interested in at least taking a look at them.
>
> MG>TC drags their feet on SSL conformance
> MG>no matter.. TLS v1.3 is the new standard and I guarantee you
> MG>TC will never catch up to that standard
> MG>implement jetty 9.4.12
I have not been closely following discussions about TLS 1.3, and in
particular don't know anything about it in relation to Tomcat, other
than Christopher's comment that the next releases of Tomcat will support
it if the JVM does.
I don't know that TLS 1.3 is super important for Solr. If somebody
places a Solr install in a network location where earlier TLS versions
really aren't good enough, they're asking for problems, no matter what
security measures they've implemented. Solr should have network-level
isolation from any people or systems that cannot be trusted. If
physical access and network access are suitably restricted, there's
little need for additional security measures, including encryption.
This doesn't mean I am opposed to encryption efforts, but I'd like to
find a way to address ease-of-use considerations. TLS with a Java
program is always a bit of a nightmare. I have some ideas about making
it a lot easier, but nowhere near as much free time as I need to explore
them.
It would be fascinating to witness knowledgeable advocates from all of
the projects we could use for network support go head to head to try and
convince us which is better long-term ... as long as the discussion
remains civil. Jetty, Tomcat, and Netty are the likely candidates.
Just for fun, I loaded up the master branch into eclipse, removed Jetty
from the ivy dependencies in Solr, and then rebuilt the eclipse
project. I was quite surprised to see there were only five compile
errors, and they showed up in only two classes, both in the test code.
Whoever wrote the Jetty integration for tests was thinking ahead and
kept that code pretty well isolated. So maybe it won't take all that
much coding to switch the test infrastructure.
I am not technically or philosophically opposed to the idea of using
Tomcat to power a new major version of Solr, if the cost/benefit ratio
is acceptable and provable.
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Academic question about Solr's embedded web server
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Erick,
On 10/14/18 00:03, Erick Erickson wrote:
> Can we stick to the question? Which is "what advantages Tomcat
> might bring to be worth the _very_ significant effort it would take
> to replace Jetty, assuming that it's a choice between the two".
I just read that someone is building an alpha SPNEGO implementation
for Solr. Tomcat ships with one. That might be one item in its favor.
Given that most servlet containers are nominally equivalent, I'm not
sure I could build a compelling case for why you *should* switch to
Tomcat. But given that there is an opportunity to re-evaluate the
container you will use (e.g. if you switch from container-hosted Solr
to Solr-hosted container), then switching becomes easier should you
choose to switch.
The decision to move to Tomcat may be made simply to prefer a product
with a clean AL2 license (Jetty has restrictions; I haven't read them
but I suspect they are not really problematic in any meaningful way).
> For the level of effort required to make the switch to Tomcat, I
> suspect the consensus would be to use neither and switch to Netty
> or similar, which makes the discussion so far pretty much a waste
> of electrons.
So, if you think switching to Netty will be worth it, I'd like to know
*those* reasons. Netty is a great product, but you only benefit if you
abandon the servlet spec and build to their non-standard APIs. That
would basically require a ground-up rebuild of Solr from scratch in
order to gain any perceivable benefit from switching to Netty.
If you guys would like to see some independent performance
evaluations, have a look at this:
https://www.javacodegeeks.com/2016/05/benchmarking-high-concurrency-http
- -servers-jvm.html
Skip to the graphs. They are nearly impossible to read, but show that
all servers benchmarked seem to have roughly the same performance
characteristics. Basically, you can't go wrong. Anything that might
seriously impact your performance will come down to resources
(hardware) or configuration (e.g. if you want fast TLS, don't use the
Java crypto provider).
I know there are some throughput-vs-CPU-usage numbers around for
Tomcat versus httpd and one of two other servers/configurations. I'm
trying to dig those up.
Thanks,
- -chris
> Erick On Sat, Oct 13, 2018 at 6:24 PM Christopher Schultz
> <ch...@christopherschultz.net> wrote:
>>
> Martin,
>
> Wow, wild conjecture with no supporting evidence. Seems like par
> for the course with you. Read on, if you dare.
>
> On 10/13/18 13:41, Martin Gainty wrote:
>>>>
>>>> -------------------------------------------------------------------
- ---
>>
>>
>>>>
- ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
>
>
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvInIUACgkQHPApP6U8
pFh0JQ/+J53IZkbqdKnWLa4ndg8v2AZrhrWha0YHSMlOih9k7ANcw+0G0jtTek6m
l2CTgQOuKGGn3/PVfP6OypEXOhQ2XR/VQBrUCZzKzLIlEcEEiyLV0vb9fm4HnmIR
KwEsgU0pnTIwsE540F8eP/MK5AqQ2ULqMlsX+m19U0/RAx6PnXQGGxAORRIVLFi/
mK7Yu1Ou2pbT/3W1iL63fbbkmDgyeRqo6lthWvwa2Q+zAq/qe/N4drJ1dAYdSg4c
L6xY5JVVYFRlS2+2Cc0TeBNB3yVsRPakz4Q4P+GKyTOS61yuXsPL/DhDpLdHqca9
WkRrG0MBVPM9X7HRd7PrNvzJIyeEJYFLOEt7UFrZ/tmMlmZD5M5w33L4na2ZJN86
MKxcCQz3M/FySfbaKmrRBu8BAIkCmmU7DAMdGUCLJKQ3sdHknc7QisLARrY0Xx17
nDjXcKxu0lrO6ZTKJ1/nHwGZz1tEe+cgJKwcEyTWlFnYm1IS0A9FlMHdePvA54Jh
x1J4cW7qut5ir1WS3Un7K52RndGGXNVTfkyTsJnCx9+65Wuq86M+lgEuyiRCuhVL
wUpCx2l3qNcTCOZjUj+i8cq1aJIEm+Ql7EeS9JEB4Y+2yCOhgPS9lKotTTLmVFbK
2ccG9nyMu8iPN2UPyT+VqN+9HmXZ/FVQj2lTfU02lAlWZODxu88=
=tcIz
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Academic question about Solr's embedded web server
Posted by Erick Erickson <er...@gmail.com>.
Can we stick to the question? Which is "what advantages Tomcat might
bring to be worth the _very_ significant effort it would take to replace Jetty,
assuming that it's a choice between the two".
For the level of effort required to make the switch to Tomcat, I suspect the
consensus would be to use neither and switch to Netty or similar, which
makes the discussion so far pretty much a waste of electrons.
Erick
Erick
On Sat, Oct 13, 2018 at 6:24 PM Christopher Schultz
<ch...@christopherschultz.net> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Martin,
>
> Wow, wild conjecture with no supporting evidence. Seems like par for
> the course with you. Read on, if you dare.
>
> On 10/13/18 13:41, Martin Gainty wrote:
> >
> > ----------------------------------------------------------------------
> - --
> >
> >
> *From:* Christopher Schultz <ch...@christopherschultz.net>
> > *Sent:* Friday, October 12, 2018 4:30 PM *To:*
> > dev@lucene.apache.org *Subject:* Re: Academic question about Solr's
> > embedded web server
> >
> > Shawn,
> >
> > On 10/12/18 15:52, Shawn Heisey wrote:
> >> On 10/12/2018 1:18 PM, Christopher Schultz wrote:
> >>> I'm curious as to why Solr uses Jetty and not Tomcat.
> >
> >> I wasn't part of the project when that decision was made. Jetty
> >> was already included with Solr when I first downloaded it --
> >> Solr version 1.4.0, back in 2009. Jetty wasn't quite as
> >> integrated as it is now, and at that time, Solr was still
> >> shipping as a WAR, suitable for any container.
> >
> >> I'm going to offer my two cents, which I admit up front is only
> >> an educated guess. Others can probably give you concrete
> >> information about discussions that happened nearly a decade ago.
> >
> >> I think the primary reason is that Jetty is more lightweight than
> >> Tomcat.
> >
> > I think this is more of a perception than actual truth. MG>dead
> > wrong
>
> Dead, eh? Tomcat performs on-par with Apache httpd. Where is the
> "heavy" in those numbers?
>
> >> And the Jetty that's included with Solr is considerably stripped
> >> down compared to a standard binary distribution, so its
> >> footprint is VERY small. Since Solr 4.0 when Solr's UI
> >> completely changed to Javascript, even JSP support is missing.
> >
> > When you consider "footprint", do you mean on-disk or in-memory? I
> > ask because Tomcat can be used in an embedded mode where you
> > basically only enable things that you want. For example, if you
> > don't want JSP, you simply don't bring-up the JSP engine on
> > startup. If Solr doesn't use Websocket, then you don't have to
> > enable that, either. Tearing-out the various JAR files might be
> > more tricky and it may be simpler to leave the unused classes
> > sitting around, unused. MG>inefficient
>
> In what was, specifically? Memory usage? CPU usage? Under what load?
> Which connector? Which JVM? I have ready answers for all of these
> questions. Are you just taking pot-shots or do you have real-world
> evidence. Because I do, and many others do as well.
>
> > It might be safe to drop a few of the stock JAR files to save some
> > disk space, but it would be best if you/we didn't have to crack
> > anything open and remove anything from existing artifacts. If there
> > is a use-case for further-decomposing Tomcat into more JAR files so
> > that more of them can be removed for e.g. Solr (and anyone else
> > using Tomcat-embedded), then that is worth considering.
> >
> >>> We'd like to make Tomcat such that, if you had the choice to
> >>> make again, you might pick Tomcat instead.
> >
> >> If you can make a very compelling argument about benefits we
> >> would see from moving to Tomcat, and can help us modify things
> >> like our testing infrastructure and scripting to make it all
> >> work, I see no reason we wouldn't give it serious consideration.
> >> It would be VERY important for the test infrastructure to use the
> >> same container as the binary distribution. That's probably the
> >> biggest source of inertia that keeps us where we are.
> >
> >> Take a look at SOLR-6733 (and its child SOLR-6734) in Jira for
> >> some ideas I've been tossing around for embedding the container
> >> directly into Solr itself, so that Solr is a standalone
> >> application. I never have the time I need to work on it, and it's
> >> a really major task. I know from work I've done using Spring
> >> Boot that Tomcat can also be embedded.
> >
> > MG>as you are using the embedded model from Spring
>
> Uh, what? Spring boot is still wet behind the ears, son. Tomcat has
> has an embedded mode since long ago. ~10 lines of code gets you going.
>
> > I didn't realize that Solr wasn't using Jetty embedded. I just
> > assumed that it was. If you are thinking about turning Jetty
> > inside-out so that you launch Solr and Solr launches Jetty, then
> > actually now might be a good time to re-consider using Tomcat
> > versus Jetty. MG>that would only be true if your support didnt
> > spend all their time on self-serving evasive answers MG>look at the
> > 100s of JIRA requests that were torpedoed by MarkTrump
>
> Hmm. I'm not sure why I even bothered responding. Maybe it's because I
> have technical arguments about things instead of resorting to ad
> homonym attacks.
>
> > Self-hosting a Tomcat instance (i.e. embedded usage) is fairly
> > straightforward: create an instance of the Tomcat class and add
> > connectors (port binding), web applications, etc. Presumably,
> > you'd add one or maybe two connectors (HTTP and/or HTTPS) and then
> > a single web application: Solr.
> >
> > MG>TC drags their feet on SSL conformance MG>no matter.. TLS v1.3
> > is the new standard and I guarantee you MG>TC will never catch up
> > to that standard MG>implement jetty 9.4.12
> > https://github.com/eclipse/jetty.project/issues/2711
> > <https://github.com/eclipse/jetty.project/issues/2711>
>
> Uhh... all supported releases of Tomcat will have TLSv1.3 support in
> the next point release: 9.0.13, 8.5.35, and 7.0.92. Both in
> NIO+OpenSSL mode (native = much faster) or NIO/BIO + JSSE mode (for
> JVMs which support it). Works with CLIENT-AUTH as well. Tomcat is on
> track to beat the mainstream web browsers to market with TLSv1.3 support
> .
>
> Feel free to tilt at whatever windmills you wish, Martin. The rest of
> us will calmly continue and ignore the noise with which you fill the air
> .
>
> I appreciate the opinions of the rest of the Solr community on this issu
> e.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvCmsgACgkQHPApP6U8
> pFingg//SmbdPqHgvfxE+y7y0uKdkou60zIZE9juwfTJIjm76wOd7GQE+eSPhCO9
> 6WCO9HSyeK1I20NhwH9qP9L8ZQl5fDqeBAQt2IpUsszB0NsmMEXW3ylapqt1Ob2i
> 39LdLRJRgNoiPUsZU75BHBcNA2XBpeRa/BryEWmIzkAkIfRuypqyCQb/B7ThbNFO
> Gaqi3jBbi/ESAy0V1h8ekczMMl+KzswGgezgnE1pNEDZDB5CuIZDnhAE9OImLzmY
> zcSGfthaYMwgZNA7ZsBEL34UZ4ZTxPNl9TZ1ibH36AoHtxTmxtrSV/1fU9vFeWlD
> Y8DBjQj682xv2vQdxEJP8bmkAGzE+NvlA6p1fIfxSM4cw00zwQO6JOReVSzEwHUv
> BuJhlhvHge9uQqTGU6qAZtTqIAmqYqgyOucHBnOY0HgFK8zpz4PacQkFcl4nKZdu
> pxOP0E47SkMKqvVStmJzVHwMZY+UIG8aSqdoveLk0dRxILXoyijsXjbTnViGjP+O
> NhNzohKNpE1NHGamZFBzszal6nVK8pOlMpPm4veJzEVvP5OsEAv1DOxYsJN9XouE
> mOrq9d4krjqvoWzhTdohU72bZpU1ldWvypCf6cxzkUSC0wLBsx9rGj6QgzMG/BOv
> k3C3OAGGXv1gaU51LMnBfQcGwX1ZeiQSB5YZxRDaKHDoQeT3S80=
> =7tP/
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Academic question about Solr's embedded web server
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Martin,
Wow, wild conjecture with no supporting evidence. Seems like par for
the course with you. Read on, if you dare.
On 10/13/18 13:41, Martin Gainty wrote:
>
> ----------------------------------------------------------------------
- --
>
>
*From:* Christopher Schultz <ch...@christopherschultz.net>
> *Sent:* Friday, October 12, 2018 4:30 PM *To:*
> dev@lucene.apache.org *Subject:* Re: Academic question about Solr's
> embedded web server
>
> Shawn,
>
> On 10/12/18 15:52, Shawn Heisey wrote:
>> On 10/12/2018 1:18 PM, Christopher Schultz wrote:
>>> I'm curious as to why Solr uses Jetty and not Tomcat.
>
>> I wasn't part of the project when that decision was made. Jetty
>> was already included with Solr when I first downloaded it --
>> Solr version 1.4.0, back in 2009. Jetty wasn't quite as
>> integrated as it is now, and at that time, Solr was still
>> shipping as a WAR, suitable for any container.
>
>> I'm going to offer my two cents, which I admit up front is only
>> an educated guess. Others can probably give you concrete
>> information about discussions that happened nearly a decade ago.
>
>> I think the primary reason is that Jetty is more lightweight than
>> Tomcat.
>
> I think this is more of a perception than actual truth. MG>dead
> wrong
Dead, eh? Tomcat performs on-par with Apache httpd. Where is the
"heavy" in those numbers?
>> And the Jetty that's included with Solr is considerably stripped
>> down compared to a standard binary distribution, so its
>> footprint is VERY small. Since Solr 4.0 when Solr's UI
>> completely changed to Javascript, even JSP support is missing.
>
> When you consider "footprint", do you mean on-disk or in-memory? I
> ask because Tomcat can be used in an embedded mode where you
> basically only enable things that you want. For example, if you
> don't want JSP, you simply don't bring-up the JSP engine on
> startup. If Solr doesn't use Websocket, then you don't have to
> enable that, either. Tearing-out the various JAR files might be
> more tricky and it may be simpler to leave the unused classes
> sitting around, unused. MG>inefficient
In what was, specifically? Memory usage? CPU usage? Under what load?
Which connector? Which JVM? I have ready answers for all of these
questions. Are you just taking pot-shots or do you have real-world
evidence. Because I do, and many others do as well.
> It might be safe to drop a few of the stock JAR files to save some
> disk space, but it would be best if you/we didn't have to crack
> anything open and remove anything from existing artifacts. If there
> is a use-case for further-decomposing Tomcat into more JAR files so
> that more of them can be removed for e.g. Solr (and anyone else
> using Tomcat-embedded), then that is worth considering.
>
>>> We'd like to make Tomcat such that, if you had the choice to
>>> make again, you might pick Tomcat instead.
>
>> If you can make a very compelling argument about benefits we
>> would see from moving to Tomcat, and can help us modify things
>> like our testing infrastructure and scripting to make it all
>> work, I see no reason we wouldn't give it serious consideration.
>> It would be VERY important for the test infrastructure to use the
>> same container as the binary distribution. That's probably the
>> biggest source of inertia that keeps us where we are.
>
>> Take a look at SOLR-6733 (and its child SOLR-6734) in Jira for
>> some ideas I've been tossing around for embedding the container
>> directly into Solr itself, so that Solr is a standalone
>> application. I never have the time I need to work on it, and it's
>> a really major task. I know from work I've done using Spring
>> Boot that Tomcat can also be embedded.
>
> MG>as you are using the embedded model from Spring
Uh, what? Spring boot is still wet behind the ears, son. Tomcat has
has an embedded mode since long ago. ~10 lines of code gets you going.
> I didn't realize that Solr wasn't using Jetty embedded. I just
> assumed that it was. If you are thinking about turning Jetty
> inside-out so that you launch Solr and Solr launches Jetty, then
> actually now might be a good time to re-consider using Tomcat
> versus Jetty. MG>that would only be true if your support didnt
> spend all their time on self-serving evasive answers MG>look at the
> 100s of JIRA requests that were torpedoed by MarkTrump
Hmm. I'm not sure why I even bothered responding. Maybe it's because I
have technical arguments about things instead of resorting to ad
homonym attacks.
> Self-hosting a Tomcat instance (i.e. embedded usage) is fairly
> straightforward: create an instance of the Tomcat class and add
> connectors (port binding), web applications, etc. Presumably,
> you'd add one or maybe two connectors (HTTP and/or HTTPS) and then
> a single web application: Solr.
>
> MG>TC drags their feet on SSL conformance MG>no matter.. TLS v1.3
> is the new standard and I guarantee you MG>TC will never catch up
> to that standard MG>implement jetty 9.4.12
> https://github.com/eclipse/jetty.project/issues/2711
> <https://github.com/eclipse/jetty.project/issues/2711>
Uhh... all supported releases of Tomcat will have TLSv1.3 support in
the next point release: 9.0.13, 8.5.35, and 7.0.92. Both in
NIO+OpenSSL mode (native = much faster) or NIO/BIO + JSSE mode (for
JVMs which support it). Works with CLIENT-AUTH as well. Tomcat is on
track to beat the mainstream web browsers to market with TLSv1.3 support
.
Feel free to tilt at whatever windmills you wish, Martin. The rest of
us will calmly continue and ignore the noise with which you fill the air
.
I appreciate the opinions of the rest of the Solr community on this issu
e.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvCmsgACgkQHPApP6U8
pFingg//SmbdPqHgvfxE+y7y0uKdkou60zIZE9juwfTJIjm76wOd7GQE+eSPhCO9
6WCO9HSyeK1I20NhwH9qP9L8ZQl5fDqeBAQt2IpUsszB0NsmMEXW3ylapqt1Ob2i
39LdLRJRgNoiPUsZU75BHBcNA2XBpeRa/BryEWmIzkAkIfRuypqyCQb/B7ThbNFO
Gaqi3jBbi/ESAy0V1h8ekczMMl+KzswGgezgnE1pNEDZDB5CuIZDnhAE9OImLzmY
zcSGfthaYMwgZNA7ZsBEL34UZ4ZTxPNl9TZ1ibH36AoHtxTmxtrSV/1fU9vFeWlD
Y8DBjQj682xv2vQdxEJP8bmkAGzE+NvlA6p1fIfxSM4cw00zwQO6JOReVSzEwHUv
BuJhlhvHge9uQqTGU6qAZtTqIAmqYqgyOucHBnOY0HgFK8zpz4PacQkFcl4nKZdu
pxOP0E47SkMKqvVStmJzVHwMZY+UIG8aSqdoveLk0dRxILXoyijsXjbTnViGjP+O
NhNzohKNpE1NHGamZFBzszal6nVK8pOlMpPm4veJzEVvP5OsEAv1DOxYsJN9XouE
mOrq9d4krjqvoWzhTdohU72bZpU1ldWvypCf6cxzkUSC0wLBsx9rGj6QgzMG/BOv
k3C3OAGGXv1gaU51LMnBfQcGwX1ZeiQSB5YZxRDaKHDoQeT3S80=
=7tP/
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Academic question about Solr's embedded web server
Posted by Martin Gainty <mg...@hotmail.com>.
________________________________
From: Christopher Schultz <ch...@christopherschultz.net>
Sent: Friday, October 12, 2018 4:30 PM
To: dev@lucene.apache.org
Subject: Re: Academic question about Solr's embedded web server
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Shawn,
On 10/12/18 15:52, Shawn Heisey wrote:
> On 10/12/2018 1:18 PM, Christopher Schultz wrote:
>> I'm curious as to why Solr uses Jetty and not Tomcat.
>
> I wasn't part of the project when that decision was made. Jetty
> was already included with Solr when I first downloaded it -- Solr
> version 1.4.0, back in 2009. Jetty wasn't quite as integrated as
> it is now, and at that time, Solr was still shipping as a WAR,
> suitable for any container.
>
> I'm going to offer my two cents, which I admit up front is only an
> educated guess. Others can probably give you concrete information
> about discussions that happened nearly a decade ago.
>
> I think the primary reason is that Jetty is more lightweight than
> Tomcat.
I think this is more of a perception than actual truth.
MG>dead wrong
> And the Jetty that's included with Solr is considerably stripped
> down compared to a standard binary distribution, so its footprint
> is VERY small. Since Solr 4.0 when Solr's UI completely changed to
> Javascript, even JSP support is missing.
When you consider "footprint", do you mean on-disk or in-memory? I ask
because Tomcat can be used in an embedded mode where you basically
only enable things that you want. For example, if you don't want JSP,
you simply don't bring-up the JSP engine on startup. If Solr doesn't
use Websocket, then you don't have to enable that, either. Tearing-out
the various JAR files might be more tricky and it may be simpler to
leave the unused classes sitting around, unused.
MG>inefficient
It might be safe to drop a few of the stock JAR files to save some
disk space, but it would be best if you/we didn't have to crack
anything open and remove anything from existing artifacts. If there is
a use-case for further-decomposing Tomcat into more JAR files so that
more of them can be removed for e.g. Solr (and anyone else using
Tomcat-embedded), then that is worth considering.
>> We'd like to make Tomcat such that, if you had the choice to
>> make again, you might pick Tomcat instead.
>
> If you can make a very compelling argument about benefits we would
> see from moving to Tomcat, and can help us modify things like our
> testing infrastructure and scripting to make it all work, I see no
> reason we wouldn't give it serious consideration. It would be VERY
> important for the test infrastructure to use the same container as
> the binary distribution. That's probably the biggest source of
> inertia that keeps us where we are.
>
> Take a look at SOLR-6733 (and its child SOLR-6734) in Jira for
> some ideas I've been tossing around for embedding the container
> directly into Solr itself, so that Solr is a standalone
> application. I never have the time I need to work on it, and it's a
> really major task. I know from work I've done using Spring Boot
> that Tomcat can also be embedded.
MG>as you are using the embedded model from Spring
I didn't realize that Solr wasn't using Jetty embedded. I just assumed
that it was. If you are thinking about turning Jetty inside-out so
that you launch Solr and Solr launches Jetty, then actually now might
be a good time to re-consider using Tomcat versus Jetty.
MG>that would only be true if your support didnt spend all their time on self-serving evasive answers
MG>look at the 100s of JIRA requests that were torpedoed by MarkTrump
Self-hosting a Tomcat instance (i.e. embedded usage) is fairly
straightforward: create an instance of the Tomcat class and add
connectors (port binding), web applications, etc. Presumably, you'd
add one or maybe two connectors (HTTP and/or HTTPS) and then a single
web application: Solr.
MG>TC drags their feet on SSL conformance
MG>no matter.. TLS v1.3 is the new standard and I guarantee you
MG>TC will never catch up to that standard
MG>implement jetty 9.4.12
https://github.com/eclipse/jetty.project/issues/2711
[https://avatars3.githubusercontent.com/u/414681?s=400&v=4]<https://github.com/eclipse/jetty.project/issues/2711>
TLS 1.3 compliance · Issue #2711 · eclipse/jetty.project<https://github.com/eclipse/jetty.project/issues/2711>
The TLS 1.3 implementation landed in JDK 11-ea+21. We need to make sure that SslConnection works well with TLS 1.3. In particular: After ClientHello and ServerHello, all other TLS handshake message...
github.com
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvBBEsACgkQHPApP6U8
pFgNxw//Xd4IZCtNo0/VHDbYlaaFTAjyWDLvsjWZuDFBZqJq9zBHbE00zMbqjFA0
eQF9/gB8wqctCHZttj124GfoaPP80p4/8SW/myBu239D+UvdWWtF1c61ItOrdkU8
46DlVROxkjsjxmguBL5R7EwJqlYB5feVTRGU7aKkiYxI1A2Jv3+NrejtqtmIxgrW
M1TPQfpI7dhyuJ0GqWafmr5oW7hfDt/zrx1f96FiYp2gWnjJXsH7UwfHRWXUvOEa
1b4SlkkpFnrtwjOmX5WiJPSghfHmkPSnshwFe1B4E4MEi3XWQnIqWY5f0ZO8i1ZT
/hw1CEBzU/NFnae5ER/uDntzZSMsnVVkmgvQTsyXk41F6VnnpGy1RNc6MOgi4B7X
DA96PlXyAbiCaDGTKz6fbPV+5AaaNgfSoJUBTegX/rMbabvHuSLUqjrapzstas03
TTBZNyjIDxICKDXgbNCmPUaJwuxmgpA55b3UWn97E4VmVYyIvohJe3MpZKnLkaf/
4HRZej0IBBfTywmyIcW0BvC9wWr6PvNN9KpkPrcwUMLN1dHGUV+XZ/ljv4vpoi6m
+0nPFtpGQRshrFh3s3BGQo37uCHB+sXTCqwzggb7RU476BtgYBHGB+koMZ+WvnRd
u3vwgPmNEhUYroqHHEzeMvRO/3NYrJzehtZ/ubi7YJBF1qXSL5k=
=v4NI
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Academic question about Solr's embedded web server
Posted by Erick Erickson <er...@gmail.com>.
I wasn't part of the original decision either. That said, this will be
an uphill battle, for reasons other than "which is better". If we're
going to consider major surgery at that level, we've been talking
about ripping out Jetty completely and going with Netty or similar.
That notion has stalled, largely due to the amount of work it would
entail, Jetty-isms are deeply embedded in, for instance, the test
framework for.
Plus, the way Solr is used would anyone notice? There's not much
exposure as far as _users_ are concerned when it comes to this, it's
considered an implementation detail. So in order to be acted upon, the
advantages would have to be compelling to justify the work. It's a far
larger task than than just substituting one for the other so needs
_really_ compelling reasons to happen.
I don't personally have much skin in this game, mostly making you
aware of a couple of the project-level issues you'll be facing during
the discussion.
Best,
Erick
On Fri, Oct 12, 2018 at 4:23 PM Shawn Heisey <ap...@elyograg.org> wrote:
>
> On 10/12/2018 2:30 PM, Christopher Schultz wrote:
> >> I think the primary reason is that Jetty is more lightweight than
> >> Tomcat.
> > I think this is more of a perception than actual truth.
>
> That seems like a very real possibility.
>
> >> And the Jetty that's included with Solr is considerably stripped
> >> down compared to a standard binary distribution, so its footprint
> >> is VERY small. Since Solr 4.0 when Solr's UI completely changed to
> >> Javascript, even JSP support is missing.
> > When you consider "footprint", do you mean on-disk or in-memory?
>
> Memory, code, download size. When it comes to disk usage, a few extra
> megabytes is peanuts. Most indexes that people build with Solr will be
> larger than the program installation -- disk usage isn't something I
> worry about.
>
> And even the footprint thing could be a matter of perception as well,
> especially if we can embed the container and have a high degree of
> control over exactly what code gets used.
>
> Convince us with concrete evidence that Tomcat will have a positive
> impact on Solr, and that the benefit will be worth the effort involved
> in switching. If you can do that, then point us at resources to ease
> that switch.
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Academic question about Solr's embedded web server
Posted by Shawn Heisey <ap...@elyograg.org>.
On 10/12/2018 2:30 PM, Christopher Schultz wrote:
>> I think the primary reason is that Jetty is more lightweight than
>> Tomcat.
> I think this is more of a perception than actual truth.
That seems like a very real possibility.
>> And the Jetty that's included with Solr is considerably stripped
>> down compared to a standard binary distribution, so its footprint
>> is VERY small. Since Solr 4.0 when Solr's UI completely changed to
>> Javascript, even JSP support is missing.
> When you consider "footprint", do you mean on-disk or in-memory?
Memory, code, download size. When it comes to disk usage, a few extra
megabytes is peanuts. Most indexes that people build with Solr will be
larger than the program installation -- disk usage isn't something I
worry about.
And even the footprint thing could be a matter of perception as well,
especially if we can embed the container and have a high degree of
control over exactly what code gets used.
Convince us with concrete evidence that Tomcat will have a positive
impact on Solr, and that the benefit will be worth the effort involved
in switching. If you can do that, then point us at resources to ease
that switch.
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Academic question about Solr's embedded web server
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Shawn,
On 10/12/18 15:52, Shawn Heisey wrote:
> On 10/12/2018 1:18 PM, Christopher Schultz wrote:
>> I'm curious as to why Solr uses Jetty and not Tomcat.
>
> I wasn't part of the project when that decision was made. Jetty
> was already included with Solr when I first downloaded it -- Solr
> version 1.4.0, back in 2009. Jetty wasn't quite as integrated as
> it is now, and at that time, Solr was still shipping as a WAR,
> suitable for any container.
>
> I'm going to offer my two cents, which I admit up front is only an
> educated guess. Others can probably give you concrete information
> about discussions that happened nearly a decade ago.
>
> I think the primary reason is that Jetty is more lightweight than
> Tomcat.
I think this is more of a perception than actual truth.
> And the Jetty that's included with Solr is considerably stripped
> down compared to a standard binary distribution, so its footprint
> is VERY small. Since Solr 4.0 when Solr's UI completely changed to
> Javascript, even JSP support is missing.
When you consider "footprint", do you mean on-disk or in-memory? I ask
because Tomcat can be used in an embedded mode where you basically
only enable things that you want. For example, if you don't want JSP,
you simply don't bring-up the JSP engine on startup. If Solr doesn't
use Websocket, then you don't have to enable that, either. Tearing-out
the various JAR files might be more tricky and it may be simpler to
leave the unused classes sitting around, unused.
It might be safe to drop a few of the stock JAR files to save some
disk space, but it would be best if you/we didn't have to crack
anything open and remove anything from existing artifacts. If there is
a use-case for further-decomposing Tomcat into more JAR files so that
more of them can be removed for e.g. Solr (and anyone else using
Tomcat-embedded), then that is worth considering.
>> We'd like to make Tomcat such that, if you had the choice to
>> make again, you might pick Tomcat instead.
>
> If you can make a very compelling argument about benefits we would
> see from moving to Tomcat, and can help us modify things like our
> testing infrastructure and scripting to make it all work, I see no
> reason we wouldn't give it serious consideration. It would be VERY
> important for the test infrastructure to use the same container as
> the binary distribution. That's probably the biggest source of
> inertia that keeps us where we are.
>
> Take a look at SOLR-6733 (and its child SOLR-6734) in Jira for
> some ideas I've been tossing around for embedding the container
> directly into Solr itself, so that Solr is a standalone
> application. I never have the time I need to work on it, and it's a
> really major task. I know from work I've done using Spring Boot
> that Tomcat can also be embedded.
I didn't realize that Solr wasn't using Jetty embedded. I just assumed
that it was. If you are thinking about turning Jetty inside-out so
that you launch Solr and Solr launches Jetty, then actually now might
be a good time to re-consider using Tomcat versus Jetty.
Self-hosting a Tomcat instance (i.e. embedded usage) is fairly
straightforward: create an instance of the Tomcat class and add
connectors (port binding), web applications, etc. Presumably, you'd
add one or maybe two connectors (HTTP and/or HTTPS) and then a single
web application: Solr.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvBBEsACgkQHPApP6U8
pFgNxw//Xd4IZCtNo0/VHDbYlaaFTAjyWDLvsjWZuDFBZqJq9zBHbE00zMbqjFA0
eQF9/gB8wqctCHZttj124GfoaPP80p4/8SW/myBu239D+UvdWWtF1c61ItOrdkU8
46DlVROxkjsjxmguBL5R7EwJqlYB5feVTRGU7aKkiYxI1A2Jv3+NrejtqtmIxgrW
M1TPQfpI7dhyuJ0GqWafmr5oW7hfDt/zrx1f96FiYp2gWnjJXsH7UwfHRWXUvOEa
1b4SlkkpFnrtwjOmX5WiJPSghfHmkPSnshwFe1B4E4MEi3XWQnIqWY5f0ZO8i1ZT
/hw1CEBzU/NFnae5ER/uDntzZSMsnVVkmgvQTsyXk41F6VnnpGy1RNc6MOgi4B7X
DA96PlXyAbiCaDGTKz6fbPV+5AaaNgfSoJUBTegX/rMbabvHuSLUqjrapzstas03
TTBZNyjIDxICKDXgbNCmPUaJwuxmgpA55b3UWn97E4VmVYyIvohJe3MpZKnLkaf/
4HRZej0IBBfTywmyIcW0BvC9wWr6PvNN9KpkPrcwUMLN1dHGUV+XZ/ljv4vpoi6m
+0nPFtpGQRshrFh3s3BGQo37uCHB+sXTCqwzggb7RU476BtgYBHGB+koMZ+WvnRd
u3vwgPmNEhUYroqHHEzeMvRO/3NYrJzehtZ/ubi7YJBF1qXSL5k=
=v4NI
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Academic question about Solr's embedded web server
Posted by Shawn Heisey <ap...@elyograg.org>.
On 10/12/2018 1:18 PM, Christopher Schultz wrote:
> I'm curious as to why Solr uses Jetty and not Tomcat.
I wasn't part of the project when that decision was made. Jetty was
already included with Solr when I first downloaded it -- Solr version
1.4.0, back in 2009. Jetty wasn't quite as integrated as it is now, and
at that time, Solr was still shipping as a WAR, suitable for any container.
I'm going to offer my two cents, which I admit up front is only an
educated guess. Others can probably give you concrete information about
discussions that happened nearly a decade ago.
I think the primary reason is that Jetty is more lightweight than
Tomcat. And the Jetty that's included with Solr is considerably
stripped down compared to a standard binary distribution, so its
footprint is VERY small. Since Solr 4.0 when Solr's UI completely
changed to Javascript, even JSP support is missing.
> We'd like to make Tomcat such that, if you had the choice to make
> again, you might pick Tomcat instead.
If you can make a very compelling argument about benefits we would see
from moving to Tomcat, and can help us modify things like our testing
infrastructure and scripting to make it all work, I see no reason we
wouldn't give it serious consideration. It would be VERY important for
the test infrastructure to use the same container as the binary
distribution. That's probably the biggest source of inertia that keeps
us where we are.
Take a look at SOLR-6733 (and its child SOLR-6734) in Jira for some
ideas I've been tossing around for embedding the container directly into
Solr itself, so that Solr is a standalone application. I never have the
time I need to work on it, and it's a really major task. I know from
work I've done using Spring Boot that Tomcat can also be embedded.
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org