Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Gus Heck <gu...@gmail.com>]

I know it's probably unrealistic because corporate environments into which
solr is deployed likely would have difficulty with it, but there is a JDK
fork that keeps and improves the Security Manager... It's supported by the
descendants of the Apache River project.
https://github.com/pfirmstone/DirtyChai

Probably not useful, but perhaps interesting. Certainly we are not the only
ones irritated by the loss of the security manager.

On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com> wrote:

> > I should clarify something.  My objections mostly relate to the insistent
> > language in the SIP requiring Solr to have a substitute for the JSM.  I'm
> > not quite against doing something but I might vote -0 on something that
> > seems to have a poor security payoff relative to the maintenance burden.
> > The more off-the-shelf, the better IMO.
> > Thank you for investing your time researching some options.
>
> The security landscape has dramatically changed during the years that we
> have enjoyed JSM protection for Solr. It's a crazy world out there and
> every
> single code flaw, existing and future, will be found and exploited or
> published
> by so called security researchers. That's why I believe we need a
> centralized
> solution.
>
> >> Rejected Alternatives
> >>
> >>   - *Staying on Java < 24* — not viable long-term; Solr must support
> >>   current Java LTS releases.
> >>   - *Removing JSM protections without any replacement* — unacceptable
> >>   security regression.
> >>
> > Both Eric Pugh and I have challenged this.
> >>
> >>   - *OS-level hardening only (systemd, seccomp)* — not cross-platform;
> >>   does not cover Windows or macOS.
> >>
> > I challenge this. Why should the Solr project burden itself with building
> > & maintaining security mechanisms already provided by off-the-shelf
> tools?
> > If a user/operator wishes to run on Windows/macOS that may not have this
> > protection mechanism, it is a risk consideration for that user to
> consider,
> > but isn't a deal-breaker.  The JSM wasn't/isn't a front-line defense;
> it's
> > a defense-in-depth strategy.  Put differently, the protections here are
> > "best effort" but not worthy of a CVE if they were to falter.  I want to
> > get Arnout's opinion on this supposition.
>
> Java Security Manager is a beast, but its file system read/write controls
> have
> saved us many a CVE in 9.x when JSM have been on by default.
> The SIP primarily focuses on sandboxing Solr for wrt file and network
> access.
> The security landscape has changed dramatically last few years, and solving
> file- and network restrictions in a central place through interception
> rathen than
> at each of the 100+ call sites is a more sustainable way.
>
> Users are free to add layers outside the JVM as well, such as systemd,
> container,
> SELinux etc. But be honest - how many small/medium organizations really do
> this?
>
> >>   - *Dynamic ZK-watcher-based network policy* — correct but
> >>   significantly more complex; adds ZK client dependency to the agent
> JAR.
> >>   Superseded by port-based wildcards for intra-cluster traffic.
> >>   - *Building a Java agent from scratch* — higher effort with no
> >>   functional advantage over adapting the Apache 2.0-licensed OpenSearch
> >>   implementation.
> >>
> >> I agree with not burdening this project with building & maintainining
> such
> > a mechanism.
> >
> > I'm not sure, to what degree, we can leverage that existing agent you
> speak
> > of without further burdening us.  It's a burden/reward trade-off.
>
> If the agent becomes a true maintenance burden (i.e. a larger burden than
> handling the CVEs it would prevent, then a valid action is to remove the
> agent again.
> Like any other feature we develop and maintain. Good this is that this is
> pure Java,
> and production-ready stable code.
>
> > In your updated proposal, you point
> > out org.apache.solr.core.CoreContainer#assertPathAllowed  That is called
> by
> > a number of places... although I *think* the original intention was only
> to
> > limit where cores are created?  Can you elaborate on what the role of
> that
> > method *should* be and how the JSM might also or work?
>
> The pathAllowed checks (also the ones pre-dating the centralization in
> CoreContainer,
> were introduced in 8.x, before JSM was introduced or enabled by default.
> Initially
> assertPathAllowed would validate end-user API input to avoid cores being
> created
> or loaded outside blessed folders. It has then been re-used for other code
> locations
> that may do file access based on user-input in API or config. It as also
> extended to
> block UNC paths after many attacks with this as a vector.
>
> I believe perhaps pathAllowed would not have been written had JSM already
> been
> enforced. Although I don't know if you can block UNC with JSM?
> The SIP does not propose to remove pathAllowed now. One benefit of early
> detection
> is that we can give better context-sensitive error- and log messages
> rather than throwing
> an exception.
>
> The agent approach laid out sandboxes four attack vectors
> * File access outside a limited set of folders and full block of Windows
> UNC
> * Network access other than peer solr nodes and zk nodes
> * Disabllow calling System.exit() (mainly useful for 3rd party plugins?)
> * Disallow spawning processes
>
> A nice thing with the agent approach is that it is unintrusive and easy to
> disable,
> so users who want to take care of all this on OS level may disable it, and
> if we or
> users don't find value in it down the road they can disable it or we can
> remove it.
>
>
> Jan
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

-- 
http://www.needhamsoftware.com (work)
https://a.co/d/b2sZLD9 (my fantasy fiction book)

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jan Høydahl <ja...@cominvent.com>]

Thanks for chiming in Jason,

I agree that there are whole industries out there that provide tools. And whole 
OPS teams configuring firewalls all day long who are more skilled than any of us. 
And there is SE-Linux and what have you.

SIP-24 is not stopping folks from doing the gold standard protection, locking
down the cluster in VLAN and with a closed-by-default policy for network etc.
Folks who want to spend that time and energy can run without the agent and
obtain the same.

But SIP-24 is intended as the batteries-included solution that will keep all the
small and medium sized Solr installs running without having to jump every single
release due to N new CVEs patching places in our code that we forgot to plug..

Or are you hinting that we'll define Solr's security model in such a way that
we'll no longer treat arbitrary filesystem read/write or arbitrary network access
as a vulnerability?

Jan

> 5. juni 2026 kl. 20:51 skrev Jason Gerlowski <ge...@gmail.com>:
> 
> To be clear - I'm not trying to -1 the SIP-24 work.  It's a (large)
> concrete improvement IMO.  Please read my previous email in the vein
> of: "This is great as-is but I'd urge us to reconsider this other
> approach one last time."
> 
> On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <ge...@gmail.com> wrote:
>> 
>> Apologies for chiming in so late on this.
>> 
>> The Java agent route seems like a huge improvement over our existing
>> approach.  I love that it's a global protection - it doesn't rely on
>> devs remembering to call the required "check the path" method in every
>> single place we might read a file in our code.
>> 
>> But ultimately I still think systemd/seccomp is a much more
>> sustainable route for us than the Java-agent approach
>> 
>> We're a Search community that's here to solve people's Search
>> problems.  I won't minimize for a second all the effort we've put into
>> security over the years. But there are entire projects that exist to
>> solve these sorts of problems.  IMO steering users towards something
>> like systemd that has a whole community of security folks behind is
>> going to leave them more secure, and be more sustainable for us.
>> 
>> (To be clear I'm not advocating for systemd specifically, but for the
>> general idea of relying on well-known, OS-level protections that exist
>> outside of the JVM and Solr.)
>> 
>> I've heard two objections to the "systemd" approach so far: that
>> systemd isn't cross-platform, and that some folks won't bother to
>> enable it.
>> 
>> To the first objection, I'd say that our responsibility has never been
>> to find one security solution that works for everyone.  We never
>> mandated that folks use a particular firewall: we let users choose how
>> they wanted to provide that network isolation.  IMO this could be
>> handled the same way: point folks to a few different tools, optionally
>> provide a few example configs, and let them pick based on their
>> OS/platform.
>> 
>> To the second objection: this is the whole point of having a security
>> model.  It's expected and normal for projects to describe how to make
>> a deployment of their software "secure".  It's neither fair nor
>> possible to secure folks who won't read our "Going to Production" docs
>> before deploying.
>> 
>> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com> wrote:
>>> 
>>> So the implementation of SIP-24 is now ready for review at
>>> 
>>> https://github.com/apache/solr/pull/4471
>>> 
>>> The PR description gives through description of the change and what to look for.
>>> I have run several rounds of Copilot reviews and Claude reviews.
>>> We have unit tests and BATS tests, so coverage should be fairly good.
>>> 
>>> The PR is fairly large +5454 lines across 62 files and 54 commits, but mostly
>>> contained in the new gradle module, plus one hook in CoreContainer.
>>> 
>>> Take it for a spin and let me know how it works.
>>> 
>>> Jan
>>> 
>>>> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
>>>> 
>>>> Hi,
>>>> 
>>>> Welcoming more feedback on this approach.
>>>> 
>>>> I'm planning to move to implementation phase on Wednesday, to get a first
>>>> version of the agent ready for testing. But it would be helpful with as much
>>>> feedback on the SIP high level design before I start implementing. Thanks David
>>>> for your intial feedback.
>>>> 
>>>> Jan
>>>> 
>>>>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
>>>>> 
>>>>> I know it's probably unrealistic because corporate environments into which
>>>>> solr is deployed likely would have difficulty with it, but there is a JDK
>>>>> fork that keeps and improves the Security Manager... It's supported by the
>>>>> descendants of the Apache River project.
>>>>> https://github.com/pfirmstone/DirtyChai
>>>>> 
>>>>> Probably not useful, but perhaps interesting. Certainly we are not the only
>>>>> ones irritated by the loss of the security manager.
>>>>> 
>>>>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com> wrote:
>>>>> 
>>>>>>> I should clarify something.  My objections mostly relate to the insistent
>>>>>>> language in the SIP requiring Solr to have a substitute for the JSM.  I'm
>>>>>>> not quite against doing something but I might vote -0 on something that
>>>>>>> seems to have a poor security payoff relative to the maintenance burden.
>>>>>>> The more off-the-shelf, the better IMO.
>>>>>>> Thank you for investing your time researching some options.
>>>>>> 
>>>>>> The security landscape has dramatically changed during the years that we
>>>>>> have enjoyed JSM protection for Solr. It's a crazy world out there and
>>>>>> every
>>>>>> single code flaw, existing and future, will be found and exploited or
>>>>>> published
>>>>>> by so called security researchers. That's why I believe we need a
>>>>>> centralized
>>>>>> solution.
>>>>>> 
>>>>>>>> Rejected Alternatives
>>>>>>>> 
>>>>>>>> - *Staying on Java < 24* — not viable long-term; Solr must support
>>>>>>>> current Java LTS releases.
>>>>>>>> - *Removing JSM protections without any replacement* — unacceptable
>>>>>>>> security regression.
>>>>>>>> 
>>>>>>> Both Eric Pugh and I have challenged this.
>>>>>>>> 
>>>>>>>> - *OS-level hardening only (systemd, seccomp)* — not cross-platform;
>>>>>>>> does not cover Windows or macOS.
>>>>>>>> 
>>>>>>> I challenge this. Why should the Solr project burden itself with building
>>>>>>> & maintaining security mechanisms already provided by off-the-shelf
>>>>>> tools?
>>>>>>> If a user/operator wishes to run on Windows/macOS that may not have this
>>>>>>> protection mechanism, it is a risk consideration for that user to
>>>>>> consider,
>>>>>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line defense;
>>>>>> it's
>>>>>>> a defense-in-depth strategy.  Put differently, the protections here are
>>>>>>> "best effort" but not worthy of a CVE if they were to falter.  I want to
>>>>>>> get Arnout's opinion on this supposition.
>>>>>> 
>>>>>> Java Security Manager is a beast, but its file system read/write controls
>>>>>> have
>>>>>> saved us many a CVE in 9.x when JSM have been on by default.
>>>>>> The SIP primarily focuses on sandboxing Solr for wrt file and network
>>>>>> access.
>>>>>> The security landscape has changed dramatically last few years, and solving
>>>>>> file- and network restrictions in a central place through interception
>>>>>> rathen than
>>>>>> at each of the 100+ call sites is a more sustainable way.
>>>>>> 
>>>>>> Users are free to add layers outside the JVM as well, such as systemd,
>>>>>> container,
>>>>>> SELinux etc. But be honest - how many small/medium organizations really do
>>>>>> this?
>>>>>> 
>>>>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
>>>>>>>> significantly more complex; adds ZK client dependency to the agent
>>>>>> JAR.
>>>>>>>> Superseded by port-based wildcards for intra-cluster traffic.
>>>>>>>> - *Building a Java agent from scratch* — higher effort with no
>>>>>>>> functional advantage over adapting the Apache 2.0-licensed OpenSearch
>>>>>>>> implementation.
>>>>>>>> 
>>>>>>>> I agree with not burdening this project with building & maintainining
>>>>>> such
>>>>>>> a mechanism.
>>>>>>> 
>>>>>>> I'm not sure, to what degree, we can leverage that existing agent you
>>>>>> speak
>>>>>>> of without further burdening us.  It's a burden/reward trade-off.
>>>>>> 
>>>>>> If the agent becomes a true maintenance burden (i.e. a larger burden than
>>>>>> handling the CVEs it would prevent, then a valid action is to remove the
>>>>>> agent again.
>>>>>> Like any other feature we develop and maintain. Good this is that this is
>>>>>> pure Java,
>>>>>> and production-ready stable code.
>>>>>> 
>>>>>>> In your updated proposal, you point
>>>>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is called
>>>>>> by
>>>>>>> a number of places... although I *think* the original intention was only
>>>>>> to
>>>>>>> limit where cores are created?  Can you elaborate on what the role of
>>>>>> that
>>>>>>> method *should* be and how the JSM might also or work?
>>>>>> 
>>>>>> The pathAllowed checks (also the ones pre-dating the centralization in
>>>>>> CoreContainer,
>>>>>> were introduced in 8.x, before JSM was introduced or enabled by default.
>>>>>> Initially
>>>>>> assertPathAllowed would validate end-user API input to avoid cores being
>>>>>> created
>>>>>> or loaded outside blessed folders. It has then been re-used for other code
>>>>>> locations
>>>>>> that may do file access based on user-input in API or config. It as also
>>>>>> extended to
>>>>>> block UNC paths after many attacks with this as a vector.
>>>>>> 
>>>>>> I believe perhaps pathAllowed would not have been written had JSM already
>>>>>> been
>>>>>> enforced. Although I don't know if you can block UNC with JSM?
>>>>>> The SIP does not propose to remove pathAllowed now. One benefit of early
>>>>>> detection
>>>>>> is that we can give better context-sensitive error- and log messages
>>>>>> rather than throwing
>>>>>> an exception.
>>>>>> 
>>>>>> The agent approach laid out sandboxes four attack vectors
>>>>>> * File access outside a limited set of folders and full block of Windows
>>>>>> UNC
>>>>>> * Network access other than peer solr nodes and zk nodes
>>>>>> * Disabllow calling System.exit() (mainly useful for 3rd party plugins?)
>>>>>> * Disallow spawning processes
>>>>>> 
>>>>>> A nice thing with the agent approach is that it is unintrusive and easy to
>>>>>> disable,
>>>>>> so users who want to take care of all this on OS level may disable it, and
>>>>>> if we or
>>>>>> users don't find value in it down the road they can disable it or we can
>>>>>> remove it.
>>>>>> 
>>>>>> 
>>>>>> Jan
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>>>> 
>>>>>> 
>>>>> 
>>>>> --
>>>>> http://www.needhamsoftware.com (work)
>>>>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
>>>> 
>>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jason Gerlowski <ge...@gmail.com>]

To be clear - I'm not trying to -1 the SIP-24 work.  It's a (large)
concrete improvement IMO.  Please read my previous email in the vein
of: "This is great as-is but I'd urge us to reconsider this other
approach one last time."

On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <ge...@gmail.com> wrote:
>
> Apologies for chiming in so late on this.
>
> The Java agent route seems like a huge improvement over our existing
> approach.  I love that it's a global protection - it doesn't rely on
> devs remembering to call the required "check the path" method in every
> single place we might read a file in our code.
>
> But ultimately I still think systemd/seccomp is a much more
> sustainable route for us than the Java-agent approach
>
> We're a Search community that's here to solve people's Search
> problems.  I won't minimize for a second all the effort we've put into
> security over the years. But there are entire projects that exist to
> solve these sorts of problems.  IMO steering users towards something
> like systemd that has a whole community of security folks behind is
> going to leave them more secure, and be more sustainable for us.
>
> (To be clear I'm not advocating for systemd specifically, but for the
> general idea of relying on well-known, OS-level protections that exist
> outside of the JVM and Solr.)
>
> I've heard two objections to the "systemd" approach so far: that
> systemd isn't cross-platform, and that some folks won't bother to
> enable it.
>
> To the first objection, I'd say that our responsibility has never been
> to find one security solution that works for everyone.  We never
> mandated that folks use a particular firewall: we let users choose how
> they wanted to provide that network isolation.  IMO this could be
> handled the same way: point folks to a few different tools, optionally
> provide a few example configs, and let them pick based on their
> OS/platform.
>
> To the second objection: this is the whole point of having a security
> model.  It's expected and normal for projects to describe how to make
> a deployment of their software "secure".  It's neither fair nor
> possible to secure folks who won't read our "Going to Production" docs
> before deploying.
>
> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com> wrote:
> >
> > So the implementation of SIP-24 is now ready for review at
> >
> > https://github.com/apache/solr/pull/4471
> >
> > The PR description gives through description of the change and what to look for.
> > I have run several rounds of Copilot reviews and Claude reviews.
> > We have unit tests and BATS tests, so coverage should be fairly good.
> >
> > The PR is fairly large +5454 lines across 62 files and 54 commits, but mostly
> > contained in the new gradle module, plus one hook in CoreContainer.
> >
> > Take it for a spin and let me know how it works.
> >
> > Jan
> >
> > > 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> > >
> > > Hi,
> > >
> > > Welcoming more feedback on this approach.
> > >
> > > I'm planning to move to implementation phase on Wednesday, to get a first
> > > version of the agent ready for testing. But it would be helpful with as much
> > > feedback on the SIP high level design before I start implementing. Thanks David
> > > for your intial feedback.
> > >
> > > Jan
> > >
> > >> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> > >>
> > >> I know it's probably unrealistic because corporate environments into which
> > >> solr is deployed likely would have difficulty with it, but there is a JDK
> > >> fork that keeps and improves the Security Manager... It's supported by the
> > >> descendants of the Apache River project.
> > >> https://github.com/pfirmstone/DirtyChai
> > >>
> > >> Probably not useful, but perhaps interesting. Certainly we are not the only
> > >> ones irritated by the loss of the security manager.
> > >>
> > >> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com> wrote:
> > >>
> > >>>> I should clarify something.  My objections mostly relate to the insistent
> > >>>> language in the SIP requiring Solr to have a substitute for the JSM.  I'm
> > >>>> not quite against doing something but I might vote -0 on something that
> > >>>> seems to have a poor security payoff relative to the maintenance burden.
> > >>>> The more off-the-shelf, the better IMO.
> > >>>> Thank you for investing your time researching some options.
> > >>>
> > >>> The security landscape has dramatically changed during the years that we
> > >>> have enjoyed JSM protection for Solr. It's a crazy world out there and
> > >>> every
> > >>> single code flaw, existing and future, will be found and exploited or
> > >>> published
> > >>> by so called security researchers. That's why I believe we need a
> > >>> centralized
> > >>> solution.
> > >>>
> > >>>>> Rejected Alternatives
> > >>>>>
> > >>>>> - *Staying on Java < 24* — not viable long-term; Solr must support
> > >>>>> current Java LTS releases.
> > >>>>> - *Removing JSM protections without any replacement* — unacceptable
> > >>>>> security regression.
> > >>>>>
> > >>>> Both Eric Pugh and I have challenged this.
> > >>>>>
> > >>>>> - *OS-level hardening only (systemd, seccomp)* — not cross-platform;
> > >>>>> does not cover Windows or macOS.
> > >>>>>
> > >>>> I challenge this. Why should the Solr project burden itself with building
> > >>>> & maintaining security mechanisms already provided by off-the-shelf
> > >>> tools?
> > >>>> If a user/operator wishes to run on Windows/macOS that may not have this
> > >>>> protection mechanism, it is a risk consideration for that user to
> > >>> consider,
> > >>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line defense;
> > >>> it's
> > >>>> a defense-in-depth strategy.  Put differently, the protections here are
> > >>>> "best effort" but not worthy of a CVE if they were to falter.  I want to
> > >>>> get Arnout's opinion on this supposition.
> > >>>
> > >>> Java Security Manager is a beast, but its file system read/write controls
> > >>> have
> > >>> saved us many a CVE in 9.x when JSM have been on by default.
> > >>> The SIP primarily focuses on sandboxing Solr for wrt file and network
> > >>> access.
> > >>> The security landscape has changed dramatically last few years, and solving
> > >>> file- and network restrictions in a central place through interception
> > >>> rathen than
> > >>> at each of the 100+ call sites is a more sustainable way.
> > >>>
> > >>> Users are free to add layers outside the JVM as well, such as systemd,
> > >>> container,
> > >>> SELinux etc. But be honest - how many small/medium organizations really do
> > >>> this?
> > >>>
> > >>>>> - *Dynamic ZK-watcher-based network policy* — correct but
> > >>>>> significantly more complex; adds ZK client dependency to the agent
> > >>> JAR.
> > >>>>> Superseded by port-based wildcards for intra-cluster traffic.
> > >>>>> - *Building a Java agent from scratch* — higher effort with no
> > >>>>> functional advantage over adapting the Apache 2.0-licensed OpenSearch
> > >>>>> implementation.
> > >>>>>
> > >>>>> I agree with not burdening this project with building & maintainining
> > >>> such
> > >>>> a mechanism.
> > >>>>
> > >>>> I'm not sure, to what degree, we can leverage that existing agent you
> > >>> speak
> > >>>> of without further burdening us.  It's a burden/reward trade-off.
> > >>>
> > >>> If the agent becomes a true maintenance burden (i.e. a larger burden than
> > >>> handling the CVEs it would prevent, then a valid action is to remove the
> > >>> agent again.
> > >>> Like any other feature we develop and maintain. Good this is that this is
> > >>> pure Java,
> > >>> and production-ready stable code.
> > >>>
> > >>>> In your updated proposal, you point
> > >>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is called
> > >>> by
> > >>>> a number of places... although I *think* the original intention was only
> > >>> to
> > >>>> limit where cores are created?  Can you elaborate on what the role of
> > >>> that
> > >>>> method *should* be and how the JSM might also or work?
> > >>>
> > >>> The pathAllowed checks (also the ones pre-dating the centralization in
> > >>> CoreContainer,
> > >>> were introduced in 8.x, before JSM was introduced or enabled by default.
> > >>> Initially
> > >>> assertPathAllowed would validate end-user API input to avoid cores being
> > >>> created
> > >>> or loaded outside blessed folders. It has then been re-used for other code
> > >>> locations
> > >>> that may do file access based on user-input in API or config. It as also
> > >>> extended to
> > >>> block UNC paths after many attacks with this as a vector.
> > >>>
> > >>> I believe perhaps pathAllowed would not have been written had JSM already
> > >>> been
> > >>> enforced. Although I don't know if you can block UNC with JSM?
> > >>> The SIP does not propose to remove pathAllowed now. One benefit of early
> > >>> detection
> > >>> is that we can give better context-sensitive error- and log messages
> > >>> rather than throwing
> > >>> an exception.
> > >>>
> > >>> The agent approach laid out sandboxes four attack vectors
> > >>> * File access outside a limited set of folders and full block of Windows
> > >>> UNC
> > >>> * Network access other than peer solr nodes and zk nodes
> > >>> * Disabllow calling System.exit() (mainly useful for 3rd party plugins?)
> > >>> * Disallow spawning processes
> > >>>
> > >>> A nice thing with the agent approach is that it is unintrusive and easy to
> > >>> disable,
> > >>> so users who want to take care of all this on OS level may disable it, and
> > >>> if we or
> > >>> users don't find value in it down the road they can disable it or we can
> > >>> remove it.
> > >>>
> > >>>
> > >>> Jan
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >>> For additional commands, e-mail: dev-help@solr.apache.org
> > >>>
> > >>>
> > >>
> > >> --
> > >> http://www.needhamsoftware.com (work)
> > >> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> > >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Eric Pugh <ep...@opensourceconnections.com>]

I had to google SGTM.    And yes, SGTM!

> On Jun 29, 2026, at 10:16 AM, Jason Gerlowski <ge...@gmail.com> wrote:
> 
> > What is the way forward? My proposal would be to proceed with SIP-24 as an ADDED layer
> of defense ASAP. If it after a year turns out it is hard to maintain
> or otherwise not worth it,
> just drop it.
> 
> I don't expect we'll have much feedback-wise in a year, given how long
> it takes many folks to upgrade. But otherwise this SGTM.
> 
> On the network side in particular I have qualms about taking ownership
> of this stuff when there's great third-party firewalls out there. But
> lacking broad community agreement on that point I don't want to let
> "perfect be the enemy of good" here. SIP-24 is undoubtedly an
> improvement, so let's move that direction!
> 
> On Fri, Jun 19, 2026 at 5:10 AM Jan Høydahl <ja...@cominvent.com> wrote:
> >
> > Hi,
> >
> > Thanks for the feedbacks and reviews David, Eric, Gus and Jason. Trying to sum up:
> >
> > Core principle is we'd like to push as much of network protection, i.e. who can talk to who,
> > downstream to user's network environment, firewalls etc.
> >
> > Encourage users to utilize OS primitives to sandbox the Solr process, whether that be
> > linux seccomp, CAP_* controls and filesystem isolation or similar on other OS'es.
> >
> > It was discussed whether we can frame our new Threat Model in a way that redefines some
> > attack vectors to be the responsibility of the USER. But we cannot push everything on them.
> >
> >
> > Even if we encourage the above, and it will limit the blast radius of a successfuly exploited
> > code vulnerability in Solr, I still sense an agreement that we do want to avoid app-level
> > code in Solr that allows unintended filesystem path traversal, file read outside known
> > solr areas, file write outside index or backup folders etc. And also that our current allowPaths
> > strategy is spotty and that the agent approach centralises and improves upon this.
> >
> > The utility for intercepting outbound network traffic in the agent is a balancing act between
> > convenience and security, and it overlaps partly with external firewall configs. However, it is
> > not uncommon to deploy Solr on the same network segment as databases and other services.
> > The agent gives us and users a tool to contain a possible SSRF resulting from a Solr bug.
> >
> >
> > So given the above, if we want to give SIP-24 and an agent a try, the concerns flagges are
> > twofold - more code to maintain and more code that could break.
> >
> > Wrt "more code to maintain" an option would be to gather core agent code into a separate
> > project, perhaps shared with Opensearch and others with similar needs.
> >
> > Wrt "more code that could break", yes there could be bugs, but I'd argue that on JDK25 without
> > JSM, the new agent would not increase attack surface, but there could be bugs making it
> > fail to protect certain cases we thought were protected.
> >
> >
> > What is the way forward? My proposal would be to proceed with SIP-24 as an ADDED layer
> > of defense ASAP. If it after a year turns out it is hard to maintain or otherwise not worth it,
> > just drop it. Once released, investigate possibility of lifting the core agent out into a new
> > project (apache-commons?) and refactor if successful in a later release. Then we get the
> > benefit of added protection immediately and get a delayed benefit of lower maintenance.
> >
> > Jan
> >
> >
> >
> > > 10. juni 2026 kl. 17:01 skrev David Smiley <ds...@apache.org>:
> > >
> > > Presumably the OpenSearch project would also appreciate reduced maintenance
> > > if this aspect of their project were extracted to an independent project.
> > > It's worth chatting with them about this, yeah? It's also maybe exciting
> > > for those involved in that mechanism to get more exposure -- more projects
> > > using their cool stuff.
> > >
> > > On Wed, Jun 10, 2026 at 4:01 AM Jason Gerlowski <ge...@gmail.com>
> > > wrote:
> > >
> > >> To clarify - I'm not assuming any particular answer to that question.
> > >> If folks think the module won't be high-maintenance, then +1. I just
> > >> want to make sure the question gets asked is all.
> > >>
> > >> (And if we're not sure about the maintenance implications, IMO it'd be
> > >> a great option to move forward with SIP-24 and aim to re-evaluate in a
> > >> year or two when we do have a better sense. I see SIP-24 as a clear
> > >> improvement, so we shouldn't let "perfect be the enemy of good" here.)
> > >>
> > >> On Wed, Jun 10, 2026 at 9:54 AM Jason Gerlowski <ge...@gmail.com>
> > >> wrote:
> > >>>
> > >>>> Or are you hinting that we'll define Solr's security model in such a
> > >> way that
> > >>>> we'll no longer treat arbitrary filesystem read/write or arbitrary
> > >> network access
> > >>>> as a vulnerability?
> > >>>
> > >>> Kindof, yeah, I think we should at least consider it. (Obviously
> > >>> communication-wise I wouldn't phrase it in terms of what reports we'd
> > >>> reject going forward. I'd frame it in terms of shifting where the
> > >>> protection is coming from. e.g. "Rather than relying on our security
> > >>> code, rely instead on proven security tooling like: <list of
> > >>> options>")
> > >>>
> > >>> I love everything about the ByteBuddy / security-agent approach, in a
> > >>> technical sense. It's a huge improvement over our current setup. But
> > >>> it does come with a maintenance cost and it's worth weighing that for
> > >>> ourselves. OpenSearch decided the tradeoff was worth it. But they
> > >>> have more corporate backing and likely more community bandwidth than
> > >>> we do, so we might weigh the same concerns and reach a different
> > >>> conclusion.
> > >>>
> > >>> If we're fully confident in the protections offered by security-agent,
> > >>> and we're OK spending at least some bandwidth on responding to future
> > >>> security-agent bugs, vuln reports, code maintenance, etc....great! If
> > >>> not, then maybe a more off-the-shelf approach to securing this stuff
> > >>> is a better fit for us.
> > >>>
> > >>> Best,
> > >>>
> > >>> Jason
> > >>>
> > >>> On Tue, Jun 9, 2026 at 7:51 AM Jan Høydahl <ja...@cominvent.com>
> > >> wrote:
> > >>>>
> > >>>> Could have tried to use stock OpenSearch agent, but they do things a
> > >> bit differently in the bootstrap and we may end up asking them to do
> > >> changes to be more generic and accommodate us.
> > >>>>
> > >>>> Another option is to start a new Apache commons sub project,
> > >> commons-sec-agent pull that as a dependency, and promote it for others to
> > >> use and help maintain.
> > >>>>
> > >>>> Jan Høydahl
> > >>>>
> > >>>>> 7. juni 2026 kl. 18:15 skrev David Smiley <ds...@apache.org>:
> > >>>>>
> > >>>>> Well written Jason! I agree 100% and I didn't communicate this
> > >> well enough
> > >>>>> previously when I used fewer words to essentially communicate the
> > >> same.
> > >>>>>
> > >>>>> I see the agent coming in SIP-24 is a forked copy, and thus we must
> > >>>>> maintain it. That's unfortunate. I hoped we could instead depend on
> > >>>>> another project regularly maintaining and producing artifacts for
> > >> 3rd party
> > >>>>> (us) consumption.
> > >>>>>
> > >>>>> Perhaps the agent will allow us to remove from Solr the
> > >> AllowUrlChecker
> > >>>>> (whatever it's called) and similar for file paths. I hope we get
> > >> some
> > >>>>> maintenance "relief".
> > >>>>>
> > >>>>>> On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <
> > >> gerlowskija@gmail.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>> Apologies for chiming in so late on this.
> > >>>>>>
> > >>>>>> The Java agent route seems like a huge improvement over our existing
> > >>>>>> approach. I love that it's a global protection - it doesn't rely on
> > >>>>>> devs remembering to call the required "check the path" method in
> > >> every
> > >>>>>> single place we might read a file in our code.
> > >>>>>>
> > >>>>>> But ultimately I still think systemd/seccomp is a much more
> > >>>>>> sustainable route for us than the Java-agent approach
> > >>>>>>
> > >>>>>> We're a Search community that's here to solve people's Search
> > >>>>>> problems. I won't minimize for a second all the effort we've put
> > >> into
> > >>>>>> security over the years. But there are entire projects that exist to
> > >>>>>> solve these sorts of problems. IMO steering users towards something
> > >>>>>> like systemd that has a whole community of security folks behind is
> > >>>>>> going to leave them more secure, and be more sustainable for us.
> > >>>>>>
> > >>>>>> (To be clear I'm not advocating for systemd specifically, but for
> > >> the
> > >>>>>> general idea of relying on well-known, OS-level protections that
> > >> exist
> > >>>>>> outside of the JVM and Solr.)
> > >>>>>>
> > >>>>>> I've heard two objections to the "systemd" approach so far: that
> > >>>>>> systemd isn't cross-platform, and that some folks won't bother to
> > >>>>>> enable it.
> > >>>>>>
> > >>>>>> To the first objection, I'd say that our responsibility has never
> > >> been
> > >>>>>> to find one security solution that works for everyone. We never
> > >>>>>> mandated that folks use a particular firewall: we let users choose
> > >> how
> > >>>>>> they wanted to provide that network isolation. IMO this could be
> > >>>>>> handled the same way: point folks to a few different tools,
> > >> optionally
> > >>>>>> provide a few example configs, and let them pick based on their
> > >>>>>> OS/platform.
> > >>>>>>
> > >>>>>> To the second objection: this is the whole point of having a
> > >> security
> > >>>>>> model. It's expected and normal for projects to describe how to
> > >> make
> > >>>>>> a deployment of their software "secure". It's neither fair nor
> > >>>>>> possible to secure folks who won't read our "Going to Production"
> > >> docs
> > >>>>>> before deploying.
> > >>>>>>
> > >>>>>>> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com>
> > >> wrote:
> > >>>>>>>
> > >>>>>>> So the implementation of SIP-24 is now ready for review at
> > >>>>>>>
> > >>>>>>> https://github.com/apache/solr/pull/4471 <https://github.com/apache/solr/pull/4471>
> > >>>>>>>
> > >>>>>>> The PR description gives through description of the change and
> > >> what to
> > >>>>>> look for.
> > >>>>>>> I have run several rounds of Copilot reviews and Claude reviews.
> > >>>>>>> We have unit tests and BATS tests, so coverage should be fairly
> > >> good.
> > >>>>>>>
> > >>>>>>> The PR is fairly large +5454 lines across 62 files and 54 commits,
> > >> but
> > >>>>>> mostly
> > >>>>>>> contained in the new gradle module, plus one hook in CoreContainer.
> > >>>>>>>
> > >>>>>>> Take it for a spin and let me know how it works.
> > >>>>>>>
> > >>>>>>> Jan
> > >>>>>>>
> > >>>>>>>> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> > >>>>>>>>
> > >>>>>>>> Hi,
> > >>>>>>>>
> > >>>>>>>> Welcoming more feedback on this approach.
> > >>>>>>>>
> > >>>>>>>> I'm planning to move to implementation phase on Wednesday, to get
> > >> a
> > >>>>>> first
> > >>>>>>>> version of the agent ready for testing. But it would be helpful
> > >> with
> > >>>>>> as much
> > >>>>>>>> feedback on the SIP high level design before I start implementing.
> > >>>>>> Thanks David
> > >>>>>>>> for your intial feedback.
> > >>>>>>>>
> > >>>>>>>> Jan
> > >>>>>>>>
> > >>>>>>>>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> > >>>>>>>>>
> > >>>>>>>>> I know it's probably unrealistic because corporate environments
> > >> into
> > >>>>>> which
> > >>>>>>>>> solr is deployed likely would have difficulty with it, but there
> > >> is a
> > >>>>>> JDK
> > >>>>>>>>> fork that keeps and improves the Security Manager... It's
> > >> supported
> > >>>>>> by the
> > >>>>>>>>> descendants of the Apache River project.
> > >>>>>>>>> https://github.com/pfirmstone/DirtyChai <https://github.com/pfirmstone/DirtyChai>
> > >>>>>>>>>
> > >>>>>>>>> Probably not useful, but perhaps interesting. Certainly we are
> > >> not
> > >>>>>> the only
> > >>>>>>>>> ones irritated by the loss of the security manager.
> > >>>>>>>>>
> > >>>>>>>>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <
> > >> jan.asf@cominvent.com>
> > >>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>>> I should clarify something. My objections mostly relate to the
> > >>>>>> insistent
> > >>>>>>>>>>> language in the SIP requiring Solr to have a substitute for the
> > >>>>>> JSM. I'm
> > >>>>>>>>>>> not quite against doing something but I might vote -0 on
> > >> something
> > >>>>>> that
> > >>>>>>>>>>> seems to have a poor security payoff relative to the
> > >> maintenance
> > >>>>>> burden.
> > >>>>>>>>>>> The more off-the-shelf, the better IMO.
> > >>>>>>>>>>> Thank you for investing your time researching some options.
> > >>>>>>>>>>
> > >>>>>>>>>> The security landscape has dramatically changed during the years
> > >>>>>> that we
> > >>>>>>>>>> have enjoyed JSM protection for Solr. It's a crazy world out
> > >> there
> > >>>>>> and
> > >>>>>>>>>> every
> > >>>>>>>>>> single code flaw, existing and future, will be found and
> > >> exploited or
> > >>>>>>>>>> published
> > >>>>>>>>>> by so called security researchers. That's why I believe we need
> > >> a
> > >>>>>>>>>> centralized
> > >>>>>>>>>> solution.
> > >>>>>>>>>>
> > >>>>>>>>>>>> Rejected Alternatives
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> - *Staying on Java < 24* — not viable long-term; Solr must
> > >> support
> > >>>>>>>>>>>> current Java LTS releases.
> > >>>>>>>>>>>> - *Removing JSM protections without any replacement* —
> > >> unacceptable
> > >>>>>>>>>>>> security regression.
> > >>>>>>>>>>>>
> > >>>>>>>>>>> Both Eric Pugh and I have challenged this.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> - *OS-level hardening only (systemd, seccomp)* — not
> > >>>>>> cross-platform;
> > >>>>>>>>>>>> does not cover Windows or macOS.
> > >>>>>>>>>>>>
> > >>>>>>>>>>> I challenge this. Why should the Solr project burden itself
> > >> with
> > >>>>>> building
> > >>>>>>>>>>> & maintaining security mechanisms already provided by
> > >> off-the-shelf
> > >>>>>>>>>> tools?
> > >>>>>>>>>>> If a user/operator wishes to run on Windows/macOS that may not
> > >> have
> > >>>>>> this
> > >>>>>>>>>>> protection mechanism, it is a risk consideration for that user
> > >> to
> > >>>>>>>>>> consider,
> > >>>>>>>>>>> but isn't a deal-breaker. The JSM wasn't/isn't a front-line
> > >>>>>> defense;
> > >>>>>>>>>> it's
> > >>>>>>>>>>> a defense-in-depth strategy. Put differently, the protections
> > >> here
> > >>>>>> are
> > >>>>>>>>>>> "best effort" but not worthy of a CVE if they were to falter.
> > >> I
> > >>>>>> want to
> > >>>>>>>>>>> get Arnout's opinion on this supposition.
> > >>>>>>>>>>
> > >>>>>>>>>> Java Security Manager is a beast, but its file system read/write
> > >>>>>> controls
> > >>>>>>>>>> have
> > >>>>>>>>>> saved us many a CVE in 9.x when JSM have been on by default.
> > >>>>>>>>>> The SIP primarily focuses on sandboxing Solr for wrt file and
> > >> network
> > >>>>>>>>>> access.
> > >>>>>>>>>> The security landscape has changed dramatically last few years,
> > >> and
> > >>>>>> solving
> > >>>>>>>>>> file- and network restrictions in a central place through
> > >>>>>> interception
> > >>>>>>>>>> rathen than
> > >>>>>>>>>> at each of the 100+ call sites is a more sustainable way.
> > >>>>>>>>>>
> > >>>>>>>>>> Users are free to add layers outside the JVM as well, such as
> > >>>>>> systemd,
> > >>>>>>>>>> container,
> > >>>>>>>>>> SELinux etc. But be honest - how many small/medium organizations
> > >>>>>> really do
> > >>>>>>>>>> this?
> > >>>>>>>>>>
> > >>>>>>>>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
> > >>>>>>>>>>>> significantly more complex; adds ZK client dependency to the
> > >> agent
> > >>>>>>>>>> JAR.
> > >>>>>>>>>>>> Superseded by port-based wildcards for intra-cluster traffic.
> > >>>>>>>>>>>> - *Building a Java agent from scratch* — higher effort with no
> > >>>>>>>>>>>> functional advantage over adapting the Apache 2.0-licensed
> > >>>>>> OpenSearch
> > >>>>>>>>>>>> implementation.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I agree with not burdening this project with building &
> > >>>>>> maintainining
> > >>>>>>>>>> such
> > >>>>>>>>>>> a mechanism.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I'm not sure, to what degree, we can leverage that existing
> > >> agent
> > >>>>>> you
> > >>>>>>>>>> speak
> > >>>>>>>>>>> of without further burdening us. It's a burden/reward
> > >> trade-off.
> > >>>>>>>>>>
> > >>>>>>>>>> If the agent becomes a true maintenance burden (i.e. a larger
> > >> burden
> > >>>>>> than
> > >>>>>>>>>> handling the CVEs it would prevent, then a valid action is to
> > >> remove
> > >>>>>> the
> > >>>>>>>>>> agent again.
> > >>>>>>>>>> Like any other feature we develop and maintain. Good this is
> > >> that
> > >>>>>> this is
> > >>>>>>>>>> pure Java,
> > >>>>>>>>>> and production-ready stable code.
> > >>>>>>>>>>
> > >>>>>>>>>>> In your updated proposal, you point
> > >>>>>>>>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed That
> > >> is
> > >>>>>> called
> > >>>>>>>>>> by
> > >>>>>>>>>>> a number of places... although I *think* the original
> > >> intention was
> > >>>>>> only
> > >>>>>>>>>> to
> > >>>>>>>>>>> limit where cores are created? Can you elaborate on what the
> > >> role
> > >>>>>> of
> > >>>>>>>>>> that
> > >>>>>>>>>>> method *should* be and how the JSM might also or work?
> > >>>>>>>>>>
> > >>>>>>>>>> The pathAllowed checks (also the ones pre-dating the
> > >> centralization
> > >>>>>> in
> > >>>>>>>>>> CoreContainer,
> > >>>>>>>>>> were introduced in 8.x, before JSM was introduced or enabled by
> > >>>>>> default.
> > >>>>>>>>>> Initially
> > >>>>>>>>>> assertPathAllowed would validate end-user API input to avoid
> > >> cores
> > >>>>>> being
> > >>>>>>>>>> created
> > >>>>>>>>>> or loaded outside blessed folders. It has then been re-used for
> > >>>>>> other code
> > >>>>>>>>>> locations
> > >>>>>>>>>> that may do file access based on user-input in API or config.
> > >> It as
> > >>>>>> also
> > >>>>>>>>>> extended to
> > >>>>>>>>>> block UNC paths after many attacks with this as a vector.
> > >>>>>>>>>>
> > >>>>>>>>>> I believe perhaps pathAllowed would not have been written had
> > >> JSM
> > >>>>>> already
> > >>>>>>>>>> been
> > >>>>>>>>>> enforced. Although I don't know if you can block UNC with JSM?
> > >>>>>>>>>> The SIP does not propose to remove pathAllowed now. One benefit
> > >> of
> > >>>>>> early
> > >>>>>>>>>> detection
> > >>>>>>>>>> is that we can give better context-sensitive error- and log
> > >> messages
> > >>>>>>>>>> rather than throwing
> > >>>>>>>>>> an exception.
> > >>>>>>>>>>
> > >>>>>>>>>> The agent approach laid out sandboxes four attack vectors
> > >>>>>>>>>> * File access outside a limited set of folders and full block of
> > >>>>>> Windows
> > >>>>>>>>>> UNC
> > >>>>>>>>>> * Network access other than peer solr nodes and zk nodes
> > >>>>>>>>>> * Disabllow calling System.exit() (mainly useful for 3rd party
> > >>>>>> plugins?)
> > >>>>>>>>>> * Disallow spawning processes
> > >>>>>>>>>>
> > >>>>>>>>>> A nice thing with the agent approach is that it is unintrusive
> > >> and
> > >>>>>> easy to
> > >>>>>>>>>> disable,
> > >>>>>>>>>> so users who want to take care of all this on OS level may
> > >> disable
> > >>>>>> it, and
> > >>>>>>>>>> if we or
> > >>>>>>>>>> users don't find value in it down the road they can disable it
> > >> or we
> > >>>>>> can
> > >>>>>>>>>> remove it.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Jan
> > >>>>>>>>>>
> > >> ---------------------------------------------------------------------
> > >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >>>>>>>>>> For additional commands, e-mail: dev-help@solr.apache.org
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> http://www.needhamsoftware.com <http://www.needhamsoftware.com> (work)
> > >>>>>>>>> https://a.co/d/b2sZLD9 <https://a.co/d/b2sZLD9> (my fantasy fiction book)
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >> ---------------------------------------------------------------------
> > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >>>>>> For additional commands, e-mail: dev-help@solr.apache.org
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>> ---------------------------------------------------------------------
> > >>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >>>> For additional commands, e-mail: dev-help@solr.apache.org
> > >>>>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >> For additional commands, e-mail: dev-help@solr.apache.org
> > >>
> > >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast, a leader in email security and cyber resilience. Mimecast integrates email defenses with brand protection, security awareness training, web security, compliance and other essential capabilities. Mimecast helps protect large and small organizations from malicious activity, human error and technology failure; and to lead the movement toward building a more resilient world. To find out more, visit our website.

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jason Gerlowski <ge...@gmail.com>]

> What is the way forward? My proposal would be to proceed with SIP-24 as an ADDED layer
of defense ASAP. If it after a year turns out it is hard to maintain
or otherwise not worth it,
just drop it.

I don't expect we'll have much feedback-wise in a year, given how long
it takes many folks to upgrade.  But otherwise this SGTM.

On the network side in particular I have qualms about taking ownership
of this stuff when there's great third-party firewalls out there.  But
lacking broad community agreement on that point I don't want to let
"perfect be the enemy of good" here.  SIP-24 is undoubtedly an
improvement, so let's move that direction!

On Fri, Jun 19, 2026 at 5:10 AM Jan Høydahl <ja...@cominvent.com> wrote:
>
> Hi,
>
> Thanks for the feedbacks and reviews David, Eric, Gus and Jason. Trying to sum up:
>
> Core principle is we'd like to push as much of network protection, i.e. who can talk to who,
> downstream to user's network environment, firewalls etc.
>
> Encourage users to utilize OS primitives to sandbox the Solr process, whether that be
> linux seccomp, CAP_* controls and filesystem isolation or similar on other OS'es.
>
> It was discussed whether we can frame our new Threat Model in a way that redefines some
> attack vectors to be the responsibility of the USER. But we cannot push everything on them.
>
>
> Even if we encourage the above, and it will limit the blast radius of a successfuly exploited
> code vulnerability in Solr, I still sense an agreement that we do want to avoid app-level
> code in Solr that allows unintended filesystem path traversal, file read outside known
> solr areas, file write outside index or backup folders etc. And also that our current allowPaths
> strategy is spotty and that the agent approach centralises and improves upon this.
>
> The utility for intercepting outbound network traffic in the agent is a balancing act between
> convenience and security, and it overlaps partly with external firewall configs. However, it is
> not uncommon to deploy Solr on the same network segment as databases and other services.
> The agent gives us and users a tool to contain a possible SSRF resulting from a Solr bug.
>
>
> So given the above, if we want to give SIP-24 and an agent a try, the concerns flagges are
> twofold - more code to maintain and more code that could break.
>
> Wrt "more code to maintain" an option would be to gather core agent code into a separate
> project, perhaps shared with Opensearch and others with similar needs.
>
> Wrt "more code that could break", yes there could be bugs, but I'd argue that on JDK25 without
> JSM, the new agent would not increase attack surface, but there could be bugs making it
> fail to protect certain cases we thought were protected.
>
>
> What is the way forward? My proposal would be to proceed with SIP-24 as an ADDED layer
> of defense ASAP. If it after a year turns out it is hard to maintain or otherwise not worth it,
> just drop it. Once released, investigate possibility of lifting the core agent out into a new
> project (apache-commons?) and refactor if successful in a later release. Then we get the
> benefit of added protection immediately and get a delayed benefit of lower maintenance.
>
> Jan
>
>
>
> > 10. juni 2026 kl. 17:01 skrev David Smiley <ds...@apache.org>:
> >
> > Presumably the OpenSearch project would also appreciate reduced maintenance
> > if this aspect of their project were extracted to an independent project.
> > It's worth chatting with them about this, yeah?  It's also maybe exciting
> > for those involved in that mechanism to get more exposure -- more projects
> > using their cool stuff.
> >
> > On Wed, Jun 10, 2026 at 4:01 AM Jason Gerlowski <ge...@gmail.com>
> > wrote:
> >
> >> To clarify - I'm not assuming any particular answer to that question.
> >> If folks think the module won't be high-maintenance, then +1.  I just
> >> want to make sure the question gets asked is all.
> >>
> >> (And if we're not sure about the maintenance implications, IMO it'd be
> >> a great option to move forward with SIP-24 and aim to re-evaluate in a
> >> year or two when we do have a better sense.  I see SIP-24 as a clear
> >> improvement, so we shouldn't let "perfect be the enemy of good" here.)
> >>
> >> On Wed, Jun 10, 2026 at 9:54 AM Jason Gerlowski <ge...@gmail.com>
> >> wrote:
> >>>
> >>>> Or are you hinting that we'll define Solr's security model in such a
> >> way that
> >>>> we'll no longer treat arbitrary filesystem read/write or arbitrary
> >> network access
> >>>> as a vulnerability?
> >>>
> >>> Kindof, yeah, I think we should at least consider it.  (Obviously
> >>> communication-wise I wouldn't phrase it in terms of what reports we'd
> >>> reject going forward.  I'd frame it in terms of shifting where the
> >>> protection is coming from.  e.g. "Rather than relying on our security
> >>> code, rely instead on proven security tooling like: <list of
> >>> options>")
> >>>
> >>> I love everything about the ByteBuddy / security-agent approach, in a
> >>> technical sense.  It's a huge improvement over our current setup.  But
> >>> it does come with a maintenance cost and it's worth weighing that for
> >>> ourselves.  OpenSearch decided the tradeoff was worth it.  But they
> >>> have more corporate backing and likely more community bandwidth than
> >>> we do, so we might weigh the same concerns and reach a different
> >>> conclusion.
> >>>
> >>> If we're fully confident in the protections offered by security-agent,
> >>> and we're OK spending at least some bandwidth on responding to future
> >>> security-agent bugs, vuln reports, code maintenance, etc....great!  If
> >>> not, then maybe a more off-the-shelf approach to securing this stuff
> >>> is a better fit for us.
> >>>
> >>> Best,
> >>>
> >>> Jason
> >>>
> >>> On Tue, Jun 9, 2026 at 7:51 AM Jan Høydahl <ja...@cominvent.com>
> >> wrote:
> >>>>
> >>>> Could have tried to use stock OpenSearch agent, but they do things a
> >> bit differently in the bootstrap and we may end up asking them to do
> >> changes to be more generic and accommodate us.
> >>>>
> >>>> Another option is to start a new Apache commons sub project,
> >> commons-sec-agent pull that as a dependency, and promote it for others to
> >> use and help maintain.
> >>>>
> >>>> Jan Høydahl
> >>>>
> >>>>> 7. juni 2026 kl. 18:15 skrev David Smiley <ds...@apache.org>:
> >>>>>
> >>>>> Well written Jason!  I agree 100% and I didn't communicate this
> >> well enough
> >>>>> previously when I used fewer words to essentially communicate the
> >> same.
> >>>>>
> >>>>> I see the agent coming in SIP-24 is a forked copy, and thus we must
> >>>>> maintain it.  That's unfortunate.  I hoped we could instead depend on
> >>>>> another project regularly maintaining and producing artifacts for
> >> 3rd party
> >>>>> (us) consumption.
> >>>>>
> >>>>> Perhaps the agent will allow us to remove from Solr the
> >> AllowUrlChecker
> >>>>> (whatever it's called) and similar for file paths.  I hope we get
> >> some
> >>>>> maintenance "relief".
> >>>>>
> >>>>>> On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <
> >> gerlowskija@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Apologies for chiming in so late on this.
> >>>>>>
> >>>>>> The Java agent route seems like a huge improvement over our existing
> >>>>>> approach.  I love that it's a global protection - it doesn't rely on
> >>>>>> devs remembering to call the required "check the path" method in
> >> every
> >>>>>> single place we might read a file in our code.
> >>>>>>
> >>>>>> But ultimately I still think systemd/seccomp is a much more
> >>>>>> sustainable route for us than the Java-agent approach
> >>>>>>
> >>>>>> We're a Search community that's here to solve people's Search
> >>>>>> problems.  I won't minimize for a second all the effort we've put
> >> into
> >>>>>> security over the years. But there are entire projects that exist to
> >>>>>> solve these sorts of problems.  IMO steering users towards something
> >>>>>> like systemd that has a whole community of security folks behind is
> >>>>>> going to leave them more secure, and be more sustainable for us.
> >>>>>>
> >>>>>> (To be clear I'm not advocating for systemd specifically, but for
> >> the
> >>>>>> general idea of relying on well-known, OS-level protections that
> >> exist
> >>>>>> outside of the JVM and Solr.)
> >>>>>>
> >>>>>> I've heard two objections to the "systemd" approach so far: that
> >>>>>> systemd isn't cross-platform, and that some folks won't bother to
> >>>>>> enable it.
> >>>>>>
> >>>>>> To the first objection, I'd say that our responsibility has never
> >> been
> >>>>>> to find one security solution that works for everyone.  We never
> >>>>>> mandated that folks use a particular firewall: we let users choose
> >> how
> >>>>>> they wanted to provide that network isolation.  IMO this could be
> >>>>>> handled the same way: point folks to a few different tools,
> >> optionally
> >>>>>> provide a few example configs, and let them pick based on their
> >>>>>> OS/platform.
> >>>>>>
> >>>>>> To the second objection: this is the whole point of having a
> >> security
> >>>>>> model.  It's expected and normal for projects to describe how to
> >> make
> >>>>>> a deployment of their software "secure".  It's neither fair nor
> >>>>>> possible to secure folks who won't read our "Going to Production"
> >> docs
> >>>>>> before deploying.
> >>>>>>
> >>>>>>> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com>
> >> wrote:
> >>>>>>>
> >>>>>>> So the implementation of SIP-24 is now ready for review at
> >>>>>>>
> >>>>>>> https://github.com/apache/solr/pull/4471
> >>>>>>>
> >>>>>>> The PR description gives through description of the change and
> >> what to
> >>>>>> look for.
> >>>>>>> I have run several rounds of Copilot reviews and Claude reviews.
> >>>>>>> We have unit tests and BATS tests, so coverage should be fairly
> >> good.
> >>>>>>>
> >>>>>>> The PR is fairly large +5454 lines across 62 files and 54 commits,
> >> but
> >>>>>> mostly
> >>>>>>> contained in the new gradle module, plus one hook in CoreContainer.
> >>>>>>>
> >>>>>>> Take it for a spin and let me know how it works.
> >>>>>>>
> >>>>>>> Jan
> >>>>>>>
> >>>>>>>> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> >>>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> Welcoming more feedback on this approach.
> >>>>>>>>
> >>>>>>>> I'm planning to move to implementation phase on Wednesday, to get
> >> a
> >>>>>> first
> >>>>>>>> version of the agent ready for testing. But it would be helpful
> >> with
> >>>>>> as much
> >>>>>>>> feedback on the SIP high level design before I start implementing.
> >>>>>> Thanks David
> >>>>>>>> for your intial feedback.
> >>>>>>>>
> >>>>>>>> Jan
> >>>>>>>>
> >>>>>>>>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> >>>>>>>>>
> >>>>>>>>> I know it's probably unrealistic because corporate environments
> >> into
> >>>>>> which
> >>>>>>>>> solr is deployed likely would have difficulty with it, but there
> >> is a
> >>>>>> JDK
> >>>>>>>>> fork that keeps and improves the Security Manager... It's
> >> supported
> >>>>>> by the
> >>>>>>>>> descendants of the Apache River project.
> >>>>>>>>> https://github.com/pfirmstone/DirtyChai
> >>>>>>>>>
> >>>>>>>>> Probably not useful, but perhaps interesting. Certainly we are
> >> not
> >>>>>> the only
> >>>>>>>>> ones irritated by the loss of the security manager.
> >>>>>>>>>
> >>>>>>>>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <
> >> jan.asf@cominvent.com>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>> I should clarify something.  My objections mostly relate to the
> >>>>>> insistent
> >>>>>>>>>>> language in the SIP requiring Solr to have a substitute for the
> >>>>>> JSM.  I'm
> >>>>>>>>>>> not quite against doing something but I might vote -0 on
> >> something
> >>>>>> that
> >>>>>>>>>>> seems to have a poor security payoff relative to the
> >> maintenance
> >>>>>> burden.
> >>>>>>>>>>> The more off-the-shelf, the better IMO.
> >>>>>>>>>>> Thank you for investing your time researching some options.
> >>>>>>>>>>
> >>>>>>>>>> The security landscape has dramatically changed during the years
> >>>>>> that we
> >>>>>>>>>> have enjoyed JSM protection for Solr. It's a crazy world out
> >> there
> >>>>>> and
> >>>>>>>>>> every
> >>>>>>>>>> single code flaw, existing and future, will be found and
> >> exploited or
> >>>>>>>>>> published
> >>>>>>>>>> by so called security researchers. That's why I believe we need
> >> a
> >>>>>>>>>> centralized
> >>>>>>>>>> solution.
> >>>>>>>>>>
> >>>>>>>>>>>> Rejected Alternatives
> >>>>>>>>>>>>
> >>>>>>>>>>>> - *Staying on Java < 24* — not viable long-term; Solr must
> >> support
> >>>>>>>>>>>> current Java LTS releases.
> >>>>>>>>>>>> - *Removing JSM protections without any replacement* —
> >> unacceptable
> >>>>>>>>>>>> security regression.
> >>>>>>>>>>>>
> >>>>>>>>>>> Both Eric Pugh and I have challenged this.
> >>>>>>>>>>>>
> >>>>>>>>>>>> - *OS-level hardening only (systemd, seccomp)* — not
> >>>>>> cross-platform;
> >>>>>>>>>>>> does not cover Windows or macOS.
> >>>>>>>>>>>>
> >>>>>>>>>>> I challenge this. Why should the Solr project burden itself
> >> with
> >>>>>> building
> >>>>>>>>>>> & maintaining security mechanisms already provided by
> >> off-the-shelf
> >>>>>>>>>> tools?
> >>>>>>>>>>> If a user/operator wishes to run on Windows/macOS that may not
> >> have
> >>>>>> this
> >>>>>>>>>>> protection mechanism, it is a risk consideration for that user
> >> to
> >>>>>>>>>> consider,
> >>>>>>>>>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line
> >>>>>> defense;
> >>>>>>>>>> it's
> >>>>>>>>>>> a defense-in-depth strategy.  Put differently, the protections
> >> here
> >>>>>> are
> >>>>>>>>>>> "best effort" but not worthy of a CVE if they were to falter.
> >> I
> >>>>>> want to
> >>>>>>>>>>> get Arnout's opinion on this supposition.
> >>>>>>>>>>
> >>>>>>>>>> Java Security Manager is a beast, but its file system read/write
> >>>>>> controls
> >>>>>>>>>> have
> >>>>>>>>>> saved us many a CVE in 9.x when JSM have been on by default.
> >>>>>>>>>> The SIP primarily focuses on sandboxing Solr for wrt file and
> >> network
> >>>>>>>>>> access.
> >>>>>>>>>> The security landscape has changed dramatically last few years,
> >> and
> >>>>>> solving
> >>>>>>>>>> file- and network restrictions in a central place through
> >>>>>> interception
> >>>>>>>>>> rathen than
> >>>>>>>>>> at each of the 100+ call sites is a more sustainable way.
> >>>>>>>>>>
> >>>>>>>>>> Users are free to add layers outside the JVM as well, such as
> >>>>>> systemd,
> >>>>>>>>>> container,
> >>>>>>>>>> SELinux etc. But be honest - how many small/medium organizations
> >>>>>> really do
> >>>>>>>>>> this?
> >>>>>>>>>>
> >>>>>>>>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
> >>>>>>>>>>>> significantly more complex; adds ZK client dependency to the
> >> agent
> >>>>>>>>>> JAR.
> >>>>>>>>>>>> Superseded by port-based wildcards for intra-cluster traffic.
> >>>>>>>>>>>> - *Building a Java agent from scratch* — higher effort with no
> >>>>>>>>>>>> functional advantage over adapting the Apache 2.0-licensed
> >>>>>> OpenSearch
> >>>>>>>>>>>> implementation.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I agree with not burdening this project with building &
> >>>>>> maintainining
> >>>>>>>>>> such
> >>>>>>>>>>> a mechanism.
> >>>>>>>>>>>
> >>>>>>>>>>> I'm not sure, to what degree, we can leverage that existing
> >> agent
> >>>>>> you
> >>>>>>>>>> speak
> >>>>>>>>>>> of without further burdening us.  It's a burden/reward
> >> trade-off.
> >>>>>>>>>>
> >>>>>>>>>> If the agent becomes a true maintenance burden (i.e. a larger
> >> burden
> >>>>>> than
> >>>>>>>>>> handling the CVEs it would prevent, then a valid action is to
> >> remove
> >>>>>> the
> >>>>>>>>>> agent again.
> >>>>>>>>>> Like any other feature we develop and maintain. Good this is
> >> that
> >>>>>> this is
> >>>>>>>>>> pure Java,
> >>>>>>>>>> and production-ready stable code.
> >>>>>>>>>>
> >>>>>>>>>>> In your updated proposal, you point
> >>>>>>>>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That
> >> is
> >>>>>> called
> >>>>>>>>>> by
> >>>>>>>>>>> a number of places... although I *think* the original
> >> intention was
> >>>>>> only
> >>>>>>>>>> to
> >>>>>>>>>>> limit where cores are created?  Can you elaborate on what the
> >> role
> >>>>>> of
> >>>>>>>>>> that
> >>>>>>>>>>> method *should* be and how the JSM might also or work?
> >>>>>>>>>>
> >>>>>>>>>> The pathAllowed checks (also the ones pre-dating the
> >> centralization
> >>>>>> in
> >>>>>>>>>> CoreContainer,
> >>>>>>>>>> were introduced in 8.x, before JSM was introduced or enabled by
> >>>>>> default.
> >>>>>>>>>> Initially
> >>>>>>>>>> assertPathAllowed would validate end-user API input to avoid
> >> cores
> >>>>>> being
> >>>>>>>>>> created
> >>>>>>>>>> or loaded outside blessed folders. It has then been re-used for
> >>>>>> other code
> >>>>>>>>>> locations
> >>>>>>>>>> that may do file access based on user-input in API or config.
> >> It as
> >>>>>> also
> >>>>>>>>>> extended to
> >>>>>>>>>> block UNC paths after many attacks with this as a vector.
> >>>>>>>>>>
> >>>>>>>>>> I believe perhaps pathAllowed would not have been written had
> >> JSM
> >>>>>> already
> >>>>>>>>>> been
> >>>>>>>>>> enforced. Although I don't know if you can block UNC with JSM?
> >>>>>>>>>> The SIP does not propose to remove pathAllowed now. One benefit
> >> of
> >>>>>> early
> >>>>>>>>>> detection
> >>>>>>>>>> is that we can give better context-sensitive error- and log
> >> messages
> >>>>>>>>>> rather than throwing
> >>>>>>>>>> an exception.
> >>>>>>>>>>
> >>>>>>>>>> The agent approach laid out sandboxes four attack vectors
> >>>>>>>>>> * File access outside a limited set of folders and full block of
> >>>>>> Windows
> >>>>>>>>>> UNC
> >>>>>>>>>> * Network access other than peer solr nodes and zk nodes
> >>>>>>>>>> * Disabllow calling System.exit() (mainly useful for 3rd party
> >>>>>> plugins?)
> >>>>>>>>>> * Disallow spawning processes
> >>>>>>>>>>
> >>>>>>>>>> A nice thing with the agent approach is that it is unintrusive
> >> and
> >>>>>> easy to
> >>>>>>>>>> disable,
> >>>>>>>>>> so users who want to take care of all this on OS level may
> >> disable
> >>>>>> it, and
> >>>>>>>>>> if we or
> >>>>>>>>>> users don't find value in it down the road they can disable it
> >> or we
> >>>>>> can
> >>>>>>>>>> remove it.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Jan
> >>>>>>>>>>
> >> ---------------------------------------------------------------------
> >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>>>>>>>>> For additional commands, e-mail: dev-help@solr.apache.org
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> http://www.needhamsoftware.com (work)
> >>>>>>>>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>>>>> For additional commands, e-mail: dev-help@solr.apache.org
> >>>>>>
> >>>>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>>> For additional commands, e-mail: dev-help@solr.apache.org
> >>>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >> For additional commands, e-mail: dev-help@solr.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jan Høydahl <ja...@cominvent.com>]

Hi,

Thanks for the feedbacks and reviews David, Eric, Gus and Jason. Trying to sum up:

Core principle is we'd like to push as much of network protection, i.e. who can talk to who,
downstream to user's network environment, firewalls etc.

Encourage users to utilize OS primitives to sandbox the Solr process, whether that be
linux seccomp, CAP_* controls and filesystem isolation or similar on other OS'es.

It was discussed whether we can frame our new Threat Model in a way that redefines some
attack vectors to be the responsibility of the USER. But we cannot push everything on them.


Even if we encourage the above, and it will limit the blast radius of a successfuly exploited
code vulnerability in Solr, I still sense an agreement that we do want to avoid app-level
code in Solr that allows unintended filesystem path traversal, file read outside known
solr areas, file write outside index or backup folders etc. And also that our current allowPaths 
strategy is spotty and that the agent approach centralises and improves upon this.

The utility for intercepting outbound network traffic in the agent is a balancing act between
convenience and security, and it overlaps partly with external firewall configs. However, it is
not uncommon to deploy Solr on the same network segment as databases and other services.
The agent gives us and users a tool to contain a possible SSRF resulting from a Solr bug.


So given the above, if we want to give SIP-24 and an agent a try, the concerns flagges are
twofold - more code to maintain and more code that could break.

Wrt "more code to maintain" an option would be to gather core agent code into a separate
project, perhaps shared with Opensearch and others with similar needs.

Wrt "more code that could break", yes there could be bugs, but I'd argue that on JDK25 without
JSM, the new agent would not increase attack surface, but there could be bugs making it
fail to protect certain cases we thought were protected.


What is the way forward? My proposal would be to proceed with SIP-24 as an ADDED layer
of defense ASAP. If it after a year turns out it is hard to maintain or otherwise not worth it, 
just drop it. Once released, investigate possibility of lifting the core agent out into a new
project (apache-commons?) and refactor if successful in a later release. Then we get the
benefit of added protection immediately and get a delayed benefit of lower maintenance.

Jan



> 10. juni 2026 kl. 17:01 skrev David Smiley <ds...@apache.org>:
> 
> Presumably the OpenSearch project would also appreciate reduced maintenance
> if this aspect of their project were extracted to an independent project.
> It's worth chatting with them about this, yeah?  It's also maybe exciting
> for those involved in that mechanism to get more exposure -- more projects
> using their cool stuff.
> 
> On Wed, Jun 10, 2026 at 4:01 AM Jason Gerlowski <ge...@gmail.com>
> wrote:
> 
>> To clarify - I'm not assuming any particular answer to that question.
>> If folks think the module won't be high-maintenance, then +1.  I just
>> want to make sure the question gets asked is all.
>> 
>> (And if we're not sure about the maintenance implications, IMO it'd be
>> a great option to move forward with SIP-24 and aim to re-evaluate in a
>> year or two when we do have a better sense.  I see SIP-24 as a clear
>> improvement, so we shouldn't let "perfect be the enemy of good" here.)
>> 
>> On Wed, Jun 10, 2026 at 9:54 AM Jason Gerlowski <ge...@gmail.com>
>> wrote:
>>> 
>>>> Or are you hinting that we'll define Solr's security model in such a
>> way that
>>>> we'll no longer treat arbitrary filesystem read/write or arbitrary
>> network access
>>>> as a vulnerability?
>>> 
>>> Kindof, yeah, I think we should at least consider it.  (Obviously
>>> communication-wise I wouldn't phrase it in terms of what reports we'd
>>> reject going forward.  I'd frame it in terms of shifting where the
>>> protection is coming from.  e.g. "Rather than relying on our security
>>> code, rely instead on proven security tooling like: <list of
>>> options>")
>>> 
>>> I love everything about the ByteBuddy / security-agent approach, in a
>>> technical sense.  It's a huge improvement over our current setup.  But
>>> it does come with a maintenance cost and it's worth weighing that for
>>> ourselves.  OpenSearch decided the tradeoff was worth it.  But they
>>> have more corporate backing and likely more community bandwidth than
>>> we do, so we might weigh the same concerns and reach a different
>>> conclusion.
>>> 
>>> If we're fully confident in the protections offered by security-agent,
>>> and we're OK spending at least some bandwidth on responding to future
>>> security-agent bugs, vuln reports, code maintenance, etc....great!  If
>>> not, then maybe a more off-the-shelf approach to securing this stuff
>>> is a better fit for us.
>>> 
>>> Best,
>>> 
>>> Jason
>>> 
>>> On Tue, Jun 9, 2026 at 7:51 AM Jan Høydahl <ja...@cominvent.com>
>> wrote:
>>>> 
>>>> Could have tried to use stock OpenSearch agent, but they do things a
>> bit differently in the bootstrap and we may end up asking them to do
>> changes to be more generic and accommodate us.
>>>> 
>>>> Another option is to start a new Apache commons sub project,
>> commons-sec-agent pull that as a dependency, and promote it for others to
>> use and help maintain.
>>>> 
>>>> Jan Høydahl
>>>> 
>>>>> 7. juni 2026 kl. 18:15 skrev David Smiley <ds...@apache.org>:
>>>>> 
>>>>> Well written Jason!  I agree 100% and I didn't communicate this
>> well enough
>>>>> previously when I used fewer words to essentially communicate the
>> same.
>>>>> 
>>>>> I see the agent coming in SIP-24 is a forked copy, and thus we must
>>>>> maintain it.  That's unfortunate.  I hoped we could instead depend on
>>>>> another project regularly maintaining and producing artifacts for
>> 3rd party
>>>>> (us) consumption.
>>>>> 
>>>>> Perhaps the agent will allow us to remove from Solr the
>> AllowUrlChecker
>>>>> (whatever it's called) and similar for file paths.  I hope we get
>> some
>>>>> maintenance "relief".
>>>>> 
>>>>>> On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <
>> gerlowskija@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>> Apologies for chiming in so late on this.
>>>>>> 
>>>>>> The Java agent route seems like a huge improvement over our existing
>>>>>> approach.  I love that it's a global protection - it doesn't rely on
>>>>>> devs remembering to call the required "check the path" method in
>> every
>>>>>> single place we might read a file in our code.
>>>>>> 
>>>>>> But ultimately I still think systemd/seccomp is a much more
>>>>>> sustainable route for us than the Java-agent approach
>>>>>> 
>>>>>> We're a Search community that's here to solve people's Search
>>>>>> problems.  I won't minimize for a second all the effort we've put
>> into
>>>>>> security over the years. But there are entire projects that exist to
>>>>>> solve these sorts of problems.  IMO steering users towards something
>>>>>> like systemd that has a whole community of security folks behind is
>>>>>> going to leave them more secure, and be more sustainable for us.
>>>>>> 
>>>>>> (To be clear I'm not advocating for systemd specifically, but for
>> the
>>>>>> general idea of relying on well-known, OS-level protections that
>> exist
>>>>>> outside of the JVM and Solr.)
>>>>>> 
>>>>>> I've heard two objections to the "systemd" approach so far: that
>>>>>> systemd isn't cross-platform, and that some folks won't bother to
>>>>>> enable it.
>>>>>> 
>>>>>> To the first objection, I'd say that our responsibility has never
>> been
>>>>>> to find one security solution that works for everyone.  We never
>>>>>> mandated that folks use a particular firewall: we let users choose
>> how
>>>>>> they wanted to provide that network isolation.  IMO this could be
>>>>>> handled the same way: point folks to a few different tools,
>> optionally
>>>>>> provide a few example configs, and let them pick based on their
>>>>>> OS/platform.
>>>>>> 
>>>>>> To the second objection: this is the whole point of having a
>> security
>>>>>> model.  It's expected and normal for projects to describe how to
>> make
>>>>>> a deployment of their software "secure".  It's neither fair nor
>>>>>> possible to secure folks who won't read our "Going to Production"
>> docs
>>>>>> before deploying.
>>>>>> 
>>>>>>> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com>
>> wrote:
>>>>>>> 
>>>>>>> So the implementation of SIP-24 is now ready for review at
>>>>>>> 
>>>>>>> https://github.com/apache/solr/pull/4471
>>>>>>> 
>>>>>>> The PR description gives through description of the change and
>> what to
>>>>>> look for.
>>>>>>> I have run several rounds of Copilot reviews and Claude reviews.
>>>>>>> We have unit tests and BATS tests, so coverage should be fairly
>> good.
>>>>>>> 
>>>>>>> The PR is fairly large +5454 lines across 62 files and 54 commits,
>> but
>>>>>> mostly
>>>>>>> contained in the new gradle module, plus one hook in CoreContainer.
>>>>>>> 
>>>>>>> Take it for a spin and let me know how it works.
>>>>>>> 
>>>>>>> Jan
>>>>>>> 
>>>>>>>> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
>>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> Welcoming more feedback on this approach.
>>>>>>>> 
>>>>>>>> I'm planning to move to implementation phase on Wednesday, to get
>> a
>>>>>> first
>>>>>>>> version of the agent ready for testing. But it would be helpful
>> with
>>>>>> as much
>>>>>>>> feedback on the SIP high level design before I start implementing.
>>>>>> Thanks David
>>>>>>>> for your intial feedback.
>>>>>>>> 
>>>>>>>> Jan
>>>>>>>> 
>>>>>>>>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
>>>>>>>>> 
>>>>>>>>> I know it's probably unrealistic because corporate environments
>> into
>>>>>> which
>>>>>>>>> solr is deployed likely would have difficulty with it, but there
>> is a
>>>>>> JDK
>>>>>>>>> fork that keeps and improves the Security Manager... It's
>> supported
>>>>>> by the
>>>>>>>>> descendants of the Apache River project.
>>>>>>>>> https://github.com/pfirmstone/DirtyChai
>>>>>>>>> 
>>>>>>>>> Probably not useful, but perhaps interesting. Certainly we are
>> not
>>>>>> the only
>>>>>>>>> ones irritated by the loss of the security manager.
>>>>>>>>> 
>>>>>>>>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <
>> jan.asf@cominvent.com>
>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>>> I should clarify something.  My objections mostly relate to the
>>>>>> insistent
>>>>>>>>>>> language in the SIP requiring Solr to have a substitute for the
>>>>>> JSM.  I'm
>>>>>>>>>>> not quite against doing something but I might vote -0 on
>> something
>>>>>> that
>>>>>>>>>>> seems to have a poor security payoff relative to the
>> maintenance
>>>>>> burden.
>>>>>>>>>>> The more off-the-shelf, the better IMO.
>>>>>>>>>>> Thank you for investing your time researching some options.
>>>>>>>>>> 
>>>>>>>>>> The security landscape has dramatically changed during the years
>>>>>> that we
>>>>>>>>>> have enjoyed JSM protection for Solr. It's a crazy world out
>> there
>>>>>> and
>>>>>>>>>> every
>>>>>>>>>> single code flaw, existing and future, will be found and
>> exploited or
>>>>>>>>>> published
>>>>>>>>>> by so called security researchers. That's why I believe we need
>> a
>>>>>>>>>> centralized
>>>>>>>>>> solution.
>>>>>>>>>> 
>>>>>>>>>>>> Rejected Alternatives
>>>>>>>>>>>> 
>>>>>>>>>>>> - *Staying on Java < 24* — not viable long-term; Solr must
>> support
>>>>>>>>>>>> current Java LTS releases.
>>>>>>>>>>>> - *Removing JSM protections without any replacement* —
>> unacceptable
>>>>>>>>>>>> security regression.
>>>>>>>>>>>> 
>>>>>>>>>>> Both Eric Pugh and I have challenged this.
>>>>>>>>>>>> 
>>>>>>>>>>>> - *OS-level hardening only (systemd, seccomp)* — not
>>>>>> cross-platform;
>>>>>>>>>>>> does not cover Windows or macOS.
>>>>>>>>>>>> 
>>>>>>>>>>> I challenge this. Why should the Solr project burden itself
>> with
>>>>>> building
>>>>>>>>>>> & maintaining security mechanisms already provided by
>> off-the-shelf
>>>>>>>>>> tools?
>>>>>>>>>>> If a user/operator wishes to run on Windows/macOS that may not
>> have
>>>>>> this
>>>>>>>>>>> protection mechanism, it is a risk consideration for that user
>> to
>>>>>>>>>> consider,
>>>>>>>>>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line
>>>>>> defense;
>>>>>>>>>> it's
>>>>>>>>>>> a defense-in-depth strategy.  Put differently, the protections
>> here
>>>>>> are
>>>>>>>>>>> "best effort" but not worthy of a CVE if they were to falter.
>> I
>>>>>> want to
>>>>>>>>>>> get Arnout's opinion on this supposition.
>>>>>>>>>> 
>>>>>>>>>> Java Security Manager is a beast, but its file system read/write
>>>>>> controls
>>>>>>>>>> have
>>>>>>>>>> saved us many a CVE in 9.x when JSM have been on by default.
>>>>>>>>>> The SIP primarily focuses on sandboxing Solr for wrt file and
>> network
>>>>>>>>>> access.
>>>>>>>>>> The security landscape has changed dramatically last few years,
>> and
>>>>>> solving
>>>>>>>>>> file- and network restrictions in a central place through
>>>>>> interception
>>>>>>>>>> rathen than
>>>>>>>>>> at each of the 100+ call sites is a more sustainable way.
>>>>>>>>>> 
>>>>>>>>>> Users are free to add layers outside the JVM as well, such as
>>>>>> systemd,
>>>>>>>>>> container,
>>>>>>>>>> SELinux etc. But be honest - how many small/medium organizations
>>>>>> really do
>>>>>>>>>> this?
>>>>>>>>>> 
>>>>>>>>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
>>>>>>>>>>>> significantly more complex; adds ZK client dependency to the
>> agent
>>>>>>>>>> JAR.
>>>>>>>>>>>> Superseded by port-based wildcards for intra-cluster traffic.
>>>>>>>>>>>> - *Building a Java agent from scratch* — higher effort with no
>>>>>>>>>>>> functional advantage over adapting the Apache 2.0-licensed
>>>>>> OpenSearch
>>>>>>>>>>>> implementation.
>>>>>>>>>>>> 
>>>>>>>>>>>> I agree with not burdening this project with building &
>>>>>> maintainining
>>>>>>>>>> such
>>>>>>>>>>> a mechanism.
>>>>>>>>>>> 
>>>>>>>>>>> I'm not sure, to what degree, we can leverage that existing
>> agent
>>>>>> you
>>>>>>>>>> speak
>>>>>>>>>>> of without further burdening us.  It's a burden/reward
>> trade-off.
>>>>>>>>>> 
>>>>>>>>>> If the agent becomes a true maintenance burden (i.e. a larger
>> burden
>>>>>> than
>>>>>>>>>> handling the CVEs it would prevent, then a valid action is to
>> remove
>>>>>> the
>>>>>>>>>> agent again.
>>>>>>>>>> Like any other feature we develop and maintain. Good this is
>> that
>>>>>> this is
>>>>>>>>>> pure Java,
>>>>>>>>>> and production-ready stable code.
>>>>>>>>>> 
>>>>>>>>>>> In your updated proposal, you point
>>>>>>>>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That
>> is
>>>>>> called
>>>>>>>>>> by
>>>>>>>>>>> a number of places... although I *think* the original
>> intention was
>>>>>> only
>>>>>>>>>> to
>>>>>>>>>>> limit where cores are created?  Can you elaborate on what the
>> role
>>>>>> of
>>>>>>>>>> that
>>>>>>>>>>> method *should* be and how the JSM might also or work?
>>>>>>>>>> 
>>>>>>>>>> The pathAllowed checks (also the ones pre-dating the
>> centralization
>>>>>> in
>>>>>>>>>> CoreContainer,
>>>>>>>>>> were introduced in 8.x, before JSM was introduced or enabled by
>>>>>> default.
>>>>>>>>>> Initially
>>>>>>>>>> assertPathAllowed would validate end-user API input to avoid
>> cores
>>>>>> being
>>>>>>>>>> created
>>>>>>>>>> or loaded outside blessed folders. It has then been re-used for
>>>>>> other code
>>>>>>>>>> locations
>>>>>>>>>> that may do file access based on user-input in API or config.
>> It as
>>>>>> also
>>>>>>>>>> extended to
>>>>>>>>>> block UNC paths after many attacks with this as a vector.
>>>>>>>>>> 
>>>>>>>>>> I believe perhaps pathAllowed would not have been written had
>> JSM
>>>>>> already
>>>>>>>>>> been
>>>>>>>>>> enforced. Although I don't know if you can block UNC with JSM?
>>>>>>>>>> The SIP does not propose to remove pathAllowed now. One benefit
>> of
>>>>>> early
>>>>>>>>>> detection
>>>>>>>>>> is that we can give better context-sensitive error- and log
>> messages
>>>>>>>>>> rather than throwing
>>>>>>>>>> an exception.
>>>>>>>>>> 
>>>>>>>>>> The agent approach laid out sandboxes four attack vectors
>>>>>>>>>> * File access outside a limited set of folders and full block of
>>>>>> Windows
>>>>>>>>>> UNC
>>>>>>>>>> * Network access other than peer solr nodes and zk nodes
>>>>>>>>>> * Disabllow calling System.exit() (mainly useful for 3rd party
>>>>>> plugins?)
>>>>>>>>>> * Disallow spawning processes
>>>>>>>>>> 
>>>>>>>>>> A nice thing with the agent approach is that it is unintrusive
>> and
>>>>>> easy to
>>>>>>>>>> disable,
>>>>>>>>>> so users who want to take care of all this on OS level may
>> disable
>>>>>> it, and
>>>>>>>>>> if we or
>>>>>>>>>> users don't find value in it down the road they can disable it
>> or we
>>>>>> can
>>>>>>>>>> remove it.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Jan
>>>>>>>>>> 
>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>>>>>>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> http://www.needhamsoftware.com (work)
>>>>>>>>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>>>> 
>>>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[David Smiley <ds...@apache.org>]

Presumably the OpenSearch project would also appreciate reduced maintenance
if this aspect of their project were extracted to an independent project.
It's worth chatting with them about this, yeah?  It's also maybe exciting
for those involved in that mechanism to get more exposure -- more projects
using their cool stuff.

On Wed, Jun 10, 2026 at 4:01 AM Jason Gerlowski <ge...@gmail.com>
wrote:

> To clarify - I'm not assuming any particular answer to that question.
> If folks think the module won't be high-maintenance, then +1.  I just
> want to make sure the question gets asked is all.
>
> (And if we're not sure about the maintenance implications, IMO it'd be
> a great option to move forward with SIP-24 and aim to re-evaluate in a
> year or two when we do have a better sense.  I see SIP-24 as a clear
> improvement, so we shouldn't let "perfect be the enemy of good" here.)
>
> On Wed, Jun 10, 2026 at 9:54 AM Jason Gerlowski <ge...@gmail.com>
> wrote:
> >
> > > Or are you hinting that we'll define Solr's security model in such a
> way that
> > > we'll no longer treat arbitrary filesystem read/write or arbitrary
> network access
> > > as a vulnerability?
> >
> > Kindof, yeah, I think we should at least consider it.  (Obviously
> > communication-wise I wouldn't phrase it in terms of what reports we'd
> > reject going forward.  I'd frame it in terms of shifting where the
> > protection is coming from.  e.g. "Rather than relying on our security
> > code, rely instead on proven security tooling like: <list of
> > options>")
> >
> > I love everything about the ByteBuddy / security-agent approach, in a
> > technical sense.  It's a huge improvement over our current setup.  But
> > it does come with a maintenance cost and it's worth weighing that for
> > ourselves.  OpenSearch decided the tradeoff was worth it.  But they
> > have more corporate backing and likely more community bandwidth than
> > we do, so we might weigh the same concerns and reach a different
> > conclusion.
> >
> > If we're fully confident in the protections offered by security-agent,
> > and we're OK spending at least some bandwidth on responding to future
> > security-agent bugs, vuln reports, code maintenance, etc....great!  If
> > not, then maybe a more off-the-shelf approach to securing this stuff
> > is a better fit for us.
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Jun 9, 2026 at 7:51 AM Jan Høydahl <ja...@cominvent.com>
> wrote:
> > >
> > > Could have tried to use stock OpenSearch agent, but they do things a
> bit differently in the bootstrap and we may end up asking them to do
> changes to be more generic and accommodate us.
> > >
> > > Another option is to start a new Apache commons sub project,
> commons-sec-agent pull that as a dependency, and promote it for others to
> use and help maintain.
> > >
> > > Jan Høydahl
> > >
> > > > 7. juni 2026 kl. 18:15 skrev David Smiley <ds...@apache.org>:
> > > >
> > > > Well written Jason!  I agree 100% and I didn't communicate this
> well enough
> > > > previously when I used fewer words to essentially communicate the
> same.
> > > >
> > > > I see the agent coming in SIP-24 is a forked copy, and thus we must
> > > > maintain it.  That's unfortunate.  I hoped we could instead depend on
> > > > another project regularly maintaining and producing artifacts for
> 3rd party
> > > > (us) consumption.
> > > >
> > > > Perhaps the agent will allow us to remove from Solr the
> AllowUrlChecker
> > > > (whatever it's called) and similar for file paths.  I hope we get
> some
> > > > maintenance "relief".
> > > >
> > > >> On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <
> gerlowskija@gmail.com>
> > > >> wrote:
> > > >>
> > > >> Apologies for chiming in so late on this.
> > > >>
> > > >> The Java agent route seems like a huge improvement over our existing
> > > >> approach.  I love that it's a global protection - it doesn't rely on
> > > >> devs remembering to call the required "check the path" method in
> every
> > > >> single place we might read a file in our code.
> > > >>
> > > >> But ultimately I still think systemd/seccomp is a much more
> > > >> sustainable route for us than the Java-agent approach
> > > >>
> > > >> We're a Search community that's here to solve people's Search
> > > >> problems.  I won't minimize for a second all the effort we've put
> into
> > > >> security over the years. But there are entire projects that exist to
> > > >> solve these sorts of problems.  IMO steering users towards something
> > > >> like systemd that has a whole community of security folks behind is
> > > >> going to leave them more secure, and be more sustainable for us.
> > > >>
> > > >> (To be clear I'm not advocating for systemd specifically, but for
> the
> > > >> general idea of relying on well-known, OS-level protections that
> exist
> > > >> outside of the JVM and Solr.)
> > > >>
> > > >> I've heard two objections to the "systemd" approach so far: that
> > > >> systemd isn't cross-platform, and that some folks won't bother to
> > > >> enable it.
> > > >>
> > > >> To the first objection, I'd say that our responsibility has never
> been
> > > >> to find one security solution that works for everyone.  We never
> > > >> mandated that folks use a particular firewall: we let users choose
> how
> > > >> they wanted to provide that network isolation.  IMO this could be
> > > >> handled the same way: point folks to a few different tools,
> optionally
> > > >> provide a few example configs, and let them pick based on their
> > > >> OS/platform.
> > > >>
> > > >> To the second objection: this is the whole point of having a
> security
> > > >> model.  It's expected and normal for projects to describe how to
> make
> > > >> a deployment of their software "secure".  It's neither fair nor
> > > >> possible to secure folks who won't read our "Going to Production"
> docs
> > > >> before deploying.
> > > >>
> > > >>> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com>
> wrote:
> > > >>>
> > > >>> So the implementation of SIP-24 is now ready for review at
> > > >>>
> > > >>> https://github.com/apache/solr/pull/4471
> > > >>>
> > > >>> The PR description gives through description of the change and
> what to
> > > >> look for.
> > > >>> I have run several rounds of Copilot reviews and Claude reviews.
> > > >>> We have unit tests and BATS tests, so coverage should be fairly
> good.
> > > >>>
> > > >>> The PR is fairly large +5454 lines across 62 files and 54 commits,
> but
> > > >> mostly
> > > >>> contained in the new gradle module, plus one hook in CoreContainer.
> > > >>>
> > > >>> Take it for a spin and let me know how it works.
> > > >>>
> > > >>> Jan
> > > >>>
> > > >>>> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> > > >>>>
> > > >>>> Hi,
> > > >>>>
> > > >>>> Welcoming more feedback on this approach.
> > > >>>>
> > > >>>> I'm planning to move to implementation phase on Wednesday, to get
> a
> > > >> first
> > > >>>> version of the agent ready for testing. But it would be helpful
> with
> > > >> as much
> > > >>>> feedback on the SIP high level design before I start implementing.
> > > >> Thanks David
> > > >>>> for your intial feedback.
> > > >>>>
> > > >>>> Jan
> > > >>>>
> > > >>>>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> > > >>>>>
> > > >>>>> I know it's probably unrealistic because corporate environments
> into
> > > >> which
> > > >>>>> solr is deployed likely would have difficulty with it, but there
> is a
> > > >> JDK
> > > >>>>> fork that keeps and improves the Security Manager... It's
> supported
> > > >> by the
> > > >>>>> descendants of the Apache River project.
> > > >>>>> https://github.com/pfirmstone/DirtyChai
> > > >>>>>
> > > >>>>> Probably not useful, but perhaps interesting. Certainly we are
> not
> > > >> the only
> > > >>>>> ones irritated by the loss of the security manager.
> > > >>>>>
> > > >>>>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <
> jan.asf@cominvent.com>
> > > >> wrote:
> > > >>>>>
> > > >>>>>>> I should clarify something.  My objections mostly relate to the
> > > >> insistent
> > > >>>>>>> language in the SIP requiring Solr to have a substitute for the
> > > >> JSM.  I'm
> > > >>>>>>> not quite against doing something but I might vote -0 on
> something
> > > >> that
> > > >>>>>>> seems to have a poor security payoff relative to the
> maintenance
> > > >> burden.
> > > >>>>>>> The more off-the-shelf, the better IMO.
> > > >>>>>>> Thank you for investing your time researching some options.
> > > >>>>>>
> > > >>>>>> The security landscape has dramatically changed during the years
> > > >> that we
> > > >>>>>> have enjoyed JSM protection for Solr. It's a crazy world out
> there
> > > >> and
> > > >>>>>> every
> > > >>>>>> single code flaw, existing and future, will be found and
> exploited or
> > > >>>>>> published
> > > >>>>>> by so called security researchers. That's why I believe we need
> a
> > > >>>>>> centralized
> > > >>>>>> solution.
> > > >>>>>>
> > > >>>>>>>> Rejected Alternatives
> > > >>>>>>>>
> > > >>>>>>>> - *Staying on Java < 24* — not viable long-term; Solr must
> support
> > > >>>>>>>> current Java LTS releases.
> > > >>>>>>>> - *Removing JSM protections without any replacement* —
> unacceptable
> > > >>>>>>>> security regression.
> > > >>>>>>>>
> > > >>>>>>> Both Eric Pugh and I have challenged this.
> > > >>>>>>>>
> > > >>>>>>>> - *OS-level hardening only (systemd, seccomp)* — not
> > > >> cross-platform;
> > > >>>>>>>> does not cover Windows or macOS.
> > > >>>>>>>>
> > > >>>>>>> I challenge this. Why should the Solr project burden itself
> with
> > > >> building
> > > >>>>>>> & maintaining security mechanisms already provided by
> off-the-shelf
> > > >>>>>> tools?
> > > >>>>>>> If a user/operator wishes to run on Windows/macOS that may not
> have
> > > >> this
> > > >>>>>>> protection mechanism, it is a risk consideration for that user
> to
> > > >>>>>> consider,
> > > >>>>>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line
> > > >> defense;
> > > >>>>>> it's
> > > >>>>>>> a defense-in-depth strategy.  Put differently, the protections
> here
> > > >> are
> > > >>>>>>> "best effort" but not worthy of a CVE if they were to falter.
> I
> > > >> want to
> > > >>>>>>> get Arnout's opinion on this supposition.
> > > >>>>>>
> > > >>>>>> Java Security Manager is a beast, but its file system read/write
> > > >> controls
> > > >>>>>> have
> > > >>>>>> saved us many a CVE in 9.x when JSM have been on by default.
> > > >>>>>> The SIP primarily focuses on sandboxing Solr for wrt file and
> network
> > > >>>>>> access.
> > > >>>>>> The security landscape has changed dramatically last few years,
> and
> > > >> solving
> > > >>>>>> file- and network restrictions in a central place through
> > > >> interception
> > > >>>>>> rathen than
> > > >>>>>> at each of the 100+ call sites is a more sustainable way.
> > > >>>>>>
> > > >>>>>> Users are free to add layers outside the JVM as well, such as
> > > >> systemd,
> > > >>>>>> container,
> > > >>>>>> SELinux etc. But be honest - how many small/medium organizations
> > > >> really do
> > > >>>>>> this?
> > > >>>>>>
> > > >>>>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
> > > >>>>>>>> significantly more complex; adds ZK client dependency to the
> agent
> > > >>>>>> JAR.
> > > >>>>>>>> Superseded by port-based wildcards for intra-cluster traffic.
> > > >>>>>>>> - *Building a Java agent from scratch* — higher effort with no
> > > >>>>>>>> functional advantage over adapting the Apache 2.0-licensed
> > > >> OpenSearch
> > > >>>>>>>> implementation.
> > > >>>>>>>>
> > > >>>>>>>> I agree with not burdening this project with building &
> > > >> maintainining
> > > >>>>>> such
> > > >>>>>>> a mechanism.
> > > >>>>>>>
> > > >>>>>>> I'm not sure, to what degree, we can leverage that existing
> agent
> > > >> you
> > > >>>>>> speak
> > > >>>>>>> of without further burdening us.  It's a burden/reward
> trade-off.
> > > >>>>>>
> > > >>>>>> If the agent becomes a true maintenance burden (i.e. a larger
> burden
> > > >> than
> > > >>>>>> handling the CVEs it would prevent, then a valid action is to
> remove
> > > >> the
> > > >>>>>> agent again.
> > > >>>>>> Like any other feature we develop and maintain. Good this is
> that
> > > >> this is
> > > >>>>>> pure Java,
> > > >>>>>> and production-ready stable code.
> > > >>>>>>
> > > >>>>>>> In your updated proposal, you point
> > > >>>>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That
> is
> > > >> called
> > > >>>>>> by
> > > >>>>>>> a number of places... although I *think* the original
> intention was
> > > >> only
> > > >>>>>> to
> > > >>>>>>> limit where cores are created?  Can you elaborate on what the
> role
> > > >> of
> > > >>>>>> that
> > > >>>>>>> method *should* be and how the JSM might also or work?
> > > >>>>>>
> > > >>>>>> The pathAllowed checks (also the ones pre-dating the
> centralization
> > > >> in
> > > >>>>>> CoreContainer,
> > > >>>>>> were introduced in 8.x, before JSM was introduced or enabled by
> > > >> default.
> > > >>>>>> Initially
> > > >>>>>> assertPathAllowed would validate end-user API input to avoid
> cores
> > > >> being
> > > >>>>>> created
> > > >>>>>> or loaded outside blessed folders. It has then been re-used for
> > > >> other code
> > > >>>>>> locations
> > > >>>>>> that may do file access based on user-input in API or config.
> It as
> > > >> also
> > > >>>>>> extended to
> > > >>>>>> block UNC paths after many attacks with this as a vector.
> > > >>>>>>
> > > >>>>>> I believe perhaps pathAllowed would not have been written had
> JSM
> > > >> already
> > > >>>>>> been
> > > >>>>>> enforced. Although I don't know if you can block UNC with JSM?
> > > >>>>>> The SIP does not propose to remove pathAllowed now. One benefit
> of
> > > >> early
> > > >>>>>> detection
> > > >>>>>> is that we can give better context-sensitive error- and log
> messages
> > > >>>>>> rather than throwing
> > > >>>>>> an exception.
> > > >>>>>>
> > > >>>>>> The agent approach laid out sandboxes four attack vectors
> > > >>>>>> * File access outside a limited set of folders and full block of
> > > >> Windows
> > > >>>>>> UNC
> > > >>>>>> * Network access other than peer solr nodes and zk nodes
> > > >>>>>> * Disabllow calling System.exit() (mainly useful for 3rd party
> > > >> plugins?)
> > > >>>>>> * Disallow spawning processes
> > > >>>>>>
> > > >>>>>> A nice thing with the agent approach is that it is unintrusive
> and
> > > >> easy to
> > > >>>>>> disable,
> > > >>>>>> so users who want to take care of all this on OS level may
> disable
> > > >> it, and
> > > >>>>>> if we or
> > > >>>>>> users don't find value in it down the road they can disable it
> or we
> > > >> can
> > > >>>>>> remove it.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Jan
> > > >>>>>>
> ---------------------------------------------------------------------
> > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > >>>>>> For additional commands, e-mail: dev-help@solr.apache.org
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> http://www.needhamsoftware.com (work)
> > > >>>>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> > > >>>>
> > > >>>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > >> For additional commands, e-mail: dev-help@solr.apache.org
> > > >>
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > For additional commands, e-mail: dev-help@solr.apache.org
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[David Smiley <ds...@apache.org>]

Presumably the OpenSearch project would also appreciate reduced maintenance
if this aspect of their project were extracted to an independent project.
It's worth chatting with them about this, yeah?  It's also maybe exciting
for those involved in that mechanism to get more exposure -- more projects
using their cool stuff.

On Wed, Jun 10, 2026 at 4:01 AM Jason Gerlowski <ge...@gmail.com>
wrote:

> To clarify - I'm not assuming any particular answer to that question.
> If folks think the module won't be high-maintenance, then +1.  I just
> want to make sure the question gets asked is all.
>
> (And if we're not sure about the maintenance implications, IMO it'd be
> a great option to move forward with SIP-24 and aim to re-evaluate in a
> year or two when we do have a better sense.  I see SIP-24 as a clear
> improvement, so we shouldn't let "perfect be the enemy of good" here.)
>
> On Wed, Jun 10, 2026 at 9:54 AM Jason Gerlowski <ge...@gmail.com>
> wrote:
> >
> > > Or are you hinting that we'll define Solr's security model in such a
> way that
> > > we'll no longer treat arbitrary filesystem read/write or arbitrary
> network access
> > > as a vulnerability?
> >
> > Kindof, yeah, I think we should at least consider it.  (Obviously
> > communication-wise I wouldn't phrase it in terms of what reports we'd
> > reject going forward.  I'd frame it in terms of shifting where the
> > protection is coming from.  e.g. "Rather than relying on our security
> > code, rely instead on proven security tooling like: <list of
> > options>")
> >
> > I love everything about the ByteBuddy / security-agent approach, in a
> > technical sense.  It's a huge improvement over our current setup.  But
> > it does come with a maintenance cost and it's worth weighing that for
> > ourselves.  OpenSearch decided the tradeoff was worth it.  But they
> > have more corporate backing and likely more community bandwidth than
> > we do, so we might weigh the same concerns and reach a different
> > conclusion.
> >
> > If we're fully confident in the protections offered by security-agent,
> > and we're OK spending at least some bandwidth on responding to future
> > security-agent bugs, vuln reports, code maintenance, etc....great!  If
> > not, then maybe a more off-the-shelf approach to securing this stuff
> > is a better fit for us.
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Jun 9, 2026 at 7:51 AM Jan Høydahl <ja...@cominvent.com>
> wrote:
> > >
> > > Could have tried to use stock OpenSearch agent, but they do things a
> bit differently in the bootstrap and we may end up asking them to do
> changes to be more generic and accommodate us.
> > >
> > > Another option is to start a new Apache commons sub project,
> commons-sec-agent pull that as a dependency, and promote it for others to
> use and help maintain.
> > >
> > > Jan Høydahl
> > >
> > > > 7. juni 2026 kl. 18:15 skrev David Smiley <ds...@apache.org>:
> > > >
> > > > Well written Jason!  I agree 100% and I didn't communicate this
> well enough
> > > > previously when I used fewer words to essentially communicate the
> same.
> > > >
> > > > I see the agent coming in SIP-24 is a forked copy, and thus we must
> > > > maintain it.  That's unfortunate.  I hoped we could instead depend on
> > > > another project regularly maintaining and producing artifacts for
> 3rd party
> > > > (us) consumption.
> > > >
> > > > Perhaps the agent will allow us to remove from Solr the
> AllowUrlChecker
> > > > (whatever it's called) and similar for file paths.  I hope we get
> some
> > > > maintenance "relief".
> > > >
> > > >> On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <
> gerlowskija@gmail.com>
> > > >> wrote:
> > > >>
> > > >> Apologies for chiming in so late on this.
> > > >>
> > > >> The Java agent route seems like a huge improvement over our existing
> > > >> approach.  I love that it's a global protection - it doesn't rely on
> > > >> devs remembering to call the required "check the path" method in
> every
> > > >> single place we might read a file in our code.
> > > >>
> > > >> But ultimately I still think systemd/seccomp is a much more
> > > >> sustainable route for us than the Java-agent approach
> > > >>
> > > >> We're a Search community that's here to solve people's Search
> > > >> problems.  I won't minimize for a second all the effort we've put
> into
> > > >> security over the years. But there are entire projects that exist to
> > > >> solve these sorts of problems.  IMO steering users towards something
> > > >> like systemd that has a whole community of security folks behind is
> > > >> going to leave them more secure, and be more sustainable for us.
> > > >>
> > > >> (To be clear I'm not advocating for systemd specifically, but for
> the
> > > >> general idea of relying on well-known, OS-level protections that
> exist
> > > >> outside of the JVM and Solr.)
> > > >>
> > > >> I've heard two objections to the "systemd" approach so far: that
> > > >> systemd isn't cross-platform, and that some folks won't bother to
> > > >> enable it.
> > > >>
> > > >> To the first objection, I'd say that our responsibility has never
> been
> > > >> to find one security solution that works for everyone.  We never
> > > >> mandated that folks use a particular firewall: we let users choose
> how
> > > >> they wanted to provide that network isolation.  IMO this could be
> > > >> handled the same way: point folks to a few different tools,
> optionally
> > > >> provide a few example configs, and let them pick based on their
> > > >> OS/platform.
> > > >>
> > > >> To the second objection: this is the whole point of having a
> security
> > > >> model.  It's expected and normal for projects to describe how to
> make
> > > >> a deployment of their software "secure".  It's neither fair nor
> > > >> possible to secure folks who won't read our "Going to Production"
> docs
> > > >> before deploying.
> > > >>
> > > >>> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com>
> wrote:
> > > >>>
> > > >>> So the implementation of SIP-24 is now ready for review at
> > > >>>
> > > >>> https://github.com/apache/solr/pull/4471
> > > >>>
> > > >>> The PR description gives through description of the change and
> what to
> > > >> look for.
> > > >>> I have run several rounds of Copilot reviews and Claude reviews.
> > > >>> We have unit tests and BATS tests, so coverage should be fairly
> good.
> > > >>>
> > > >>> The PR is fairly large +5454 lines across 62 files and 54 commits,
> but
> > > >> mostly
> > > >>> contained in the new gradle module, plus one hook in CoreContainer.
> > > >>>
> > > >>> Take it for a spin and let me know how it works.
> > > >>>
> > > >>> Jan
> > > >>>
> > > >>>> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> > > >>>>
> > > >>>> Hi,
> > > >>>>
> > > >>>> Welcoming more feedback on this approach.
> > > >>>>
> > > >>>> I'm planning to move to implementation phase on Wednesday, to get
> a
> > > >> first
> > > >>>> version of the agent ready for testing. But it would be helpful
> with
> > > >> as much
> > > >>>> feedback on the SIP high level design before I start implementing.
> > > >> Thanks David
> > > >>>> for your intial feedback.
> > > >>>>
> > > >>>> Jan
> > > >>>>
> > > >>>>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> > > >>>>>
> > > >>>>> I know it's probably unrealistic because corporate environments
> into
> > > >> which
> > > >>>>> solr is deployed likely would have difficulty with it, but there
> is a
> > > >> JDK
> > > >>>>> fork that keeps and improves the Security Manager... It's
> supported
> > > >> by the
> > > >>>>> descendants of the Apache River project.
> > > >>>>> https://github.com/pfirmstone/DirtyChai
> > > >>>>>
> > > >>>>> Probably not useful, but perhaps interesting. Certainly we are
> not
> > > >> the only
> > > >>>>> ones irritated by the loss of the security manager.
> > > >>>>>
> > > >>>>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <
> jan.asf@cominvent.com>
> > > >> wrote:
> > > >>>>>
> > > >>>>>>> I should clarify something.  My objections mostly relate to the
> > > >> insistent
> > > >>>>>>> language in the SIP requiring Solr to have a substitute for the
> > > >> JSM.  I'm
> > > >>>>>>> not quite against doing something but I might vote -0 on
> something
> > > >> that
> > > >>>>>>> seems to have a poor security payoff relative to the
> maintenance
> > > >> burden.
> > > >>>>>>> The more off-the-shelf, the better IMO.
> > > >>>>>>> Thank you for investing your time researching some options.
> > > >>>>>>
> > > >>>>>> The security landscape has dramatically changed during the years
> > > >> that we
> > > >>>>>> have enjoyed JSM protection for Solr. It's a crazy world out
> there
> > > >> and
> > > >>>>>> every
> > > >>>>>> single code flaw, existing and future, will be found and
> exploited or
> > > >>>>>> published
> > > >>>>>> by so called security researchers. That's why I believe we need
> a
> > > >>>>>> centralized
> > > >>>>>> solution.
> > > >>>>>>
> > > >>>>>>>> Rejected Alternatives
> > > >>>>>>>>
> > > >>>>>>>> - *Staying on Java < 24* — not viable long-term; Solr must
> support
> > > >>>>>>>> current Java LTS releases.
> > > >>>>>>>> - *Removing JSM protections without any replacement* —
> unacceptable
> > > >>>>>>>> security regression.
> > > >>>>>>>>
> > > >>>>>>> Both Eric Pugh and I have challenged this.
> > > >>>>>>>>
> > > >>>>>>>> - *OS-level hardening only (systemd, seccomp)* — not
> > > >> cross-platform;
> > > >>>>>>>> does not cover Windows or macOS.
> > > >>>>>>>>
> > > >>>>>>> I challenge this. Why should the Solr project burden itself
> with
> > > >> building
> > > >>>>>>> & maintaining security mechanisms already provided by
> off-the-shelf
> > > >>>>>> tools?
> > > >>>>>>> If a user/operator wishes to run on Windows/macOS that may not
> have
> > > >> this
> > > >>>>>>> protection mechanism, it is a risk consideration for that user
> to
> > > >>>>>> consider,
> > > >>>>>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line
> > > >> defense;
> > > >>>>>> it's
> > > >>>>>>> a defense-in-depth strategy.  Put differently, the protections
> here
> > > >> are
> > > >>>>>>> "best effort" but not worthy of a CVE if they were to falter.
> I
> > > >> want to
> > > >>>>>>> get Arnout's opinion on this supposition.
> > > >>>>>>
> > > >>>>>> Java Security Manager is a beast, but its file system read/write
> > > >> controls
> > > >>>>>> have
> > > >>>>>> saved us many a CVE in 9.x when JSM have been on by default.
> > > >>>>>> The SIP primarily focuses on sandboxing Solr for wrt file and
> network
> > > >>>>>> access.
> > > >>>>>> The security landscape has changed dramatically last few years,
> and
> > > >> solving
> > > >>>>>> file- and network restrictions in a central place through
> > > >> interception
> > > >>>>>> rathen than
> > > >>>>>> at each of the 100+ call sites is a more sustainable way.
> > > >>>>>>
> > > >>>>>> Users are free to add layers outside the JVM as well, such as
> > > >> systemd,
> > > >>>>>> container,
> > > >>>>>> SELinux etc. But be honest - how many small/medium organizations
> > > >> really do
> > > >>>>>> this?
> > > >>>>>>
> > > >>>>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
> > > >>>>>>>> significantly more complex; adds ZK client dependency to the
> agent
> > > >>>>>> JAR.
> > > >>>>>>>> Superseded by port-based wildcards for intra-cluster traffic.
> > > >>>>>>>> - *Building a Java agent from scratch* — higher effort with no
> > > >>>>>>>> functional advantage over adapting the Apache 2.0-licensed
> > > >> OpenSearch
> > > >>>>>>>> implementation.
> > > >>>>>>>>
> > > >>>>>>>> I agree with not burdening this project with building &
> > > >> maintainining
> > > >>>>>> such
> > > >>>>>>> a mechanism.
> > > >>>>>>>
> > > >>>>>>> I'm not sure, to what degree, we can leverage that existing
> agent
> > > >> you
> > > >>>>>> speak
> > > >>>>>>> of without further burdening us.  It's a burden/reward
> trade-off.
> > > >>>>>>
> > > >>>>>> If the agent becomes a true maintenance burden (i.e. a larger
> burden
> > > >> than
> > > >>>>>> handling the CVEs it would prevent, then a valid action is to
> remove
> > > >> the
> > > >>>>>> agent again.
> > > >>>>>> Like any other feature we develop and maintain. Good this is
> that
> > > >> this is
> > > >>>>>> pure Java,
> > > >>>>>> and production-ready stable code.
> > > >>>>>>
> > > >>>>>>> In your updated proposal, you point
> > > >>>>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That
> is
> > > >> called
> > > >>>>>> by
> > > >>>>>>> a number of places... although I *think* the original
> intention was
> > > >> only
> > > >>>>>> to
> > > >>>>>>> limit where cores are created?  Can you elaborate on what the
> role
> > > >> of
> > > >>>>>> that
> > > >>>>>>> method *should* be and how the JSM might also or work?
> > > >>>>>>
> > > >>>>>> The pathAllowed checks (also the ones pre-dating the
> centralization
> > > >> in
> > > >>>>>> CoreContainer,
> > > >>>>>> were introduced in 8.x, before JSM was introduced or enabled by
> > > >> default.
> > > >>>>>> Initially
> > > >>>>>> assertPathAllowed would validate end-user API input to avoid
> cores
> > > >> being
> > > >>>>>> created
> > > >>>>>> or loaded outside blessed folders. It has then been re-used for
> > > >> other code
> > > >>>>>> locations
> > > >>>>>> that may do file access based on user-input in API or config.
> It as
> > > >> also
> > > >>>>>> extended to
> > > >>>>>> block UNC paths after many attacks with this as a vector.
> > > >>>>>>
> > > >>>>>> I believe perhaps pathAllowed would not have been written had
> JSM
> > > >> already
> > > >>>>>> been
> > > >>>>>> enforced. Although I don't know if you can block UNC with JSM?
> > > >>>>>> The SIP does not propose to remove pathAllowed now. One benefit
> of
> > > >> early
> > > >>>>>> detection
> > > >>>>>> is that we can give better context-sensitive error- and log
> messages
> > > >>>>>> rather than throwing
> > > >>>>>> an exception.
> > > >>>>>>
> > > >>>>>> The agent approach laid out sandboxes four attack vectors
> > > >>>>>> * File access outside a limited set of folders and full block of
> > > >> Windows
> > > >>>>>> UNC
> > > >>>>>> * Network access other than peer solr nodes and zk nodes
> > > >>>>>> * Disabllow calling System.exit() (mainly useful for 3rd party
> > > >> plugins?)
> > > >>>>>> * Disallow spawning processes
> > > >>>>>>
> > > >>>>>> A nice thing with the agent approach is that it is unintrusive
> and
> > > >> easy to
> > > >>>>>> disable,
> > > >>>>>> so users who want to take care of all this on OS level may
> disable
> > > >> it, and
> > > >>>>>> if we or
> > > >>>>>> users don't find value in it down the road they can disable it
> or we
> > > >> can
> > > >>>>>> remove it.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Jan
> > > >>>>>>
> ---------------------------------------------------------------------
> > > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > >>>>>> For additional commands, e-mail: dev-help@solr.apache.org
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> http://www.needhamsoftware.com (work)
> > > >>>>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> > > >>>>
> > > >>>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > >> For additional commands, e-mail: dev-help@solr.apache.org
> > > >>
> > > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > > For additional commands, e-mail: dev-help@solr.apache.org
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jason Gerlowski <ge...@gmail.com>]

To clarify - I'm not assuming any particular answer to that question.
If folks think the module won't be high-maintenance, then +1.  I just
want to make sure the question gets asked is all.

(And if we're not sure about the maintenance implications, IMO it'd be
a great option to move forward with SIP-24 and aim to re-evaluate in a
year or two when we do have a better sense.  I see SIP-24 as a clear
improvement, so we shouldn't let "perfect be the enemy of good" here.)

On Wed, Jun 10, 2026 at 9:54 AM Jason Gerlowski <ge...@gmail.com> wrote:
>
> > Or are you hinting that we'll define Solr's security model in such a way that
> > we'll no longer treat arbitrary filesystem read/write or arbitrary network access
> > as a vulnerability?
>
> Kindof, yeah, I think we should at least consider it.  (Obviously
> communication-wise I wouldn't phrase it in terms of what reports we'd
> reject going forward.  I'd frame it in terms of shifting where the
> protection is coming from.  e.g. "Rather than relying on our security
> code, rely instead on proven security tooling like: <list of
> options>")
>
> I love everything about the ByteBuddy / security-agent approach, in a
> technical sense.  It's a huge improvement over our current setup.  But
> it does come with a maintenance cost and it's worth weighing that for
> ourselves.  OpenSearch decided the tradeoff was worth it.  But they
> have more corporate backing and likely more community bandwidth than
> we do, so we might weigh the same concerns and reach a different
> conclusion.
>
> If we're fully confident in the protections offered by security-agent,
> and we're OK spending at least some bandwidth on responding to future
> security-agent bugs, vuln reports, code maintenance, etc....great!  If
> not, then maybe a more off-the-shelf approach to securing this stuff
> is a better fit for us.
>
> Best,
>
> Jason
>
> On Tue, Jun 9, 2026 at 7:51 AM Jan Høydahl <ja...@cominvent.com> wrote:
> >
> > Could have tried to use stock OpenSearch agent, but they do things a bit differently in the bootstrap and we may end up asking them to do changes to be more generic and accommodate us.
> >
> > Another option is to start a new Apache commons sub project, commons-sec-agent pull that as a dependency, and promote it for others to use and help maintain.
> >
> > Jan Høydahl
> >
> > > 7. juni 2026 kl. 18:15 skrev David Smiley <ds...@apache.org>:
> > >
> > > Well written Jason!  I agree 100% and I didn't communicate this well enough
> > > previously when I used fewer words to essentially communicate the same.
> > >
> > > I see the agent coming in SIP-24 is a forked copy, and thus we must
> > > maintain it.  That's unfortunate.  I hoped we could instead depend on
> > > another project regularly maintaining and producing artifacts for 3rd party
> > > (us) consumption.
> > >
> > > Perhaps the agent will allow us to remove from Solr the AllowUrlChecker
> > > (whatever it's called) and similar for file paths.  I hope we get some
> > > maintenance "relief".
> > >
> > >> On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <ge...@gmail.com>
> > >> wrote:
> > >>
> > >> Apologies for chiming in so late on this.
> > >>
> > >> The Java agent route seems like a huge improvement over our existing
> > >> approach.  I love that it's a global protection - it doesn't rely on
> > >> devs remembering to call the required "check the path" method in every
> > >> single place we might read a file in our code.
> > >>
> > >> But ultimately I still think systemd/seccomp is a much more
> > >> sustainable route for us than the Java-agent approach
> > >>
> > >> We're a Search community that's here to solve people's Search
> > >> problems.  I won't minimize for a second all the effort we've put into
> > >> security over the years. But there are entire projects that exist to
> > >> solve these sorts of problems.  IMO steering users towards something
> > >> like systemd that has a whole community of security folks behind is
> > >> going to leave them more secure, and be more sustainable for us.
> > >>
> > >> (To be clear I'm not advocating for systemd specifically, but for the
> > >> general idea of relying on well-known, OS-level protections that exist
> > >> outside of the JVM and Solr.)
> > >>
> > >> I've heard two objections to the "systemd" approach so far: that
> > >> systemd isn't cross-platform, and that some folks won't bother to
> > >> enable it.
> > >>
> > >> To the first objection, I'd say that our responsibility has never been
> > >> to find one security solution that works for everyone.  We never
> > >> mandated that folks use a particular firewall: we let users choose how
> > >> they wanted to provide that network isolation.  IMO this could be
> > >> handled the same way: point folks to a few different tools, optionally
> > >> provide a few example configs, and let them pick based on their
> > >> OS/platform.
> > >>
> > >> To the second objection: this is the whole point of having a security
> > >> model.  It's expected and normal for projects to describe how to make
> > >> a deployment of their software "secure".  It's neither fair nor
> > >> possible to secure folks who won't read our "Going to Production" docs
> > >> before deploying.
> > >>
> > >>> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com> wrote:
> > >>>
> > >>> So the implementation of SIP-24 is now ready for review at
> > >>>
> > >>> https://github.com/apache/solr/pull/4471
> > >>>
> > >>> The PR description gives through description of the change and what to
> > >> look for.
> > >>> I have run several rounds of Copilot reviews and Claude reviews.
> > >>> We have unit tests and BATS tests, so coverage should be fairly good.
> > >>>
> > >>> The PR is fairly large +5454 lines across 62 files and 54 commits, but
> > >> mostly
> > >>> contained in the new gradle module, plus one hook in CoreContainer.
> > >>>
> > >>> Take it for a spin and let me know how it works.
> > >>>
> > >>> Jan
> > >>>
> > >>>> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> > >>>>
> > >>>> Hi,
> > >>>>
> > >>>> Welcoming more feedback on this approach.
> > >>>>
> > >>>> I'm planning to move to implementation phase on Wednesday, to get a
> > >> first
> > >>>> version of the agent ready for testing. But it would be helpful with
> > >> as much
> > >>>> feedback on the SIP high level design before I start implementing.
> > >> Thanks David
> > >>>> for your intial feedback.
> > >>>>
> > >>>> Jan
> > >>>>
> > >>>>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> > >>>>>
> > >>>>> I know it's probably unrealistic because corporate environments into
> > >> which
> > >>>>> solr is deployed likely would have difficulty with it, but there is a
> > >> JDK
> > >>>>> fork that keeps and improves the Security Manager... It's supported
> > >> by the
> > >>>>> descendants of the Apache River project.
> > >>>>> https://github.com/pfirmstone/DirtyChai
> > >>>>>
> > >>>>> Probably not useful, but perhaps interesting. Certainly we are not
> > >> the only
> > >>>>> ones irritated by the loss of the security manager.
> > >>>>>
> > >>>>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com>
> > >> wrote:
> > >>>>>
> > >>>>>>> I should clarify something.  My objections mostly relate to the
> > >> insistent
> > >>>>>>> language in the SIP requiring Solr to have a substitute for the
> > >> JSM.  I'm
> > >>>>>>> not quite against doing something but I might vote -0 on something
> > >> that
> > >>>>>>> seems to have a poor security payoff relative to the maintenance
> > >> burden.
> > >>>>>>> The more off-the-shelf, the better IMO.
> > >>>>>>> Thank you for investing your time researching some options.
> > >>>>>>
> > >>>>>> The security landscape has dramatically changed during the years
> > >> that we
> > >>>>>> have enjoyed JSM protection for Solr. It's a crazy world out there
> > >> and
> > >>>>>> every
> > >>>>>> single code flaw, existing and future, will be found and exploited or
> > >>>>>> published
> > >>>>>> by so called security researchers. That's why I believe we need a
> > >>>>>> centralized
> > >>>>>> solution.
> > >>>>>>
> > >>>>>>>> Rejected Alternatives
> > >>>>>>>>
> > >>>>>>>> - *Staying on Java < 24* — not viable long-term; Solr must support
> > >>>>>>>> current Java LTS releases.
> > >>>>>>>> - *Removing JSM protections without any replacement* — unacceptable
> > >>>>>>>> security regression.
> > >>>>>>>>
> > >>>>>>> Both Eric Pugh and I have challenged this.
> > >>>>>>>>
> > >>>>>>>> - *OS-level hardening only (systemd, seccomp)* — not
> > >> cross-platform;
> > >>>>>>>> does not cover Windows or macOS.
> > >>>>>>>>
> > >>>>>>> I challenge this. Why should the Solr project burden itself with
> > >> building
> > >>>>>>> & maintaining security mechanisms already provided by off-the-shelf
> > >>>>>> tools?
> > >>>>>>> If a user/operator wishes to run on Windows/macOS that may not have
> > >> this
> > >>>>>>> protection mechanism, it is a risk consideration for that user to
> > >>>>>> consider,
> > >>>>>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line
> > >> defense;
> > >>>>>> it's
> > >>>>>>> a defense-in-depth strategy.  Put differently, the protections here
> > >> are
> > >>>>>>> "best effort" but not worthy of a CVE if they were to falter.  I
> > >> want to
> > >>>>>>> get Arnout's opinion on this supposition.
> > >>>>>>
> > >>>>>> Java Security Manager is a beast, but its file system read/write
> > >> controls
> > >>>>>> have
> > >>>>>> saved us many a CVE in 9.x when JSM have been on by default.
> > >>>>>> The SIP primarily focuses on sandboxing Solr for wrt file and network
> > >>>>>> access.
> > >>>>>> The security landscape has changed dramatically last few years, and
> > >> solving
> > >>>>>> file- and network restrictions in a central place through
> > >> interception
> > >>>>>> rathen than
> > >>>>>> at each of the 100+ call sites is a more sustainable way.
> > >>>>>>
> > >>>>>> Users are free to add layers outside the JVM as well, such as
> > >> systemd,
> > >>>>>> container,
> > >>>>>> SELinux etc. But be honest - how many small/medium organizations
> > >> really do
> > >>>>>> this?
> > >>>>>>
> > >>>>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
> > >>>>>>>> significantly more complex; adds ZK client dependency to the agent
> > >>>>>> JAR.
> > >>>>>>>> Superseded by port-based wildcards for intra-cluster traffic.
> > >>>>>>>> - *Building a Java agent from scratch* — higher effort with no
> > >>>>>>>> functional advantage over adapting the Apache 2.0-licensed
> > >> OpenSearch
> > >>>>>>>> implementation.
> > >>>>>>>>
> > >>>>>>>> I agree with not burdening this project with building &
> > >> maintainining
> > >>>>>> such
> > >>>>>>> a mechanism.
> > >>>>>>>
> > >>>>>>> I'm not sure, to what degree, we can leverage that existing agent
> > >> you
> > >>>>>> speak
> > >>>>>>> of without further burdening us.  It's a burden/reward trade-off.
> > >>>>>>
> > >>>>>> If the agent becomes a true maintenance burden (i.e. a larger burden
> > >> than
> > >>>>>> handling the CVEs it would prevent, then a valid action is to remove
> > >> the
> > >>>>>> agent again.
> > >>>>>> Like any other feature we develop and maintain. Good this is that
> > >> this is
> > >>>>>> pure Java,
> > >>>>>> and production-ready stable code.
> > >>>>>>
> > >>>>>>> In your updated proposal, you point
> > >>>>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is
> > >> called
> > >>>>>> by
> > >>>>>>> a number of places... although I *think* the original intention was
> > >> only
> > >>>>>> to
> > >>>>>>> limit where cores are created?  Can you elaborate on what the role
> > >> of
> > >>>>>> that
> > >>>>>>> method *should* be and how the JSM might also or work?
> > >>>>>>
> > >>>>>> The pathAllowed checks (also the ones pre-dating the centralization
> > >> in
> > >>>>>> CoreContainer,
> > >>>>>> were introduced in 8.x, before JSM was introduced or enabled by
> > >> default.
> > >>>>>> Initially
> > >>>>>> assertPathAllowed would validate end-user API input to avoid cores
> > >> being
> > >>>>>> created
> > >>>>>> or loaded outside blessed folders. It has then been re-used for
> > >> other code
> > >>>>>> locations
> > >>>>>> that may do file access based on user-input in API or config. It as
> > >> also
> > >>>>>> extended to
> > >>>>>> block UNC paths after many attacks with this as a vector.
> > >>>>>>
> > >>>>>> I believe perhaps pathAllowed would not have been written had JSM
> > >> already
> > >>>>>> been
> > >>>>>> enforced. Although I don't know if you can block UNC with JSM?
> > >>>>>> The SIP does not propose to remove pathAllowed now. One benefit of
> > >> early
> > >>>>>> detection
> > >>>>>> is that we can give better context-sensitive error- and log messages
> > >>>>>> rather than throwing
> > >>>>>> an exception.
> > >>>>>>
> > >>>>>> The agent approach laid out sandboxes four attack vectors
> > >>>>>> * File access outside a limited set of folders and full block of
> > >> Windows
> > >>>>>> UNC
> > >>>>>> * Network access other than peer solr nodes and zk nodes
> > >>>>>> * Disabllow calling System.exit() (mainly useful for 3rd party
> > >> plugins?)
> > >>>>>> * Disallow spawning processes
> > >>>>>>
> > >>>>>> A nice thing with the agent approach is that it is unintrusive and
> > >> easy to
> > >>>>>> disable,
> > >>>>>> so users who want to take care of all this on OS level may disable
> > >> it, and
> > >>>>>> if we or
> > >>>>>> users don't find value in it down the road they can disable it or we
> > >> can
> > >>>>>> remove it.
> > >>>>>>
> > >>>>>>
> > >>>>>> Jan
> > >>>>>> ---------------------------------------------------------------------
> > >>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >>>>>> For additional commands, e-mail: dev-help@solr.apache.org
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>> --
> > >>>>> http://www.needhamsoftware.com (work)
> > >>>>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> > >>>>
> > >>>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >> For additional commands, e-mail: dev-help@solr.apache.org
> > >>
> > >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > For additional commands, e-mail: dev-help@solr.apache.org
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jason Gerlowski <ge...@gmail.com>]

> Or are you hinting that we'll define Solr's security model in such a way that
> we'll no longer treat arbitrary filesystem read/write or arbitrary network access
> as a vulnerability?

Kindof, yeah, I think we should at least consider it.  (Obviously
communication-wise I wouldn't phrase it in terms of what reports we'd
reject going forward.  I'd frame it in terms of shifting where the
protection is coming from.  e.g. "Rather than relying on our security
code, rely instead on proven security tooling like: <list of
options>")

I love everything about the ByteBuddy / security-agent approach, in a
technical sense.  It's a huge improvement over our current setup.  But
it does come with a maintenance cost and it's worth weighing that for
ourselves.  OpenSearch decided the tradeoff was worth it.  But they
have more corporate backing and likely more community bandwidth than
we do, so we might weigh the same concerns and reach a different
conclusion.

If we're fully confident in the protections offered by security-agent,
and we're OK spending at least some bandwidth on responding to future
security-agent bugs, vuln reports, code maintenance, etc....great!  If
not, then maybe a more off-the-shelf approach to securing this stuff
is a better fit for us.

Best,

Jason

On Tue, Jun 9, 2026 at 7:51 AM Jan Høydahl <ja...@cominvent.com> wrote:
>
> Could have tried to use stock OpenSearch agent, but they do things a bit differently in the bootstrap and we may end up asking them to do changes to be more generic and accommodate us.
>
> Another option is to start a new Apache commons sub project, commons-sec-agent pull that as a dependency, and promote it for others to use and help maintain.
>
> Jan Høydahl
>
> > 7. juni 2026 kl. 18:15 skrev David Smiley <ds...@apache.org>:
> >
> > Well written Jason!  I agree 100% and I didn't communicate this well enough
> > previously when I used fewer words to essentially communicate the same.
> >
> > I see the agent coming in SIP-24 is a forked copy, and thus we must
> > maintain it.  That's unfortunate.  I hoped we could instead depend on
> > another project regularly maintaining and producing artifacts for 3rd party
> > (us) consumption.
> >
> > Perhaps the agent will allow us to remove from Solr the AllowUrlChecker
> > (whatever it's called) and similar for file paths.  I hope we get some
> > maintenance "relief".
> >
> >> On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <ge...@gmail.com>
> >> wrote:
> >>
> >> Apologies for chiming in so late on this.
> >>
> >> The Java agent route seems like a huge improvement over our existing
> >> approach.  I love that it's a global protection - it doesn't rely on
> >> devs remembering to call the required "check the path" method in every
> >> single place we might read a file in our code.
> >>
> >> But ultimately I still think systemd/seccomp is a much more
> >> sustainable route for us than the Java-agent approach
> >>
> >> We're a Search community that's here to solve people's Search
> >> problems.  I won't minimize for a second all the effort we've put into
> >> security over the years. But there are entire projects that exist to
> >> solve these sorts of problems.  IMO steering users towards something
> >> like systemd that has a whole community of security folks behind is
> >> going to leave them more secure, and be more sustainable for us.
> >>
> >> (To be clear I'm not advocating for systemd specifically, but for the
> >> general idea of relying on well-known, OS-level protections that exist
> >> outside of the JVM and Solr.)
> >>
> >> I've heard two objections to the "systemd" approach so far: that
> >> systemd isn't cross-platform, and that some folks won't bother to
> >> enable it.
> >>
> >> To the first objection, I'd say that our responsibility has never been
> >> to find one security solution that works for everyone.  We never
> >> mandated that folks use a particular firewall: we let users choose how
> >> they wanted to provide that network isolation.  IMO this could be
> >> handled the same way: point folks to a few different tools, optionally
> >> provide a few example configs, and let them pick based on their
> >> OS/platform.
> >>
> >> To the second objection: this is the whole point of having a security
> >> model.  It's expected and normal for projects to describe how to make
> >> a deployment of their software "secure".  It's neither fair nor
> >> possible to secure folks who won't read our "Going to Production" docs
> >> before deploying.
> >>
> >>> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com> wrote:
> >>>
> >>> So the implementation of SIP-24 is now ready for review at
> >>>
> >>> https://github.com/apache/solr/pull/4471
> >>>
> >>> The PR description gives through description of the change and what to
> >> look for.
> >>> I have run several rounds of Copilot reviews and Claude reviews.
> >>> We have unit tests and BATS tests, so coverage should be fairly good.
> >>>
> >>> The PR is fairly large +5454 lines across 62 files and 54 commits, but
> >> mostly
> >>> contained in the new gradle module, plus one hook in CoreContainer.
> >>>
> >>> Take it for a spin and let me know how it works.
> >>>
> >>> Jan
> >>>
> >>>> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> >>>>
> >>>> Hi,
> >>>>
> >>>> Welcoming more feedback on this approach.
> >>>>
> >>>> I'm planning to move to implementation phase on Wednesday, to get a
> >> first
> >>>> version of the agent ready for testing. But it would be helpful with
> >> as much
> >>>> feedback on the SIP high level design before I start implementing.
> >> Thanks David
> >>>> for your intial feedback.
> >>>>
> >>>> Jan
> >>>>
> >>>>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> >>>>>
> >>>>> I know it's probably unrealistic because corporate environments into
> >> which
> >>>>> solr is deployed likely would have difficulty with it, but there is a
> >> JDK
> >>>>> fork that keeps and improves the Security Manager... It's supported
> >> by the
> >>>>> descendants of the Apache River project.
> >>>>> https://github.com/pfirmstone/DirtyChai
> >>>>>
> >>>>> Probably not useful, but perhaps interesting. Certainly we are not
> >> the only
> >>>>> ones irritated by the loss of the security manager.
> >>>>>
> >>>>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com>
> >> wrote:
> >>>>>
> >>>>>>> I should clarify something.  My objections mostly relate to the
> >> insistent
> >>>>>>> language in the SIP requiring Solr to have a substitute for the
> >> JSM.  I'm
> >>>>>>> not quite against doing something but I might vote -0 on something
> >> that
> >>>>>>> seems to have a poor security payoff relative to the maintenance
> >> burden.
> >>>>>>> The more off-the-shelf, the better IMO.
> >>>>>>> Thank you for investing your time researching some options.
> >>>>>>
> >>>>>> The security landscape has dramatically changed during the years
> >> that we
> >>>>>> have enjoyed JSM protection for Solr. It's a crazy world out there
> >> and
> >>>>>> every
> >>>>>> single code flaw, existing and future, will be found and exploited or
> >>>>>> published
> >>>>>> by so called security researchers. That's why I believe we need a
> >>>>>> centralized
> >>>>>> solution.
> >>>>>>
> >>>>>>>> Rejected Alternatives
> >>>>>>>>
> >>>>>>>> - *Staying on Java < 24* — not viable long-term; Solr must support
> >>>>>>>> current Java LTS releases.
> >>>>>>>> - *Removing JSM protections without any replacement* — unacceptable
> >>>>>>>> security regression.
> >>>>>>>>
> >>>>>>> Both Eric Pugh and I have challenged this.
> >>>>>>>>
> >>>>>>>> - *OS-level hardening only (systemd, seccomp)* — not
> >> cross-platform;
> >>>>>>>> does not cover Windows or macOS.
> >>>>>>>>
> >>>>>>> I challenge this. Why should the Solr project burden itself with
> >> building
> >>>>>>> & maintaining security mechanisms already provided by off-the-shelf
> >>>>>> tools?
> >>>>>>> If a user/operator wishes to run on Windows/macOS that may not have
> >> this
> >>>>>>> protection mechanism, it is a risk consideration for that user to
> >>>>>> consider,
> >>>>>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line
> >> defense;
> >>>>>> it's
> >>>>>>> a defense-in-depth strategy.  Put differently, the protections here
> >> are
> >>>>>>> "best effort" but not worthy of a CVE if they were to falter.  I
> >> want to
> >>>>>>> get Arnout's opinion on this supposition.
> >>>>>>
> >>>>>> Java Security Manager is a beast, but its file system read/write
> >> controls
> >>>>>> have
> >>>>>> saved us many a CVE in 9.x when JSM have been on by default.
> >>>>>> The SIP primarily focuses on sandboxing Solr for wrt file and network
> >>>>>> access.
> >>>>>> The security landscape has changed dramatically last few years, and
> >> solving
> >>>>>> file- and network restrictions in a central place through
> >> interception
> >>>>>> rathen than
> >>>>>> at each of the 100+ call sites is a more sustainable way.
> >>>>>>
> >>>>>> Users are free to add layers outside the JVM as well, such as
> >> systemd,
> >>>>>> container,
> >>>>>> SELinux etc. But be honest - how many small/medium organizations
> >> really do
> >>>>>> this?
> >>>>>>
> >>>>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
> >>>>>>>> significantly more complex; adds ZK client dependency to the agent
> >>>>>> JAR.
> >>>>>>>> Superseded by port-based wildcards for intra-cluster traffic.
> >>>>>>>> - *Building a Java agent from scratch* — higher effort with no
> >>>>>>>> functional advantage over adapting the Apache 2.0-licensed
> >> OpenSearch
> >>>>>>>> implementation.
> >>>>>>>>
> >>>>>>>> I agree with not burdening this project with building &
> >> maintainining
> >>>>>> such
> >>>>>>> a mechanism.
> >>>>>>>
> >>>>>>> I'm not sure, to what degree, we can leverage that existing agent
> >> you
> >>>>>> speak
> >>>>>>> of without further burdening us.  It's a burden/reward trade-off.
> >>>>>>
> >>>>>> If the agent becomes a true maintenance burden (i.e. a larger burden
> >> than
> >>>>>> handling the CVEs it would prevent, then a valid action is to remove
> >> the
> >>>>>> agent again.
> >>>>>> Like any other feature we develop and maintain. Good this is that
> >> this is
> >>>>>> pure Java,
> >>>>>> and production-ready stable code.
> >>>>>>
> >>>>>>> In your updated proposal, you point
> >>>>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is
> >> called
> >>>>>> by
> >>>>>>> a number of places... although I *think* the original intention was
> >> only
> >>>>>> to
> >>>>>>> limit where cores are created?  Can you elaborate on what the role
> >> of
> >>>>>> that
> >>>>>>> method *should* be and how the JSM might also or work?
> >>>>>>
> >>>>>> The pathAllowed checks (also the ones pre-dating the centralization
> >> in
> >>>>>> CoreContainer,
> >>>>>> were introduced in 8.x, before JSM was introduced or enabled by
> >> default.
> >>>>>> Initially
> >>>>>> assertPathAllowed would validate end-user API input to avoid cores
> >> being
> >>>>>> created
> >>>>>> or loaded outside blessed folders. It has then been re-used for
> >> other code
> >>>>>> locations
> >>>>>> that may do file access based on user-input in API or config. It as
> >> also
> >>>>>> extended to
> >>>>>> block UNC paths after many attacks with this as a vector.
> >>>>>>
> >>>>>> I believe perhaps pathAllowed would not have been written had JSM
> >> already
> >>>>>> been
> >>>>>> enforced. Although I don't know if you can block UNC with JSM?
> >>>>>> The SIP does not propose to remove pathAllowed now. One benefit of
> >> early
> >>>>>> detection
> >>>>>> is that we can give better context-sensitive error- and log messages
> >>>>>> rather than throwing
> >>>>>> an exception.
> >>>>>>
> >>>>>> The agent approach laid out sandboxes four attack vectors
> >>>>>> * File access outside a limited set of folders and full block of
> >> Windows
> >>>>>> UNC
> >>>>>> * Network access other than peer solr nodes and zk nodes
> >>>>>> * Disabllow calling System.exit() (mainly useful for 3rd party
> >> plugins?)
> >>>>>> * Disallow spawning processes
> >>>>>>
> >>>>>> A nice thing with the agent approach is that it is unintrusive and
> >> easy to
> >>>>>> disable,
> >>>>>> so users who want to take care of all this on OS level may disable
> >> it, and
> >>>>>> if we or
> >>>>>> users don't find value in it down the road they can disable it or we
> >> can
> >>>>>> remove it.
> >>>>>>
> >>>>>>
> >>>>>> Jan
> >>>>>> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>>>>> For additional commands, e-mail: dev-help@solr.apache.org
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> http://www.needhamsoftware.com (work)
> >>>>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> >>>>
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >> For additional commands, e-mail: dev-help@solr.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jan Høydahl <ja...@cominvent.com>]

Could have tried to use stock OpenSearch agent, but they do things a bit differently in the bootstrap and we may end up asking them to do changes to be more generic and accommodate us.

Another option is to start a new Apache commons sub project, commons-sec-agent pull that as a dependency, and promote it for others to use and help maintain.

Jan Høydahl

> 7. juni 2026 kl. 18:15 skrev David Smiley <ds...@apache.org>:
> 
> Well written Jason!  I agree 100% and I didn't communicate this well enough
> previously when I used fewer words to essentially communicate the same.
> 
> I see the agent coming in SIP-24 is a forked copy, and thus we must
> maintain it.  That's unfortunate.  I hoped we could instead depend on
> another project regularly maintaining and producing artifacts for 3rd party
> (us) consumption.
> 
> Perhaps the agent will allow us to remove from Solr the AllowUrlChecker
> (whatever it's called) and similar for file paths.  I hope we get some
> maintenance "relief".
> 
>> On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <ge...@gmail.com>
>> wrote:
>> 
>> Apologies for chiming in so late on this.
>> 
>> The Java agent route seems like a huge improvement over our existing
>> approach.  I love that it's a global protection - it doesn't rely on
>> devs remembering to call the required "check the path" method in every
>> single place we might read a file in our code.
>> 
>> But ultimately I still think systemd/seccomp is a much more
>> sustainable route for us than the Java-agent approach
>> 
>> We're a Search community that's here to solve people's Search
>> problems.  I won't minimize for a second all the effort we've put into
>> security over the years. But there are entire projects that exist to
>> solve these sorts of problems.  IMO steering users towards something
>> like systemd that has a whole community of security folks behind is
>> going to leave them more secure, and be more sustainable for us.
>> 
>> (To be clear I'm not advocating for systemd specifically, but for the
>> general idea of relying on well-known, OS-level protections that exist
>> outside of the JVM and Solr.)
>> 
>> I've heard two objections to the "systemd" approach so far: that
>> systemd isn't cross-platform, and that some folks won't bother to
>> enable it.
>> 
>> To the first objection, I'd say that our responsibility has never been
>> to find one security solution that works for everyone.  We never
>> mandated that folks use a particular firewall: we let users choose how
>> they wanted to provide that network isolation.  IMO this could be
>> handled the same way: point folks to a few different tools, optionally
>> provide a few example configs, and let them pick based on their
>> OS/platform.
>> 
>> To the second objection: this is the whole point of having a security
>> model.  It's expected and normal for projects to describe how to make
>> a deployment of their software "secure".  It's neither fair nor
>> possible to secure folks who won't read our "Going to Production" docs
>> before deploying.
>> 
>>> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com> wrote:
>>> 
>>> So the implementation of SIP-24 is now ready for review at
>>> 
>>> https://github.com/apache/solr/pull/4471
>>> 
>>> The PR description gives through description of the change and what to
>> look for.
>>> I have run several rounds of Copilot reviews and Claude reviews.
>>> We have unit tests and BATS tests, so coverage should be fairly good.
>>> 
>>> The PR is fairly large +5454 lines across 62 files and 54 commits, but
>> mostly
>>> contained in the new gradle module, plus one hook in CoreContainer.
>>> 
>>> Take it for a spin and let me know how it works.
>>> 
>>> Jan
>>> 
>>>> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
>>>> 
>>>> Hi,
>>>> 
>>>> Welcoming more feedback on this approach.
>>>> 
>>>> I'm planning to move to implementation phase on Wednesday, to get a
>> first
>>>> version of the agent ready for testing. But it would be helpful with
>> as much
>>>> feedback on the SIP high level design before I start implementing.
>> Thanks David
>>>> for your intial feedback.
>>>> 
>>>> Jan
>>>> 
>>>>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
>>>>> 
>>>>> I know it's probably unrealistic because corporate environments into
>> which
>>>>> solr is deployed likely would have difficulty with it, but there is a
>> JDK
>>>>> fork that keeps and improves the Security Manager... It's supported
>> by the
>>>>> descendants of the Apache River project.
>>>>> https://github.com/pfirmstone/DirtyChai
>>>>> 
>>>>> Probably not useful, but perhaps interesting. Certainly we are not
>> the only
>>>>> ones irritated by the loss of the security manager.
>>>>> 
>>>>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com>
>> wrote:
>>>>> 
>>>>>>> I should clarify something.  My objections mostly relate to the
>> insistent
>>>>>>> language in the SIP requiring Solr to have a substitute for the
>> JSM.  I'm
>>>>>>> not quite against doing something but I might vote -0 on something
>> that
>>>>>>> seems to have a poor security payoff relative to the maintenance
>> burden.
>>>>>>> The more off-the-shelf, the better IMO.
>>>>>>> Thank you for investing your time researching some options.
>>>>>> 
>>>>>> The security landscape has dramatically changed during the years
>> that we
>>>>>> have enjoyed JSM protection for Solr. It's a crazy world out there
>> and
>>>>>> every
>>>>>> single code flaw, existing and future, will be found and exploited or
>>>>>> published
>>>>>> by so called security researchers. That's why I believe we need a
>>>>>> centralized
>>>>>> solution.
>>>>>> 
>>>>>>>> Rejected Alternatives
>>>>>>>> 
>>>>>>>> - *Staying on Java < 24* — not viable long-term; Solr must support
>>>>>>>> current Java LTS releases.
>>>>>>>> - *Removing JSM protections without any replacement* — unacceptable
>>>>>>>> security regression.
>>>>>>>> 
>>>>>>> Both Eric Pugh and I have challenged this.
>>>>>>>> 
>>>>>>>> - *OS-level hardening only (systemd, seccomp)* — not
>> cross-platform;
>>>>>>>> does not cover Windows or macOS.
>>>>>>>> 
>>>>>>> I challenge this. Why should the Solr project burden itself with
>> building
>>>>>>> & maintaining security mechanisms already provided by off-the-shelf
>>>>>> tools?
>>>>>>> If a user/operator wishes to run on Windows/macOS that may not have
>> this
>>>>>>> protection mechanism, it is a risk consideration for that user to
>>>>>> consider,
>>>>>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line
>> defense;
>>>>>> it's
>>>>>>> a defense-in-depth strategy.  Put differently, the protections here
>> are
>>>>>>> "best effort" but not worthy of a CVE if they were to falter.  I
>> want to
>>>>>>> get Arnout's opinion on this supposition.
>>>>>> 
>>>>>> Java Security Manager is a beast, but its file system read/write
>> controls
>>>>>> have
>>>>>> saved us many a CVE in 9.x when JSM have been on by default.
>>>>>> The SIP primarily focuses on sandboxing Solr for wrt file and network
>>>>>> access.
>>>>>> The security landscape has changed dramatically last few years, and
>> solving
>>>>>> file- and network restrictions in a central place through
>> interception
>>>>>> rathen than
>>>>>> at each of the 100+ call sites is a more sustainable way.
>>>>>> 
>>>>>> Users are free to add layers outside the JVM as well, such as
>> systemd,
>>>>>> container,
>>>>>> SELinux etc. But be honest - how many small/medium organizations
>> really do
>>>>>> this?
>>>>>> 
>>>>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
>>>>>>>> significantly more complex; adds ZK client dependency to the agent
>>>>>> JAR.
>>>>>>>> Superseded by port-based wildcards for intra-cluster traffic.
>>>>>>>> - *Building a Java agent from scratch* — higher effort with no
>>>>>>>> functional advantage over adapting the Apache 2.0-licensed
>> OpenSearch
>>>>>>>> implementation.
>>>>>>>> 
>>>>>>>> I agree with not burdening this project with building &
>> maintainining
>>>>>> such
>>>>>>> a mechanism.
>>>>>>> 
>>>>>>> I'm not sure, to what degree, we can leverage that existing agent
>> you
>>>>>> speak
>>>>>>> of without further burdening us.  It's a burden/reward trade-off.
>>>>>> 
>>>>>> If the agent becomes a true maintenance burden (i.e. a larger burden
>> than
>>>>>> handling the CVEs it would prevent, then a valid action is to remove
>> the
>>>>>> agent again.
>>>>>> Like any other feature we develop and maintain. Good this is that
>> this is
>>>>>> pure Java,
>>>>>> and production-ready stable code.
>>>>>> 
>>>>>>> In your updated proposal, you point
>>>>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is
>> called
>>>>>> by
>>>>>>> a number of places... although I *think* the original intention was
>> only
>>>>>> to
>>>>>>> limit where cores are created?  Can you elaborate on what the role
>> of
>>>>>> that
>>>>>>> method *should* be and how the JSM might also or work?
>>>>>> 
>>>>>> The pathAllowed checks (also the ones pre-dating the centralization
>> in
>>>>>> CoreContainer,
>>>>>> were introduced in 8.x, before JSM was introduced or enabled by
>> default.
>>>>>> Initially
>>>>>> assertPathAllowed would validate end-user API input to avoid cores
>> being
>>>>>> created
>>>>>> or loaded outside blessed folders. It has then been re-used for
>> other code
>>>>>> locations
>>>>>> that may do file access based on user-input in API or config. It as
>> also
>>>>>> extended to
>>>>>> block UNC paths after many attacks with this as a vector.
>>>>>> 
>>>>>> I believe perhaps pathAllowed would not have been written had JSM
>> already
>>>>>> been
>>>>>> enforced. Although I don't know if you can block UNC with JSM?
>>>>>> The SIP does not propose to remove pathAllowed now. One benefit of
>> early
>>>>>> detection
>>>>>> is that we can give better context-sensitive error- and log messages
>>>>>> rather than throwing
>>>>>> an exception.
>>>>>> 
>>>>>> The agent approach laid out sandboxes four attack vectors
>>>>>> * File access outside a limited set of folders and full block of
>> Windows
>>>>>> UNC
>>>>>> * Network access other than peer solr nodes and zk nodes
>>>>>> * Disabllow calling System.exit() (mainly useful for 3rd party
>> plugins?)
>>>>>> * Disallow spawning processes
>>>>>> 
>>>>>> A nice thing with the agent approach is that it is unintrusive and
>> easy to
>>>>>> disable,
>>>>>> so users who want to take care of all this on OS level may disable
>> it, and
>>>>>> if we or
>>>>>> users don't find value in it down the road they can disable it or we
>> can
>>>>>> remove it.
>>>>>> 
>>>>>> 
>>>>>> Jan
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>>>>> For additional commands, e-mail: dev-help@solr.apache.org
>>>>>> 
>>>>>> 
>>>>> 
>>>>> --
>>>>> http://www.needhamsoftware.com (work)
>>>>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
>>>> 
>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>> 
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[David Smiley <ds...@apache.org>]

Well written Jason!  I agree 100% and I didn't communicate this well enough
previously when I used fewer words to essentially communicate the same.

I see the agent coming in SIP-24 is a forked copy, and thus we must
maintain it.  That's unfortunate.  I hoped we could instead depend on
another project regularly maintaining and producing artifacts for 3rd party
(us) consumption.

Perhaps the agent will allow us to remove from Solr the AllowUrlChecker
(whatever it's called) and similar for file paths.  I hope we get some
maintenance "relief".

On Fri, Jun 5, 2026 at 2:48 PM Jason Gerlowski <ge...@gmail.com>
wrote:

> Apologies for chiming in so late on this.
>
> The Java agent route seems like a huge improvement over our existing
> approach.  I love that it's a global protection - it doesn't rely on
> devs remembering to call the required "check the path" method in every
> single place we might read a file in our code.
>
> But ultimately I still think systemd/seccomp is a much more
> sustainable route for us than the Java-agent approach
>
> We're a Search community that's here to solve people's Search
> problems.  I won't minimize for a second all the effort we've put into
> security over the years. But there are entire projects that exist to
> solve these sorts of problems.  IMO steering users towards something
> like systemd that has a whole community of security folks behind is
> going to leave them more secure, and be more sustainable for us.
>
> (To be clear I'm not advocating for systemd specifically, but for the
> general idea of relying on well-known, OS-level protections that exist
> outside of the JVM and Solr.)
>
> I've heard two objections to the "systemd" approach so far: that
> systemd isn't cross-platform, and that some folks won't bother to
> enable it.
>
> To the first objection, I'd say that our responsibility has never been
> to find one security solution that works for everyone.  We never
> mandated that folks use a particular firewall: we let users choose how
> they wanted to provide that network isolation.  IMO this could be
> handled the same way: point folks to a few different tools, optionally
> provide a few example configs, and let them pick based on their
> OS/platform.
>
> To the second objection: this is the whole point of having a security
> model.  It's expected and normal for projects to describe how to make
> a deployment of their software "secure".  It's neither fair nor
> possible to secure folks who won't read our "Going to Production" docs
> before deploying.
>
> On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com> wrote:
> >
> > So the implementation of SIP-24 is now ready for review at
> >
> > https://github.com/apache/solr/pull/4471
> >
> > The PR description gives through description of the change and what to
> look for.
> > I have run several rounds of Copilot reviews and Claude reviews.
> > We have unit tests and BATS tests, so coverage should be fairly good.
> >
> > The PR is fairly large +5454 lines across 62 files and 54 commits, but
> mostly
> > contained in the new gradle module, plus one hook in CoreContainer.
> >
> > Take it for a spin and let me know how it works.
> >
> > Jan
> >
> > > 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> > >
> > > Hi,
> > >
> > > Welcoming more feedback on this approach.
> > >
> > > I'm planning to move to implementation phase on Wednesday, to get a
> first
> > > version of the agent ready for testing. But it would be helpful with
> as much
> > > feedback on the SIP high level design before I start implementing.
> Thanks David
> > > for your intial feedback.
> > >
> > > Jan
> > >
> > >> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> > >>
> > >> I know it's probably unrealistic because corporate environments into
> which
> > >> solr is deployed likely would have difficulty with it, but there is a
> JDK
> > >> fork that keeps and improves the Security Manager... It's supported
> by the
> > >> descendants of the Apache River project.
> > >> https://github.com/pfirmstone/DirtyChai
> > >>
> > >> Probably not useful, but perhaps interesting. Certainly we are not
> the only
> > >> ones irritated by the loss of the security manager.
> > >>
> > >> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com>
> wrote:
> > >>
> > >>>> I should clarify something.  My objections mostly relate to the
> insistent
> > >>>> language in the SIP requiring Solr to have a substitute for the
> JSM.  I'm
> > >>>> not quite against doing something but I might vote -0 on something
> that
> > >>>> seems to have a poor security payoff relative to the maintenance
> burden.
> > >>>> The more off-the-shelf, the better IMO.
> > >>>> Thank you for investing your time researching some options.
> > >>>
> > >>> The security landscape has dramatically changed during the years
> that we
> > >>> have enjoyed JSM protection for Solr. It's a crazy world out there
> and
> > >>> every
> > >>> single code flaw, existing and future, will be found and exploited or
> > >>> published
> > >>> by so called security researchers. That's why I believe we need a
> > >>> centralized
> > >>> solution.
> > >>>
> > >>>>> Rejected Alternatives
> > >>>>>
> > >>>>> - *Staying on Java < 24* — not viable long-term; Solr must support
> > >>>>> current Java LTS releases.
> > >>>>> - *Removing JSM protections without any replacement* — unacceptable
> > >>>>> security regression.
> > >>>>>
> > >>>> Both Eric Pugh and I have challenged this.
> > >>>>>
> > >>>>> - *OS-level hardening only (systemd, seccomp)* — not
> cross-platform;
> > >>>>> does not cover Windows or macOS.
> > >>>>>
> > >>>> I challenge this. Why should the Solr project burden itself with
> building
> > >>>> & maintaining security mechanisms already provided by off-the-shelf
> > >>> tools?
> > >>>> If a user/operator wishes to run on Windows/macOS that may not have
> this
> > >>>> protection mechanism, it is a risk consideration for that user to
> > >>> consider,
> > >>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line
> defense;
> > >>> it's
> > >>>> a defense-in-depth strategy.  Put differently, the protections here
> are
> > >>>> "best effort" but not worthy of a CVE if they were to falter.  I
> want to
> > >>>> get Arnout's opinion on this supposition.
> > >>>
> > >>> Java Security Manager is a beast, but its file system read/write
> controls
> > >>> have
> > >>> saved us many a CVE in 9.x when JSM have been on by default.
> > >>> The SIP primarily focuses on sandboxing Solr for wrt file and network
> > >>> access.
> > >>> The security landscape has changed dramatically last few years, and
> solving
> > >>> file- and network restrictions in a central place through
> interception
> > >>> rathen than
> > >>> at each of the 100+ call sites is a more sustainable way.
> > >>>
> > >>> Users are free to add layers outside the JVM as well, such as
> systemd,
> > >>> container,
> > >>> SELinux etc. But be honest - how many small/medium organizations
> really do
> > >>> this?
> > >>>
> > >>>>> - *Dynamic ZK-watcher-based network policy* — correct but
> > >>>>> significantly more complex; adds ZK client dependency to the agent
> > >>> JAR.
> > >>>>> Superseded by port-based wildcards for intra-cluster traffic.
> > >>>>> - *Building a Java agent from scratch* — higher effort with no
> > >>>>> functional advantage over adapting the Apache 2.0-licensed
> OpenSearch
> > >>>>> implementation.
> > >>>>>
> > >>>>> I agree with not burdening this project with building &
> maintainining
> > >>> such
> > >>>> a mechanism.
> > >>>>
> > >>>> I'm not sure, to what degree, we can leverage that existing agent
> you
> > >>> speak
> > >>>> of without further burdening us.  It's a burden/reward trade-off.
> > >>>
> > >>> If the agent becomes a true maintenance burden (i.e. a larger burden
> than
> > >>> handling the CVEs it would prevent, then a valid action is to remove
> the
> > >>> agent again.
> > >>> Like any other feature we develop and maintain. Good this is that
> this is
> > >>> pure Java,
> > >>> and production-ready stable code.
> > >>>
> > >>>> In your updated proposal, you point
> > >>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is
> called
> > >>> by
> > >>>> a number of places... although I *think* the original intention was
> only
> > >>> to
> > >>>> limit where cores are created?  Can you elaborate on what the role
> of
> > >>> that
> > >>>> method *should* be and how the JSM might also or work?
> > >>>
> > >>> The pathAllowed checks (also the ones pre-dating the centralization
> in
> > >>> CoreContainer,
> > >>> were introduced in 8.x, before JSM was introduced or enabled by
> default.
> > >>> Initially
> > >>> assertPathAllowed would validate end-user API input to avoid cores
> being
> > >>> created
> > >>> or loaded outside blessed folders. It has then been re-used for
> other code
> > >>> locations
> > >>> that may do file access based on user-input in API or config. It as
> also
> > >>> extended to
> > >>> block UNC paths after many attacks with this as a vector.
> > >>>
> > >>> I believe perhaps pathAllowed would not have been written had JSM
> already
> > >>> been
> > >>> enforced. Although I don't know if you can block UNC with JSM?
> > >>> The SIP does not propose to remove pathAllowed now. One benefit of
> early
> > >>> detection
> > >>> is that we can give better context-sensitive error- and log messages
> > >>> rather than throwing
> > >>> an exception.
> > >>>
> > >>> The agent approach laid out sandboxes four attack vectors
> > >>> * File access outside a limited set of folders and full block of
> Windows
> > >>> UNC
> > >>> * Network access other than peer solr nodes and zk nodes
> > >>> * Disabllow calling System.exit() (mainly useful for 3rd party
> plugins?)
> > >>> * Disallow spawning processes
> > >>>
> > >>> A nice thing with the agent approach is that it is unintrusive and
> easy to
> > >>> disable,
> > >>> so users who want to take care of all this on OS level may disable
> it, and
> > >>> if we or
> > >>> users don't find value in it down the road they can disable it or we
> can
> > >>> remove it.
> > >>>
> > >>>
> > >>> Jan
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> > >>> For additional commands, e-mail: dev-help@solr.apache.org
> > >>>
> > >>>
> > >>
> > >> --
> > >> http://www.needhamsoftware.com (work)
> > >> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> > >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> For additional commands, e-mail: dev-help@solr.apache.org
>
>

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jason Gerlowski <ge...@gmail.com>]

Apologies for chiming in so late on this.

The Java agent route seems like a huge improvement over our existing
approach.  I love that it's a global protection - it doesn't rely on
devs remembering to call the required "check the path" method in every
single place we might read a file in our code.

But ultimately I still think systemd/seccomp is a much more
sustainable route for us than the Java-agent approach

We're a Search community that's here to solve people's Search
problems.  I won't minimize for a second all the effort we've put into
security over the years. But there are entire projects that exist to
solve these sorts of problems.  IMO steering users towards something
like systemd that has a whole community of security folks behind is
going to leave them more secure, and be more sustainable for us.

(To be clear I'm not advocating for systemd specifically, but for the
general idea of relying on well-known, OS-level protections that exist
outside of the JVM and Solr.)

I've heard two objections to the "systemd" approach so far: that
systemd isn't cross-platform, and that some folks won't bother to
enable it.

To the first objection, I'd say that our responsibility has never been
to find one security solution that works for everyone.  We never
mandated that folks use a particular firewall: we let users choose how
they wanted to provide that network isolation.  IMO this could be
handled the same way: point folks to a few different tools, optionally
provide a few example configs, and let them pick based on their
OS/platform.

To the second objection: this is the whole point of having a security
model.  It's expected and normal for projects to describe how to make
a deployment of their software "secure".  It's neither fair nor
possible to secure folks who won't read our "Going to Production" docs
before deploying.

On Thu, May 28, 2026 at 8:06 AM Jan Høydahl <ja...@cominvent.com> wrote:
>
> So the implementation of SIP-24 is now ready for review at
>
> https://github.com/apache/solr/pull/4471
>
> The PR description gives through description of the change and what to look for.
> I have run several rounds of Copilot reviews and Claude reviews.
> We have unit tests and BATS tests, so coverage should be fairly good.
>
> The PR is fairly large +5454 lines across 62 files and 54 commits, but mostly
> contained in the new gradle module, plus one hook in CoreContainer.
>
> Take it for a spin and let me know how it works.
>
> Jan
>
> > 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> >
> > Hi,
> >
> > Welcoming more feedback on this approach.
> >
> > I'm planning to move to implementation phase on Wednesday, to get a first
> > version of the agent ready for testing. But it would be helpful with as much
> > feedback on the SIP high level design before I start implementing. Thanks David
> > for your intial feedback.
> >
> > Jan
> >
> >> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> >>
> >> I know it's probably unrealistic because corporate environments into which
> >> solr is deployed likely would have difficulty with it, but there is a JDK
> >> fork that keeps and improves the Security Manager... It's supported by the
> >> descendants of the Apache River project.
> >> https://github.com/pfirmstone/DirtyChai
> >>
> >> Probably not useful, but perhaps interesting. Certainly we are not the only
> >> ones irritated by the loss of the security manager.
> >>
> >> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com> wrote:
> >>
> >>>> I should clarify something.  My objections mostly relate to the insistent
> >>>> language in the SIP requiring Solr to have a substitute for the JSM.  I'm
> >>>> not quite against doing something but I might vote -0 on something that
> >>>> seems to have a poor security payoff relative to the maintenance burden.
> >>>> The more off-the-shelf, the better IMO.
> >>>> Thank you for investing your time researching some options.
> >>>
> >>> The security landscape has dramatically changed during the years that we
> >>> have enjoyed JSM protection for Solr. It's a crazy world out there and
> >>> every
> >>> single code flaw, existing and future, will be found and exploited or
> >>> published
> >>> by so called security researchers. That's why I believe we need a
> >>> centralized
> >>> solution.
> >>>
> >>>>> Rejected Alternatives
> >>>>>
> >>>>> - *Staying on Java < 24* — not viable long-term; Solr must support
> >>>>> current Java LTS releases.
> >>>>> - *Removing JSM protections without any replacement* — unacceptable
> >>>>> security regression.
> >>>>>
> >>>> Both Eric Pugh and I have challenged this.
> >>>>>
> >>>>> - *OS-level hardening only (systemd, seccomp)* — not cross-platform;
> >>>>> does not cover Windows or macOS.
> >>>>>
> >>>> I challenge this. Why should the Solr project burden itself with building
> >>>> & maintaining security mechanisms already provided by off-the-shelf
> >>> tools?
> >>>> If a user/operator wishes to run on Windows/macOS that may not have this
> >>>> protection mechanism, it is a risk consideration for that user to
> >>> consider,
> >>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line defense;
> >>> it's
> >>>> a defense-in-depth strategy.  Put differently, the protections here are
> >>>> "best effort" but not worthy of a CVE if they were to falter.  I want to
> >>>> get Arnout's opinion on this supposition.
> >>>
> >>> Java Security Manager is a beast, but its file system read/write controls
> >>> have
> >>> saved us many a CVE in 9.x when JSM have been on by default.
> >>> The SIP primarily focuses on sandboxing Solr for wrt file and network
> >>> access.
> >>> The security landscape has changed dramatically last few years, and solving
> >>> file- and network restrictions in a central place through interception
> >>> rathen than
> >>> at each of the 100+ call sites is a more sustainable way.
> >>>
> >>> Users are free to add layers outside the JVM as well, such as systemd,
> >>> container,
> >>> SELinux etc. But be honest - how many small/medium organizations really do
> >>> this?
> >>>
> >>>>> - *Dynamic ZK-watcher-based network policy* — correct but
> >>>>> significantly more complex; adds ZK client dependency to the agent
> >>> JAR.
> >>>>> Superseded by port-based wildcards for intra-cluster traffic.
> >>>>> - *Building a Java agent from scratch* — higher effort with no
> >>>>> functional advantage over adapting the Apache 2.0-licensed OpenSearch
> >>>>> implementation.
> >>>>>
> >>>>> I agree with not burdening this project with building & maintainining
> >>> such
> >>>> a mechanism.
> >>>>
> >>>> I'm not sure, to what degree, we can leverage that existing agent you
> >>> speak
> >>>> of without further burdening us.  It's a burden/reward trade-off.
> >>>
> >>> If the agent becomes a true maintenance burden (i.e. a larger burden than
> >>> handling the CVEs it would prevent, then a valid action is to remove the
> >>> agent again.
> >>> Like any other feature we develop and maintain. Good this is that this is
> >>> pure Java,
> >>> and production-ready stable code.
> >>>
> >>>> In your updated proposal, you point
> >>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is called
> >>> by
> >>>> a number of places... although I *think* the original intention was only
> >>> to
> >>>> limit where cores are created?  Can you elaborate on what the role of
> >>> that
> >>>> method *should* be and how the JSM might also or work?
> >>>
> >>> The pathAllowed checks (also the ones pre-dating the centralization in
> >>> CoreContainer,
> >>> were introduced in 8.x, before JSM was introduced or enabled by default.
> >>> Initially
> >>> assertPathAllowed would validate end-user API input to avoid cores being
> >>> created
> >>> or loaded outside blessed folders. It has then been re-used for other code
> >>> locations
> >>> that may do file access based on user-input in API or config. It as also
> >>> extended to
> >>> block UNC paths after many attacks with this as a vector.
> >>>
> >>> I believe perhaps pathAllowed would not have been written had JSM already
> >>> been
> >>> enforced. Although I don't know if you can block UNC with JSM?
> >>> The SIP does not propose to remove pathAllowed now. One benefit of early
> >>> detection
> >>> is that we can give better context-sensitive error- and log messages
> >>> rather than throwing
> >>> an exception.
> >>>
> >>> The agent approach laid out sandboxes four attack vectors
> >>> * File access outside a limited set of folders and full block of Windows
> >>> UNC
> >>> * Network access other than peer solr nodes and zk nodes
> >>> * Disabllow calling System.exit() (mainly useful for 3rd party plugins?)
> >>> * Disallow spawning processes
> >>>
> >>> A nice thing with the agent approach is that it is unintrusive and easy to
> >>> disable,
> >>> so users who want to take care of all this on OS level may disable it, and
> >>> if we or
> >>> users don't find value in it down the road they can disable it or we can
> >>> remove it.
> >>>
> >>>
> >>> Jan
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
> >>> For additional commands, e-mail: dev-help@solr.apache.org
> >>>
> >>>
> >>
> >> --
> >> http://www.needhamsoftware.com (work)
> >> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> >
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jan Høydahl <ja...@cominvent.com>]

So the implementation of SIP-24 is now ready for review at

https://github.com/apache/solr/pull/4471

The PR description gives through description of the change and what to look for.
I have run several rounds of Copilot reviews and Claude reviews.
We have unit tests and BATS tests, so coverage should be fairly good.

The PR is fairly large +5454 lines across 62 files and 54 commits, but mostly
contained in the new gradle module, plus one hook in CoreContainer.

Take it for a spin and let me know how it works.

Jan

> 11. mai 2026 kl. 00:04 skrev Jan Høydahl <ja...@cominvent.com>:
> 
> Hi,
> 
> Welcoming more feedback on this approach.
> 
> I'm planning to move to implementation phase on Wednesday, to get a first 
> version of the agent ready for testing. But it would be helpful with as much 
> feedback on the SIP high level design before I start implementing. Thanks David
> for your intial feedback.
> 
> Jan
> 
>> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
>> 
>> I know it's probably unrealistic because corporate environments into which
>> solr is deployed likely would have difficulty with it, but there is a JDK
>> fork that keeps and improves the Security Manager... It's supported by the
>> descendants of the Apache River project.
>> https://github.com/pfirmstone/DirtyChai
>> 
>> Probably not useful, but perhaps interesting. Certainly we are not the only
>> ones irritated by the loss of the security manager.
>> 
>> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com> wrote:
>> 
>>>> I should clarify something.  My objections mostly relate to the insistent
>>>> language in the SIP requiring Solr to have a substitute for the JSM.  I'm
>>>> not quite against doing something but I might vote -0 on something that
>>>> seems to have a poor security payoff relative to the maintenance burden.
>>>> The more off-the-shelf, the better IMO.
>>>> Thank you for investing your time researching some options.
>>> 
>>> The security landscape has dramatically changed during the years that we
>>> have enjoyed JSM protection for Solr. It's a crazy world out there and
>>> every
>>> single code flaw, existing and future, will be found and exploited or
>>> published
>>> by so called security researchers. That's why I believe we need a
>>> centralized
>>> solution.
>>> 
>>>>> Rejected Alternatives
>>>>> 
>>>>> - *Staying on Java < 24* — not viable long-term; Solr must support
>>>>> current Java LTS releases.
>>>>> - *Removing JSM protections without any replacement* — unacceptable
>>>>> security regression.
>>>>> 
>>>> Both Eric Pugh and I have challenged this.
>>>>> 
>>>>> - *OS-level hardening only (systemd, seccomp)* — not cross-platform;
>>>>> does not cover Windows or macOS.
>>>>> 
>>>> I challenge this. Why should the Solr project burden itself with building
>>>> & maintaining security mechanisms already provided by off-the-shelf
>>> tools?
>>>> If a user/operator wishes to run on Windows/macOS that may not have this
>>>> protection mechanism, it is a risk consideration for that user to
>>> consider,
>>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line defense;
>>> it's
>>>> a defense-in-depth strategy.  Put differently, the protections here are
>>>> "best effort" but not worthy of a CVE if they were to falter.  I want to
>>>> get Arnout's opinion on this supposition.
>>> 
>>> Java Security Manager is a beast, but its file system read/write controls
>>> have
>>> saved us many a CVE in 9.x when JSM have been on by default.
>>> The SIP primarily focuses on sandboxing Solr for wrt file and network
>>> access.
>>> The security landscape has changed dramatically last few years, and solving
>>> file- and network restrictions in a central place through interception
>>> rathen than
>>> at each of the 100+ call sites is a more sustainable way.
>>> 
>>> Users are free to add layers outside the JVM as well, such as systemd,
>>> container,
>>> SELinux etc. But be honest - how many small/medium organizations really do
>>> this?
>>> 
>>>>> - *Dynamic ZK-watcher-based network policy* — correct but
>>>>> significantly more complex; adds ZK client dependency to the agent
>>> JAR.
>>>>> Superseded by port-based wildcards for intra-cluster traffic.
>>>>> - *Building a Java agent from scratch* — higher effort with no
>>>>> functional advantage over adapting the Apache 2.0-licensed OpenSearch
>>>>> implementation.
>>>>> 
>>>>> I agree with not burdening this project with building & maintainining
>>> such
>>>> a mechanism.
>>>> 
>>>> I'm not sure, to what degree, we can leverage that existing agent you
>>> speak
>>>> of without further burdening us.  It's a burden/reward trade-off.
>>> 
>>> If the agent becomes a true maintenance burden (i.e. a larger burden than
>>> handling the CVEs it would prevent, then a valid action is to remove the
>>> agent again.
>>> Like any other feature we develop and maintain. Good this is that this is
>>> pure Java,
>>> and production-ready stable code.
>>> 
>>>> In your updated proposal, you point
>>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is called
>>> by
>>>> a number of places... although I *think* the original intention was only
>>> to
>>>> limit where cores are created?  Can you elaborate on what the role of
>>> that
>>>> method *should* be and how the JSM might also or work?
>>> 
>>> The pathAllowed checks (also the ones pre-dating the centralization in
>>> CoreContainer,
>>> were introduced in 8.x, before JSM was introduced or enabled by default.
>>> Initially
>>> assertPathAllowed would validate end-user API input to avoid cores being
>>> created
>>> or loaded outside blessed folders. It has then been re-used for other code
>>> locations
>>> that may do file access based on user-input in API or config. It as also
>>> extended to
>>> block UNC paths after many attacks with this as a vector.
>>> 
>>> I believe perhaps pathAllowed would not have been written had JSM already
>>> been
>>> enforced. Although I don't know if you can block UNC with JSM?
>>> The SIP does not propose to remove pathAllowed now. One benefit of early
>>> detection
>>> is that we can give better context-sensitive error- and log messages
>>> rather than throwing
>>> an exception.
>>> 
>>> The agent approach laid out sandboxes four attack vectors
>>> * File access outside a limited set of folders and full block of Windows
>>> UNC
>>> * Network access other than peer solr nodes and zk nodes
>>> * Disabllow calling System.exit() (mainly useful for 3rd party plugins?)
>>> * Disallow spawning processes
>>> 
>>> A nice thing with the agent approach is that it is unintrusive and easy to
>>> disable,
>>> so users who want to take care of all this on OS level may disable it, and
>>> if we or
>>> users don't find value in it down the road they can disable it or we can
>>> remove it.
>>> 
>>> 
>>> Jan
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>>> For additional commands, e-mail: dev-help@solr.apache.org
>>> 
>>> 
>> 
>> -- 
>> http://www.needhamsoftware.com (work)
>> https://a.co/d/b2sZLD9 (my fantasy fiction book)
> 

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jan Høydahl <ja...@cominvent.com>]

Hi,

Welcoming more feedback on this approach.

I'm planning to move to implementation phase on Wednesday, to get a first 
version of the agent ready for testing. But it would be helpful with as much 
feedback on the SIP high level design before I start implementing. Thanks David
for your intial feedback.

Jan

> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> 
> I know it's probably unrealistic because corporate environments into which
> solr is deployed likely would have difficulty with it, but there is a JDK
> fork that keeps and improves the Security Manager... It's supported by the
> descendants of the Apache River project.
> https://github.com/pfirmstone/DirtyChai
> 
> Probably not useful, but perhaps interesting. Certainly we are not the only
> ones irritated by the loss of the security manager.
> 
> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com> wrote:
> 
>>> I should clarify something.  My objections mostly relate to the insistent
>>> language in the SIP requiring Solr to have a substitute for the JSM.  I'm
>>> not quite against doing something but I might vote -0 on something that
>>> seems to have a poor security payoff relative to the maintenance burden.
>>> The more off-the-shelf, the better IMO.
>>> Thank you for investing your time researching some options.
>> 
>> The security landscape has dramatically changed during the years that we
>> have enjoyed JSM protection for Solr. It's a crazy world out there and
>> every
>> single code flaw, existing and future, will be found and exploited or
>> published
>> by so called security researchers. That's why I believe we need a
>> centralized
>> solution.
>> 
>>>> Rejected Alternatives
>>>> 
>>>>  - *Staying on Java < 24* — not viable long-term; Solr must support
>>>>  current Java LTS releases.
>>>>  - *Removing JSM protections without any replacement* — unacceptable
>>>>  security regression.
>>>> 
>>> Both Eric Pugh and I have challenged this.
>>>> 
>>>>  - *OS-level hardening only (systemd, seccomp)* — not cross-platform;
>>>>  does not cover Windows or macOS.
>>>> 
>>> I challenge this. Why should the Solr project burden itself with building
>>> & maintaining security mechanisms already provided by off-the-shelf
>> tools?
>>> If a user/operator wishes to run on Windows/macOS that may not have this
>>> protection mechanism, it is a risk consideration for that user to
>> consider,
>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line defense;
>> it's
>>> a defense-in-depth strategy.  Put differently, the protections here are
>>> "best effort" but not worthy of a CVE if they were to falter.  I want to
>>> get Arnout's opinion on this supposition.
>> 
>> Java Security Manager is a beast, but its file system read/write controls
>> have
>> saved us many a CVE in 9.x when JSM have been on by default.
>> The SIP primarily focuses on sandboxing Solr for wrt file and network
>> access.
>> The security landscape has changed dramatically last few years, and solving
>> file- and network restrictions in a central place through interception
>> rathen than
>> at each of the 100+ call sites is a more sustainable way.
>> 
>> Users are free to add layers outside the JVM as well, such as systemd,
>> container,
>> SELinux etc. But be honest - how many small/medium organizations really do
>> this?
>> 
>>>>  - *Dynamic ZK-watcher-based network policy* — correct but
>>>>  significantly more complex; adds ZK client dependency to the agent
>> JAR.
>>>>  Superseded by port-based wildcards for intra-cluster traffic.
>>>>  - *Building a Java agent from scratch* — higher effort with no
>>>>  functional advantage over adapting the Apache 2.0-licensed OpenSearch
>>>>  implementation.
>>>> 
>>>> I agree with not burdening this project with building & maintainining
>> such
>>> a mechanism.
>>> 
>>> I'm not sure, to what degree, we can leverage that existing agent you
>> speak
>>> of without further burdening us.  It's a burden/reward trade-off.
>> 
>> If the agent becomes a true maintenance burden (i.e. a larger burden than
>> handling the CVEs it would prevent, then a valid action is to remove the
>> agent again.
>> Like any other feature we develop and maintain. Good this is that this is
>> pure Java,
>> and production-ready stable code.
>> 
>>> In your updated proposal, you point
>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is called
>> by
>>> a number of places... although I *think* the original intention was only
>> to
>>> limit where cores are created?  Can you elaborate on what the role of
>> that
>>> method *should* be and how the JSM might also or work?
>> 
>> The pathAllowed checks (also the ones pre-dating the centralization in
>> CoreContainer,
>> were introduced in 8.x, before JSM was introduced or enabled by default.
>> Initially
>> assertPathAllowed would validate end-user API input to avoid cores being
>> created
>> or loaded outside blessed folders. It has then been re-used for other code
>> locations
>> that may do file access based on user-input in API or config. It as also
>> extended to
>> block UNC paths after many attacks with this as a vector.
>> 
>> I believe perhaps pathAllowed would not have been written had JSM already
>> been
>> enforced. Although I don't know if you can block UNC with JSM?
>> The SIP does not propose to remove pathAllowed now. One benefit of early
>> detection
>> is that we can give better context-sensitive error- and log messages
>> rather than throwing
>> an exception.
>> 
>> The agent approach laid out sandboxes four attack vectors
>> * File access outside a limited set of folders and full block of Windows
>> UNC
>> * Network access other than peer solr nodes and zk nodes
>> * Disabllow calling System.exit() (mainly useful for 3rd party plugins?)
>> * Disallow spawning processes
>> 
>> A nice thing with the agent approach is that it is unintrusive and easy to
>> disable,
>> so users who want to take care of all this on OS level may disable it, and
>> if we or
>> users don't find value in it down the road they can disable it or we can
>> remove it.
>> 
>> 
>> Jan
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>> 
>> 
> 
> -- 
> http://www.needhamsoftware.com (work)
> https://a.co/d/b2sZLD9 (my fantasy fiction book)


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org

Re: [DISCUSS] SIP-24 Java Security Manager replacement

[Jan Høydahl <ja...@cominvent.com>]

Hi,

Welcoming more feedback on this approach.

I'm planning to move to implementation phase on Wednesday, to get a first 
version of the agent ready for testing. But it would be helpful with as much 
feedback on the SIP high level design before I start implementing. Thanks David
for your intial feedback.

Jan

> 1. mai 2026 kl. 14:30 skrev Gus Heck <gu...@gmail.com>:
> 
> I know it's probably unrealistic because corporate environments into which
> solr is deployed likely would have difficulty with it, but there is a JDK
> fork that keeps and improves the Security Manager... It's supported by the
> descendants of the Apache River project.
> https://github.com/pfirmstone/DirtyChai
> 
> Probably not useful, but perhaps interesting. Certainly we are not the only
> ones irritated by the loss of the security manager.
> 
> On Thu, Apr 30, 2026 at 4:08 AM Jan Høydahl <ja...@cominvent.com> wrote:
> 
>>> I should clarify something.  My objections mostly relate to the insistent
>>> language in the SIP requiring Solr to have a substitute for the JSM.  I'm
>>> not quite against doing something but I might vote -0 on something that
>>> seems to have a poor security payoff relative to the maintenance burden.
>>> The more off-the-shelf, the better IMO.
>>> Thank you for investing your time researching some options.
>> 
>> The security landscape has dramatically changed during the years that we
>> have enjoyed JSM protection for Solr. It's a crazy world out there and
>> every
>> single code flaw, existing and future, will be found and exploited or
>> published
>> by so called security researchers. That's why I believe we need a
>> centralized
>> solution.
>> 
>>>> Rejected Alternatives
>>>> 
>>>>  - *Staying on Java < 24* — not viable long-term; Solr must support
>>>>  current Java LTS releases.
>>>>  - *Removing JSM protections without any replacement* — unacceptable
>>>>  security regression.
>>>> 
>>> Both Eric Pugh and I have challenged this.
>>>> 
>>>>  - *OS-level hardening only (systemd, seccomp)* — not cross-platform;
>>>>  does not cover Windows or macOS.
>>>> 
>>> I challenge this. Why should the Solr project burden itself with building
>>> & maintaining security mechanisms already provided by off-the-shelf
>> tools?
>>> If a user/operator wishes to run on Windows/macOS that may not have this
>>> protection mechanism, it is a risk consideration for that user to
>> consider,
>>> but isn't a deal-breaker.  The JSM wasn't/isn't a front-line defense;
>> it's
>>> a defense-in-depth strategy.  Put differently, the protections here are
>>> "best effort" but not worthy of a CVE if they were to falter.  I want to
>>> get Arnout's opinion on this supposition.
>> 
>> Java Security Manager is a beast, but its file system read/write controls
>> have
>> saved us many a CVE in 9.x when JSM have been on by default.
>> The SIP primarily focuses on sandboxing Solr for wrt file and network
>> access.
>> The security landscape has changed dramatically last few years, and solving
>> file- and network restrictions in a central place through interception
>> rathen than
>> at each of the 100+ call sites is a more sustainable way.
>> 
>> Users are free to add layers outside the JVM as well, such as systemd,
>> container,
>> SELinux etc. But be honest - how many small/medium organizations really do
>> this?
>> 
>>>>  - *Dynamic ZK-watcher-based network policy* — correct but
>>>>  significantly more complex; adds ZK client dependency to the agent
>> JAR.
>>>>  Superseded by port-based wildcards for intra-cluster traffic.
>>>>  - *Building a Java agent from scratch* — higher effort with no
>>>>  functional advantage over adapting the Apache 2.0-licensed OpenSearch
>>>>  implementation.
>>>> 
>>>> I agree with not burdening this project with building & maintainining
>> such
>>> a mechanism.
>>> 
>>> I'm not sure, to what degree, we can leverage that existing agent you
>> speak
>>> of without further burdening us.  It's a burden/reward trade-off.
>> 
>> If the agent becomes a true maintenance burden (i.e. a larger burden than
>> handling the CVEs it would prevent, then a valid action is to remove the
>> agent again.
>> Like any other feature we develop and maintain. Good this is that this is
>> pure Java,
>> and production-ready stable code.
>> 
>>> In your updated proposal, you point
>>> out org.apache.solr.core.CoreContainer#assertPathAllowed  That is called
>> by
>>> a number of places... although I *think* the original intention was only
>> to
>>> limit where cores are created?  Can you elaborate on what the role of
>> that
>>> method *should* be and how the JSM might also or work?
>> 
>> The pathAllowed checks (also the ones pre-dating the centralization in
>> CoreContainer,
>> were introduced in 8.x, before JSM was introduced or enabled by default.
>> Initially
>> assertPathAllowed would validate end-user API input to avoid cores being
>> created
>> or loaded outside blessed folders. It has then been re-used for other code
>> locations
>> that may do file access based on user-input in API or config. It as also
>> extended to
>> block UNC paths after many attacks with this as a vector.
>> 
>> I believe perhaps pathAllowed would not have been written had JSM already
>> been
>> enforced. Although I don't know if you can block UNC with JSM?
>> The SIP does not propose to remove pathAllowed now. One benefit of early
>> detection
>> is that we can give better context-sensitive error- and log messages
>> rather than throwing
>> an exception.
>> 
>> The agent approach laid out sandboxes four attack vectors
>> * File access outside a limited set of folders and full block of Windows
>> UNC
>> * Network access other than peer solr nodes and zk nodes
>> * Disabllow calling System.exit() (mainly useful for 3rd party plugins?)
>> * Disallow spawning processes
>> 
>> A nice thing with the agent approach is that it is unintrusive and easy to
>> disable,
>> so users who want to take care of all this on OS level may disable it, and
>> if we or
>> users don't find value in it down the road they can disable it or we can
>> remove it.
>> 
>> 
>> Jan
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
>> For additional commands, e-mail: dev-help@solr.apache.org
>> 
>> 
> 
> -- 
> http://www.needhamsoftware.com (work)
> https://a.co/d/b2sZLD9 (my fantasy fiction book)


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@solr.apache.org
For additional commands, e-mail: dev-help@solr.apache.org