You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Erick Erickson <er...@gmail.com> on 2020/08/29 12:54:01 UTC
Annoying but harmless exceptions due to filepermissions when running
tests
These exceptions are handled in the code and don’t affect running tests, but they can be a distraction when trying to figure out what’s causing a failure. When CoreContainer is being initialized, these two paths get “Permission Denied” errors since they try to create directories/files.
java.security.AccessControlException: access denied ("java.io.FilePermission" "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filestore" "write”)
java.security.AccessControlException: access denied ("java.io.FilePermission" "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userfiles" "write”)
In both cases, the code logs a warning like "Features requiring this directory may fail”.
“build” is permitted this way, so I guess gradle runs as some other user?
drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
Any hints on an easy way to avoid these? It’s not worth much effort I don’t think since they’re handled, but if it’s easy I’d like to not see them.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Annoying but harmless exceptions due to filepermissions when
running tests
Posted by Erick Erickson <er...@gmail.com>.
I think the best option is to learn to ignore the harmless error message in test logs……
This was one of those “if there were just a few, we can fix the test to set itself up in a writable directory”. But Hoss is right, it’s pretty common to lots and lots and lots of tests and it’s just a minor annoyance when trying to read through the output of a test that fails, it has no functional significance.
So I closed the JIRA “invalid” and y’all should go back to your regularly scheduled programming ;)
Sorry for the noise
Erick
> On Sep 2, 2020, at 9:54 PM, Alexandre Rafalovitch <ar...@gmail.com> wrote:
>
> There is a flag to disable package manager. Can that code path avoid creating directory? Or maybe it does already.
>
> Then tests that don't test that specifically could have disable flag on.
>
> Regards,
> Alex
>
> On Wed., Sep. 2, 2020, 9:41 p.m. Noble Paul, <no...@gmail.com> wrote:
> The filestore dir is where the packages live. If we move it to another location, existing installations might fail. So, it's a backward incompatible change.
>
> What are our options?
>
> Is it possible to have these directories precreated in the distro/code to ensure that these warnings don't come.
>
> On Thu, Sep 3, 2020, 4:58 AM Erick Erickson <er...@gmail.com> wrote:
> Oh bother. Somehow I thought I’d looked and only a handful of tests reported this.
>
> So I looked again and I _wish_ I was able to blame drugs cause you’re right, there
> are over a thousand of them.
>
> Never mind...
>
> > On Sep 1, 2020, at 8:55 PM, Chris Hostetter <ho...@fucit.org> wrote:
> >
> >
> > : Hmmm, that’s kind of an dilemma then. Are you saying that
> > : he test can see that the directory appears writable then tries
> > : to write to it then gets tripped up by the framework?
> > :
> > : Seems to me that a test that tries to write, thinks it can and then
> > : can’t should fail anyway.
> > :
> > : Well, I don’t think there are very many tests that have this problem
> > : anyway, so maybe I can examine them one-by-one and not
> > : introduce new failures...
> >
> > You keep using the phrase "the test" in the context of (trying to) create
> > these directories ("userfiles" and "filestore") ... the "test CODE" isn't
> > making any choices about trying to write those files -- the choice is
> > being made by the "CoreContainer CODE".
> >
> > These features were added with the explicit implementation choice to _TRY_
> > to write the "usersfiles" (and/or "filestore") directory to "solr home" IF
> > POSSIBLE, and if so then enable a bunch of features -- if NOT then log a
> > WARNing and don't enable those features.
> >
> > So what you're seeing here isn't an artifact/result of any particular
> > choices "a test" or "the test" makes -- it's a concious choice of the
> > developer who added this feature to solr. These WARN messages that show
> > up in tests where the solr home dir isn't writable (which is actaully the
> > vast majority of tests because of how the test framework works) are the
> > same types of WARN messages that a "real" solr deployment might get if
> > their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir} to
> > point to a diff drive).
> >
> >
> >
> > :
> > : > On Aug 31, 2020, at 1:29 PM, Chris Hostetter <ho...@fucit.org> wrote:
> > : >
> > : >
> > : > Some tests "create" a new solr home dir and copy config files there, but
> > : > you'll see this type of WARN logging for any test that just uses the test
> > : > configs "in place" because of how the code is designed to _try_ and create
> > : > a userfiles directory in the solr home if it's writable.
> > : >
> > : >
> > : > : Date: Sat, 29 Aug 2020 09:25:17 -0400
> > : > : From: Erick Erickson <er...@gmail.com>
> > : > : Reply-To: dev@lucene.apache.org
> > : > : To: dev@lucene.apache.org
> > : > : Subject: Re: Annoying but harmless exceptions due to filepermissions when
> > : > : running tests
> > : > :
> > : > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
> > : > :
> > : > : In CoreContainer [364] there’s code like this:
> > : > :
> > : > : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make configurable on cfg?
> > : > : try {
> > : > : Files.createDirectories(userFilesPath); // does nothing if already exists
> > : > : } catch (Exception e) {
> > : > : log.warn("Unable to create [{}]. Features requiring this directory may fail.", userFilesPath, e);
> > : > : }
> > : > :
> > : > : So I assumed it would happen on most every test, at least in cloud mode. But when I tried to make it happen on a different test, there was no exception.
> > : > :
> > : > : I’ll have to poke some more to see what’s really happening…
> > : > :
> > : > : Never Mind….
> > : > :
> > : > : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> > : > : >
> > : > : > Hi,
> > : > : >
> > : > : > this is a bug in the test! It should never ever modify files outside its sandbox. It should not even modify files in build directory. It is fully valid to reject those writes - has nothing to do with users, it's just forbidden by the test framework. Modifying this file is harmful, as it may affect later tests.
> > : > : >
> > : > : > So the correct way is to copy those files to the solr container running inside test's sandbox. That's one of the main advantages of the Test sandbox: No write access anywhere outside the test, see policy file.
> > : > : >
> > : > : > Uwe
> > : > : >
> > : > : > -----
> > : > : > Uwe Schindler
> > : > : > Achterdiek 19, D-28357 Bremen
> > : > : > https://www.thetaphi.de
> > : > : > eMail: uwe@thetaphi.de
> > : > : >
> > : > : >> -----Original Message-----
> > : > : >> From: Erick Erickson <er...@gmail.com>
> > : > : >> Sent: Saturday, August 29, 2020 2:54 PM
> > : > : >> To: dev@lucene.apache.org
> > : > : >> Subject: Annoying but harmless exceptions due to filepermissions when running
> > : > : >> tests
> > : > : >>
> > : > : >> These exceptions are handled in the code and don’t affect running tests, but
> > : > : >> they can be a distraction when trying to figure out what’s causing a failure.
> > : > : >> When CoreContainer is being initialized, these two paths get “Permission
> > : > : >> Denied” errors since they try to create directories/files.
> > : > : >>
> > : > : >> java.security.AccessControlException: access denied ("java.io.FilePermission"
> > : > : >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
> > : > : >> ore" "write”)
> > : > : >>
> > : > : >> java.security.AccessControlException: access denied ("java.io.FilePermission"
> > : > : >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
> > : > : >> iles" "write”)
> > : > : >>
> > : > : >> In both cases, the code logs a warning like "Features requiring this directory
> > : > : >> may fail”.
> > : > : >>
> > : > : >> “build” is permitted this way, so I guess gradle runs as some other user?
> > : > : >>
> > : > : >> drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
> > : > : >>
> > : > : >> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
> > : > : >> think since they’re handled, but if it’s easy I’d like to not see them.
> > : > : >>
> > : > : >>
> > : > : >>
> > : > : >> ---------------------------------------------------------------------
> > : > : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : > : >> For additional commands, e-mail: dev-help@lucene.apache.org
> > : > : >
> > : > : >
> > : > : > ---------------------------------------------------------------------
> > : > : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : > : > For additional commands, e-mail: dev-help@lucene.apache.org
> > : > : >
> > : > :
> > : > :
> > : > : ---------------------------------------------------------------------
> > : > : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : > : For additional commands, e-mail: dev-help@lucene.apache.org
> > : > :
> > : > :
> > : >
> > : > -Hoss
> > : > http://www.lucidworks.com/
> > : >
> > : > ---------------------------------------------------------------------
> > : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : > For additional commands, e-mail: dev-help@lucene.apache.org
> > :
> > :
> > : ---------------------------------------------------------------------
> > : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : For additional commands, e-mail: dev-help@lucene.apache.org
> > :
> > :
> >
> > -Hoss
> > http://www.lucidworks.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Annoying but harmless exceptions due to filepermissions when
running tests
Posted by Alexandre Rafalovitch <ar...@gmail.com>.
There is a flag to disable package manager. Can that code path avoid
creating directory? Or maybe it does already.
Then tests that don't test that specifically could have disable flag on.
Regards,
Alex
On Wed., Sep. 2, 2020, 9:41 p.m. Noble Paul, <no...@gmail.com> wrote:
> The filestore dir is where the packages live. If we move it to another
> location, existing installations might fail. So, it's a backward
> incompatible change.
>
> What are our options?
>
> Is it possible to have these directories precreated in the distro/code to
> ensure that these warnings don't come.
>
> On Thu, Sep 3, 2020, 4:58 AM Erick Erickson <er...@gmail.com>
> wrote:
>
>> Oh bother. Somehow I thought I’d looked and only a handful of tests
>> reported this.
>>
>> So I looked again and I _wish_ I was able to blame drugs cause you’re
>> right, there
>> are over a thousand of them.
>>
>> Never mind...
>>
>> > On Sep 1, 2020, at 8:55 PM, Chris Hostetter <ho...@fucit.org>
>> wrote:
>> >
>> >
>> > : Hmmm, that’s kind of an dilemma then. Are you saying that
>> > : he test can see that the directory appears writable then tries
>> > : to write to it then gets tripped up by the framework?
>> > :
>> > : Seems to me that a test that tries to write, thinks it can and then
>> > : can’t should fail anyway.
>> > :
>> > : Well, I don’t think there are very many tests that have this problem
>> > : anyway, so maybe I can examine them one-by-one and not
>> > : introduce new failures...
>> >
>> > You keep using the phrase "the test" in the context of (trying to)
>> create
>> > these directories ("userfiles" and "filestore") ... the "test CODE"
>> isn't
>> > making any choices about trying to write those files -- the choice is
>> > being made by the "CoreContainer CODE".
>> >
>> > These features were added with the explicit implementation choice to
>> _TRY_
>> > to write the "usersfiles" (and/or "filestore") directory to "solr home"
>> IF
>> > POSSIBLE, and if so then enable a bunch of features -- if NOT then log
>> a
>> > WARNing and don't enable those features.
>> >
>> > So what you're seeing here isn't an artifact/result of any particular
>> > choices "a test" or "the test" makes -- it's a concious choice of the
>> > developer who added this feature to solr. These WARN messages that
>> show
>> > up in tests where the solr home dir isn't writable (which is actaully
>> the
>> > vast majority of tests because of how the test framework works) are the
>> > same types of WARN messages that a "real" solr deployment might get if
>> > their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir}
>> to
>> > point to a diff drive).
>> >
>> >
>> >
>> > :
>> > : > On Aug 31, 2020, at 1:29 PM, Chris Hostetter <
>> hossman_lucene@fucit.org> wrote:
>> > : >
>> > : >
>> > : > Some tests "create" a new solr home dir and copy config files
>> there, but
>> > : > you'll see this type of WARN logging for any test that just uses
>> the test
>> > : > configs "in place" because of how the code is designed to _try_ and
>> create
>> > : > a userfiles directory in the solr home if it's writable.
>> > : >
>> > : >
>> > : > : Date: Sat, 29 Aug 2020 09:25:17 -0400
>> > : > : From: Erick Erickson <er...@gmail.com>
>> > : > : Reply-To: dev@lucene.apache.org
>> > : > : To: dev@lucene.apache.org
>> > : > : Subject: Re: Annoying but harmless exceptions due to
>> filepermissions when
>> > : > : running tests
>> > : > :
>> > : > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
>> > : > :
>> > : > : In CoreContainer [364] there’s code like this:
>> > : > :
>> > : > : Path userFilesPath = return solrHome.resolve("userfiles"); //
>> TODO make configurable on cfg?
>> > : > : try {
>> > : > : Files.createDirectories(userFilesPath); // does nothing if
>> already exists
>> > : > : } catch (Exception e) {
>> > : > : log.warn("Unable to create [{}]. Features requiring this
>> directory may fail.", userFilesPath, e);
>> > : > : }
>> > : > :
>> > : > : So I assumed it would happen on most every test, at least in
>> cloud mode. But when I tried to make it happen on a different test, there
>> was no exception.
>> > : > :
>> > : > : I’ll have to poke some more to see what’s really happening…
>> > : > :
>> > : > : Never Mind….
>> > : > :
>> > : > : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler <uw...@thetaphi.de>
>> wrote:
>> > : > : >
>> > : > : > Hi,
>> > : > : >
>> > : > : > this is a bug in the test! It should never ever modify files
>> outside its sandbox. It should not even modify files in build directory. It
>> is fully valid to reject those writes - has nothing to do with users, it's
>> just forbidden by the test framework. Modifying this file is harmful, as it
>> may affect later tests.
>> > : > : >
>> > : > : > So the correct way is to copy those files to the solr container
>> running inside test's sandbox. That's one of the main advantages of the
>> Test sandbox: No write access anywhere outside the test, see policy file.
>> > : > : >
>> > : > : > Uwe
>> > : > : >
>> > : > : > -----
>> > : > : > Uwe Schindler
>> > : > : > Achterdiek 19, D-28357 Bremen
>> > : > : > https://www.thetaphi.de
>> > : > : > eMail: uwe@thetaphi.de
>> > : > : >
>> > : > : >> -----Original Message-----
>> > : > : >> From: Erick Erickson <er...@gmail.com>
>> > : > : >> Sent: Saturday, August 29, 2020 2:54 PM
>> > : > : >> To: dev@lucene.apache.org
>> > : > : >> Subject: Annoying but harmless exceptions due to
>> filepermissions when running
>> > : > : >> tests
>> > : > : >>
>> > : > : >> These exceptions are handled in the code and don’t affect
>> running tests, but
>> > : > : >> they can be a distraction when trying to figure out what’s
>> causing a failure.
>> > : > : >> When CoreContainer is being initialized, these two paths get
>> “Permission
>> > : > : >> Denied” errors since they try to create directories/files.
>> > : > : >>
>> > : > : >> java.security.AccessControlException: access denied
>> ("java.io.FilePermission"
>> > : > : >>
>> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
>> > : > : >> ore" "write”)
>> > : > : >>
>> > : > : >> java.security.AccessControlException: access denied
>> ("java.io.FilePermission"
>> > : > : >>
>> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
>> > : > : >> iles" "write”)
>> > : > : >>
>> > : > : >> In both cases, the code logs a warning like "Features
>> requiring this directory
>> > : > : >> may fail”.
>> > : > : >>
>> > : > : >> “build” is permitted this way, so I guess gradle runs as some
>> other user?
>> > : > : >>
>> > : > : >> drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
>> > : > : >>
>> > : > : >> Any hints on an easy way to avoid these? It’s not worth much
>> effort I don’t
>> > : > : >> think since they’re handled, but if it’s easy I’d like to not
>> see them.
>> > : > : >>
>> > : > : >>
>> > : > : >>
>> > : > : >>
>> ---------------------------------------------------------------------
>> > : > : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > : > : >> For additional commands, e-mail: dev-help@lucene.apache.org
>> > : > : >
>> > : > : >
>> > : > : >
>> ---------------------------------------------------------------------
>> > : > : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > : > : > For additional commands, e-mail: dev-help@lucene.apache.org
>> > : > : >
>> > : > :
>> > : > :
>> > : > :
>> ---------------------------------------------------------------------
>> > : > : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > : > : For additional commands, e-mail: dev-help@lucene.apache.org
>> > : > :
>> > : > :
>> > : >
>> > : > -Hoss
>> > : > http://www.lucidworks.com/
>> > : >
>> > : >
>> ---------------------------------------------------------------------
>> > : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > : > For additional commands, e-mail: dev-help@lucene.apache.org
>> > :
>> > :
>> > : ---------------------------------------------------------------------
>> > : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > : For additional commands, e-mail: dev-help@lucene.apache.org
>> > :
>> > :
>> >
>> > -Hoss
>> > http://www.lucidworks.com/
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
Re: Annoying but harmless exceptions due to filepermissions when
running tests
Posted by Noble Paul <no...@gmail.com>.
The filestore dir is where the packages live. If we move it to another
location, existing installations might fail. So, it's a backward
incompatible change.
What are our options?
Is it possible to have these directories precreated in the distro/code to
ensure that these warnings don't come.
On Thu, Sep 3, 2020, 4:58 AM Erick Erickson <er...@gmail.com> wrote:
> Oh bother. Somehow I thought I’d looked and only a handful of tests
> reported this.
>
> So I looked again and I _wish_ I was able to blame drugs cause you’re
> right, there
> are over a thousand of them.
>
> Never mind...
>
> > On Sep 1, 2020, at 8:55 PM, Chris Hostetter <ho...@fucit.org>
> wrote:
> >
> >
> > : Hmmm, that’s kind of an dilemma then. Are you saying that
> > : he test can see that the directory appears writable then tries
> > : to write to it then gets tripped up by the framework?
> > :
> > : Seems to me that a test that tries to write, thinks it can and then
> > : can’t should fail anyway.
> > :
> > : Well, I don’t think there are very many tests that have this problem
> > : anyway, so maybe I can examine them one-by-one and not
> > : introduce new failures...
> >
> > You keep using the phrase "the test" in the context of (trying to)
> create
> > these directories ("userfiles" and "filestore") ... the "test CODE"
> isn't
> > making any choices about trying to write those files -- the choice is
> > being made by the "CoreContainer CODE".
> >
> > These features were added with the explicit implementation choice to
> _TRY_
> > to write the "usersfiles" (and/or "filestore") directory to "solr home"
> IF
> > POSSIBLE, and if so then enable a bunch of features -- if NOT then log a
> > WARNing and don't enable those features.
> >
> > So what you're seeing here isn't an artifact/result of any particular
> > choices "a test" or "the test" makes -- it's a concious choice of the
> > developer who added this feature to solr. These WARN messages that show
> > up in tests where the solr home dir isn't writable (which is actaully
> the
> > vast majority of tests because of how the test framework works) are the
> > same types of WARN messages that a "real" solr deployment might get if
> > their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir} to
> > point to a diff drive).
> >
> >
> >
> > :
> > : > On Aug 31, 2020, at 1:29 PM, Chris Hostetter <
> hossman_lucene@fucit.org> wrote:
> > : >
> > : >
> > : > Some tests "create" a new solr home dir and copy config files there,
> but
> > : > you'll see this type of WARN logging for any test that just uses the
> test
> > : > configs "in place" because of how the code is designed to _try_ and
> create
> > : > a userfiles directory in the solr home if it's writable.
> > : >
> > : >
> > : > : Date: Sat, 29 Aug 2020 09:25:17 -0400
> > : > : From: Erick Erickson <er...@gmail.com>
> > : > : Reply-To: dev@lucene.apache.org
> > : > : To: dev@lucene.apache.org
> > : > : Subject: Re: Annoying but harmless exceptions due to
> filepermissions when
> > : > : running tests
> > : > :
> > : > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
> > : > :
> > : > : In CoreContainer [364] there’s code like this:
> > : > :
> > : > : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO
> make configurable on cfg?
> > : > : try {
> > : > : Files.createDirectories(userFilesPath); // does nothing if
> already exists
> > : > : } catch (Exception e) {
> > : > : log.warn("Unable to create [{}]. Features requiring this
> directory may fail.", userFilesPath, e);
> > : > : }
> > : > :
> > : > : So I assumed it would happen on most every test, at least in cloud
> mode. But when I tried to make it happen on a different test, there was no
> exception.
> > : > :
> > : > : I’ll have to poke some more to see what’s really happening…
> > : > :
> > : > : Never Mind….
> > : > :
> > : > : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler <uw...@thetaphi.de>
> wrote:
> > : > : >
> > : > : > Hi,
> > : > : >
> > : > : > this is a bug in the test! It should never ever modify files
> outside its sandbox. It should not even modify files in build directory. It
> is fully valid to reject those writes - has nothing to do with users, it's
> just forbidden by the test framework. Modifying this file is harmful, as it
> may affect later tests.
> > : > : >
> > : > : > So the correct way is to copy those files to the solr container
> running inside test's sandbox. That's one of the main advantages of the
> Test sandbox: No write access anywhere outside the test, see policy file.
> > : > : >
> > : > : > Uwe
> > : > : >
> > : > : > -----
> > : > : > Uwe Schindler
> > : > : > Achterdiek 19, D-28357 Bremen
> > : > : > https://www.thetaphi.de
> > : > : > eMail: uwe@thetaphi.de
> > : > : >
> > : > : >> -----Original Message-----
> > : > : >> From: Erick Erickson <er...@gmail.com>
> > : > : >> Sent: Saturday, August 29, 2020 2:54 PM
> > : > : >> To: dev@lucene.apache.org
> > : > : >> Subject: Annoying but harmless exceptions due to
> filepermissions when running
> > : > : >> tests
> > : > : >>
> > : > : >> These exceptions are handled in the code and don’t affect
> running tests, but
> > : > : >> they can be a distraction when trying to figure out what’s
> causing a failure.
> > : > : >> When CoreContainer is being initialized, these two paths get
> “Permission
> > : > : >> Denied” errors since they try to create directories/files.
> > : > : >>
> > : > : >> java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> > : > : >>
> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
> > : > : >> ore" "write”)
> > : > : >>
> > : > : >> java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> > : > : >>
> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
> > : > : >> iles" "write”)
> > : > : >>
> > : > : >> In both cases, the code logs a warning like "Features requiring
> this directory
> > : > : >> may fail”.
> > : > : >>
> > : > : >> “build” is permitted this way, so I guess gradle runs as some
> other user?
> > : > : >>
> > : > : >> drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
> > : > : >>
> > : > : >> Any hints on an easy way to avoid these? It’s not worth much
> effort I don’t
> > : > : >> think since they’re handled, but if it’s easy I’d like to not
> see them.
> > : > : >>
> > : > : >>
> > : > : >>
> > : > : >>
> ---------------------------------------------------------------------
> > : > : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : > : >> For additional commands, e-mail: dev-help@lucene.apache.org
> > : > : >
> > : > : >
> > : > : >
> ---------------------------------------------------------------------
> > : > : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : > : > For additional commands, e-mail: dev-help@lucene.apache.org
> > : > : >
> > : > :
> > : > :
> > : > :
> ---------------------------------------------------------------------
> > : > : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : > : For additional commands, e-mail: dev-help@lucene.apache.org
> > : > :
> > : > :
> > : >
> > : > -Hoss
> > : > http://www.lucidworks.com/
> > : >
> > : > ---------------------------------------------------------------------
> > : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : > For additional commands, e-mail: dev-help@lucene.apache.org
> > :
> > :
> > : ---------------------------------------------------------------------
> > : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > : For additional commands, e-mail: dev-help@lucene.apache.org
> > :
> > :
> >
> > -Hoss
> > http://www.lucidworks.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Annoying but harmless exceptions due to filepermissions when
running tests
Posted by Erick Erickson <er...@gmail.com>.
Oh bother. Somehow I thought I’d looked and only a handful of tests reported this.
So I looked again and I _wish_ I was able to blame drugs cause you’re right, there
are over a thousand of them.
Never mind...
> On Sep 1, 2020, at 8:55 PM, Chris Hostetter <ho...@fucit.org> wrote:
>
>
> : Hmmm, that’s kind of an dilemma then. Are you saying that
> : he test can see that the directory appears writable then tries
> : to write to it then gets tripped up by the framework?
> :
> : Seems to me that a test that tries to write, thinks it can and then
> : can’t should fail anyway.
> :
> : Well, I don’t think there are very many tests that have this problem
> : anyway, so maybe I can examine them one-by-one and not
> : introduce new failures...
>
> You keep using the phrase "the test" in the context of (trying to) create
> these directories ("userfiles" and "filestore") ... the "test CODE" isn't
> making any choices about trying to write those files -- the choice is
> being made by the "CoreContainer CODE".
>
> These features were added with the explicit implementation choice to _TRY_
> to write the "usersfiles" (and/or "filestore") directory to "solr home" IF
> POSSIBLE, and if so then enable a bunch of features -- if NOT then log a
> WARNing and don't enable those features.
>
> So what you're seeing here isn't an artifact/result of any particular
> choices "a test" or "the test" makes -- it's a concious choice of the
> developer who added this feature to solr. These WARN messages that show
> up in tests where the solr home dir isn't writable (which is actaully the
> vast majority of tests because of how the test framework works) are the
> same types of WARN messages that a "real" solr deployment might get if
> their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir} to
> point to a diff drive).
>
>
>
> :
> : > On Aug 31, 2020, at 1:29 PM, Chris Hostetter <ho...@fucit.org> wrote:
> : >
> : >
> : > Some tests "create" a new solr home dir and copy config files there, but
> : > you'll see this type of WARN logging for any test that just uses the test
> : > configs "in place" because of how the code is designed to _try_ and create
> : > a userfiles directory in the solr home if it's writable.
> : >
> : >
> : > : Date: Sat, 29 Aug 2020 09:25:17 -0400
> : > : From: Erick Erickson <er...@gmail.com>
> : > : Reply-To: dev@lucene.apache.org
> : > : To: dev@lucene.apache.org
> : > : Subject: Re: Annoying but harmless exceptions due to filepermissions when
> : > : running tests
> : > :
> : > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
> : > :
> : > : In CoreContainer [364] there’s code like this:
> : > :
> : > : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make configurable on cfg?
> : > : try {
> : > : Files.createDirectories(userFilesPath); // does nothing if already exists
> : > : } catch (Exception e) {
> : > : log.warn("Unable to create [{}]. Features requiring this directory may fail.", userFilesPath, e);
> : > : }
> : > :
> : > : So I assumed it would happen on most every test, at least in cloud mode. But when I tried to make it happen on a different test, there was no exception.
> : > :
> : > : I’ll have to poke some more to see what’s really happening…
> : > :
> : > : Never Mind….
> : > :
> : > : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> : > : >
> : > : > Hi,
> : > : >
> : > : > this is a bug in the test! It should never ever modify files outside its sandbox. It should not even modify files in build directory. It is fully valid to reject those writes - has nothing to do with users, it's just forbidden by the test framework. Modifying this file is harmful, as it may affect later tests.
> : > : >
> : > : > So the correct way is to copy those files to the solr container running inside test's sandbox. That's one of the main advantages of the Test sandbox: No write access anywhere outside the test, see policy file.
> : > : >
> : > : > Uwe
> : > : >
> : > : > -----
> : > : > Uwe Schindler
> : > : > Achterdiek 19, D-28357 Bremen
> : > : > https://www.thetaphi.de
> : > : > eMail: uwe@thetaphi.de
> : > : >
> : > : >> -----Original Message-----
> : > : >> From: Erick Erickson <er...@gmail.com>
> : > : >> Sent: Saturday, August 29, 2020 2:54 PM
> : > : >> To: dev@lucene.apache.org
> : > : >> Subject: Annoying but harmless exceptions due to filepermissions when running
> : > : >> tests
> : > : >>
> : > : >> These exceptions are handled in the code and don’t affect running tests, but
> : > : >> they can be a distraction when trying to figure out what’s causing a failure.
> : > : >> When CoreContainer is being initialized, these two paths get “Permission
> : > : >> Denied” errors since they try to create directories/files.
> : > : >>
> : > : >> java.security.AccessControlException: access denied ("java.io.FilePermission"
> : > : >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
> : > : >> ore" "write”)
> : > : >>
> : > : >> java.security.AccessControlException: access denied ("java.io.FilePermission"
> : > : >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
> : > : >> iles" "write”)
> : > : >>
> : > : >> In both cases, the code logs a warning like "Features requiring this directory
> : > : >> may fail”.
> : > : >>
> : > : >> “build” is permitted this way, so I guess gradle runs as some other user?
> : > : >>
> : > : >> drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
> : > : >>
> : > : >> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
> : > : >> think since they’re handled, but if it’s easy I’d like to not see them.
> : > : >>
> : > : >>
> : > : >>
> : > : >> ---------------------------------------------------------------------
> : > : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : > : >> For additional commands, e-mail: dev-help@lucene.apache.org
> : > : >
> : > : >
> : > : > ---------------------------------------------------------------------
> : > : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : > : > For additional commands, e-mail: dev-help@lucene.apache.org
> : > : >
> : > :
> : > :
> : > : ---------------------------------------------------------------------
> : > : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : > : For additional commands, e-mail: dev-help@lucene.apache.org
> : > :
> : > :
> : >
> : > -Hoss
> : > http://www.lucidworks.com/
> : >
> : > ---------------------------------------------------------------------
> : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : > For additional commands, e-mail: dev-help@lucene.apache.org
> :
> :
> : ---------------------------------------------------------------------
> : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : For additional commands, e-mail: dev-help@lucene.apache.org
> :
> :
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Annoying but harmless exceptions due to filepermissions when
running tests
Posted by Chris Hostetter <ho...@fucit.org>.
: Hmmm, that’s kind of an dilemma then. Are you saying that
: he test can see that the directory appears writable then tries
: to write to it then gets tripped up by the framework?
:
: Seems to me that a test that tries to write, thinks it can and then
: can’t should fail anyway.
:
: Well, I don’t think there are very many tests that have this problem
: anyway, so maybe I can examine them one-by-one and not
: introduce new failures...
You keep using the phrase "the test" in the context of (trying to) create
these directories ("userfiles" and "filestore") ... the "test CODE" isn't
making any choices about trying to write those files -- the choice is
being made by the "CoreContainer CODE".
These features were added with the explicit implementation choice to _TRY_
to write the "usersfiles" (and/or "filestore") directory to "solr home" IF
POSSIBLE, and if so then enable a bunch of features -- if NOT then log a
WARNing and don't enable those features.
So what you're seeing here isn't an artifact/result of any particular
choices "a test" or "the test" makes -- it's a concious choice of the
developer who added this feature to solr. These WARN messages that show
up in tests where the solr home dir isn't writable (which is actaully the
vast majority of tests because of how the test framework works) are the
same types of WARN messages that a "real" solr deployment might get if
their solr home dir isn't wriable (ie: maybe the use ${solr.data.dir} to
point to a diff drive).
:
: > On Aug 31, 2020, at 1:29 PM, Chris Hostetter <ho...@fucit.org> wrote:
: >
: >
: > Some tests "create" a new solr home dir and copy config files there, but
: > you'll see this type of WARN logging for any test that just uses the test
: > configs "in place" because of how the code is designed to _try_ and create
: > a userfiles directory in the solr home if it's writable.
: >
: >
: > : Date: Sat, 29 Aug 2020 09:25:17 -0400
: > : From: Erick Erickson <er...@gmail.com>
: > : Reply-To: dev@lucene.apache.org
: > : To: dev@lucene.apache.org
: > : Subject: Re: Annoying but harmless exceptions due to filepermissions when
: > : running tests
: > :
: > : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
: > :
: > : In CoreContainer [364] there’s code like this:
: > :
: > : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make configurable on cfg?
: > : try {
: > : Files.createDirectories(userFilesPath); // does nothing if already exists
: > : } catch (Exception e) {
: > : log.warn("Unable to create [{}]. Features requiring this directory may fail.", userFilesPath, e);
: > : }
: > :
: > : So I assumed it would happen on most every test, at least in cloud mode. But when I tried to make it happen on a different test, there was no exception.
: > :
: > : I’ll have to poke some more to see what’s really happening…
: > :
: > : Never Mind….
: > :
: > : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
: > : >
: > : > Hi,
: > : >
: > : > this is a bug in the test! It should never ever modify files outside its sandbox. It should not even modify files in build directory. It is fully valid to reject those writes - has nothing to do with users, it's just forbidden by the test framework. Modifying this file is harmful, as it may affect later tests.
: > : >
: > : > So the correct way is to copy those files to the solr container running inside test's sandbox. That's one of the main advantages of the Test sandbox: No write access anywhere outside the test, see policy file.
: > : >
: > : > Uwe
: > : >
: > : > -----
: > : > Uwe Schindler
: > : > Achterdiek 19, D-28357 Bremen
: > : > https://www.thetaphi.de
: > : > eMail: uwe@thetaphi.de
: > : >
: > : >> -----Original Message-----
: > : >> From: Erick Erickson <er...@gmail.com>
: > : >> Sent: Saturday, August 29, 2020 2:54 PM
: > : >> To: dev@lucene.apache.org
: > : >> Subject: Annoying but harmless exceptions due to filepermissions when running
: > : >> tests
: > : >>
: > : >> These exceptions are handled in the code and don’t affect running tests, but
: > : >> they can be a distraction when trying to figure out what’s causing a failure.
: > : >> When CoreContainer is being initialized, these two paths get “Permission
: > : >> Denied” errors since they try to create directories/files.
: > : >>
: > : >> java.security.AccessControlException: access denied ("java.io.FilePermission"
: > : >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
: > : >> ore" "write”)
: > : >>
: > : >> java.security.AccessControlException: access denied ("java.io.FilePermission"
: > : >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
: > : >> iles" "write”)
: > : >>
: > : >> In both cases, the code logs a warning like "Features requiring this directory
: > : >> may fail”.
: > : >>
: > : >> “build” is permitted this way, so I guess gradle runs as some other user?
: > : >>
: > : >> drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
: > : >>
: > : >> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
: > : >> think since they’re handled, but if it’s easy I’d like to not see them.
: > : >>
: > : >>
: > : >>
: > : >> ---------------------------------------------------------------------
: > : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > : >> For additional commands, e-mail: dev-help@lucene.apache.org
: > : >
: > : >
: > : > ---------------------------------------------------------------------
: > : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > : > For additional commands, e-mail: dev-help@lucene.apache.org
: > : >
: > :
: > :
: > : ---------------------------------------------------------------------
: > : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > : For additional commands, e-mail: dev-help@lucene.apache.org
: > :
: > :
: >
: > -Hoss
: > http://www.lucidworks.com/
: >
: > ---------------------------------------------------------------------
: > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > For additional commands, e-mail: dev-help@lucene.apache.org
:
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: For additional commands, e-mail: dev-help@lucene.apache.org
:
:
-Hoss
http://www.lucidworks.com/
Re: Annoying but harmless exceptions due to filepermissions when
running tests
Posted by Erick Erickson <er...@gmail.com>.
Hmmm, that’s kind of an dilemma then. Are you saying that
he test can see that the directory appears writable then tries
to write to it then gets tripped up by the framework?
Seems to me that a test that tries to write, thinks it can and then
can’t should fail anyway.
Well, I don’t think there are very many tests that have this problem
anyway, so maybe I can examine them one-by-one and not
introduce new failures...
> On Aug 31, 2020, at 1:29 PM, Chris Hostetter <ho...@fucit.org> wrote:
>
>
> Some tests "create" a new solr home dir and copy config files there, but
> you'll see this type of WARN logging for any test that just uses the test
> configs "in place" because of how the code is designed to _try_ and create
> a userfiles directory in the solr home if it's writable.
>
>
> : Date: Sat, 29 Aug 2020 09:25:17 -0400
> : From: Erick Erickson <er...@gmail.com>
> : Reply-To: dev@lucene.apache.org
> : To: dev@lucene.apache.org
> : Subject: Re: Annoying but harmless exceptions due to filepermissions when
> : running tests
> :
> : Well, as Uwe and I discussed offline, he’s right and I’m wrong.
> :
> : In CoreContainer [364] there’s code like this:
> :
> : Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make configurable on cfg?
> : try {
> : Files.createDirectories(userFilesPath); // does nothing if already exists
> : } catch (Exception e) {
> : log.warn("Unable to create [{}]. Features requiring this directory may fail.", userFilesPath, e);
> : }
> :
> : So I assumed it would happen on most every test, at least in cloud mode. But when I tried to make it happen on a different test, there was no exception.
> :
> : I’ll have to poke some more to see what’s really happening…
> :
> : Never Mind….
> :
> : > On Aug 29, 2020, at 8:59 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> : >
> : > Hi,
> : >
> : > this is a bug in the test! It should never ever modify files outside its sandbox. It should not even modify files in build directory. It is fully valid to reject those writes - has nothing to do with users, it's just forbidden by the test framework. Modifying this file is harmful, as it may affect later tests.
> : >
> : > So the correct way is to copy those files to the solr container running inside test's sandbox. That's one of the main advantages of the Test sandbox: No write access anywhere outside the test, see policy file.
> : >
> : > Uwe
> : >
> : > -----
> : > Uwe Schindler
> : > Achterdiek 19, D-28357 Bremen
> : > https://www.thetaphi.de
> : > eMail: uwe@thetaphi.de
> : >
> : >> -----Original Message-----
> : >> From: Erick Erickson <er...@gmail.com>
> : >> Sent: Saturday, August 29, 2020 2:54 PM
> : >> To: dev@lucene.apache.org
> : >> Subject: Annoying but harmless exceptions due to filepermissions when running
> : >> tests
> : >>
> : >> These exceptions are handled in the code and don’t affect running tests, but
> : >> they can be a distraction when trying to figure out what’s causing a failure.
> : >> When CoreContainer is being initialized, these two paths get “Permission
> : >> Denied” errors since they try to create directories/files.
> : >>
> : >> java.security.AccessControlException: access denied ("java.io.FilePermission"
> : >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
> : >> ore" "write”)
> : >>
> : >> java.security.AccessControlException: access denied ("java.io.FilePermission"
> : >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
> : >> iles" "write”)
> : >>
> : >> In both cases, the code logs a warning like "Features requiring this directory
> : >> may fail”.
> : >>
> : >> “build” is permitted this way, so I guess gradle runs as some other user?
> : >>
> : >> drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
> : >>
> : >> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
> : >> think since they’re handled, but if it’s easy I’d like to not see them.
> : >>
> : >>
> : >>
> : >> ---------------------------------------------------------------------
> : >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : >> For additional commands, e-mail: dev-help@lucene.apache.org
> : >
> : >
> : > ---------------------------------------------------------------------
> : > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : > For additional commands, e-mail: dev-help@lucene.apache.org
> : >
> :
> :
> : ---------------------------------------------------------------------
> : To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> : For additional commands, e-mail: dev-help@lucene.apache.org
> :
> :
>
> -Hoss
> http://www.lucidworks.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Annoying but harmless exceptions due to filepermissions when
running tests
Posted by Chris Hostetter <ho...@fucit.org>.
Some tests "create" a new solr home dir and copy config files there, but
you'll see this type of WARN logging for any test that just uses the test
configs "in place" because of how the code is designed to _try_ and create
a userfiles directory in the solr home if it's writable.
: Date: Sat, 29 Aug 2020 09:25:17 -0400
: From: Erick Erickson <er...@gmail.com>
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: Re: Annoying but harmless exceptions due to filepermissions when
: running tests
:
: Well, as Uwe and I discussed offline, he’s right and I’m wrong.
:
: In CoreContainer [364] there’s code like this:
:
: Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make configurable on cfg?
: try {
: Files.createDirectories(userFilesPath); // does nothing if already exists
: } catch (Exception e) {
: log.warn("Unable to create [{}]. Features requiring this directory may fail.", userFilesPath, e);
: }
:
: So I assumed it would happen on most every test, at least in cloud mode. But when I tried to make it happen on a different test, there was no exception.
:
: I’ll have to poke some more to see what’s really happening…
:
: Never Mind….
:
: > On Aug 29, 2020, at 8:59 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
: >
: > Hi,
: >
: > this is a bug in the test! It should never ever modify files outside its sandbox. It should not even modify files in build directory. It is fully valid to reject those writes - has nothing to do with users, it's just forbidden by the test framework. Modifying this file is harmful, as it may affect later tests.
: >
: > So the correct way is to copy those files to the solr container running inside test's sandbox. That's one of the main advantages of the Test sandbox: No write access anywhere outside the test, see policy file.
: >
: > Uwe
: >
: > -----
: > Uwe Schindler
: > Achterdiek 19, D-28357 Bremen
: > https://www.thetaphi.de
: > eMail: uwe@thetaphi.de
: >
: >> -----Original Message-----
: >> From: Erick Erickson <er...@gmail.com>
: >> Sent: Saturday, August 29, 2020 2:54 PM
: >> To: dev@lucene.apache.org
: >> Subject: Annoying but harmless exceptions due to filepermissions when running
: >> tests
: >>
: >> These exceptions are handled in the code and don’t affect running tests, but
: >> they can be a distraction when trying to figure out what’s causing a failure.
: >> When CoreContainer is being initialized, these two paths get “Permission
: >> Denied” errors since they try to create directories/files.
: >>
: >> java.security.AccessControlException: access denied ("java.io.FilePermission"
: >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
: >> ore" "write”)
: >>
: >> java.security.AccessControlException: access denied ("java.io.FilePermission"
: >> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
: >> iles" "write”)
: >>
: >> In both cases, the code logs a warning like "Features requiring this directory
: >> may fail”.
: >>
: >> “build” is permitted this way, so I guess gradle runs as some other user?
: >>
: >> drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
: >>
: >> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
: >> think since they’re handled, but if it’s easy I’d like to not see them.
: >>
: >>
: >>
: >> ---------------------------------------------------------------------
: >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: >> For additional commands, e-mail: dev-help@lucene.apache.org
: >
: >
: > ---------------------------------------------------------------------
: > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > For additional commands, e-mail: dev-help@lucene.apache.org
: >
:
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: For additional commands, e-mail: dev-help@lucene.apache.org
:
:
-Hoss
http://www.lucidworks.com/
Re: Annoying but harmless exceptions due to filepermissions when
running tests
Posted by Erick Erickson <er...@gmail.com>.
Well, as Uwe and I discussed offline, he’s right and I’m wrong.
In CoreContainer [364] there’s code like this:
Path userFilesPath = return solrHome.resolve("userfiles"); // TODO make configurable on cfg?
try {
Files.createDirectories(userFilesPath); // does nothing if already exists
} catch (Exception e) {
log.warn("Unable to create [{}]. Features requiring this directory may fail.", userFilesPath, e);
}
So I assumed it would happen on most every test, at least in cloud mode. But when I tried to make it happen on a different test, there was no exception.
I’ll have to poke some more to see what’s really happening…
Never Mind….
> On Aug 29, 2020, at 8:59 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
>
> Hi,
>
> this is a bug in the test! It should never ever modify files outside its sandbox. It should not even modify files in build directory. It is fully valid to reject those writes - has nothing to do with users, it's just forbidden by the test framework. Modifying this file is harmful, as it may affect later tests.
>
> So the correct way is to copy those files to the solr container running inside test's sandbox. That's one of the main advantages of the Test sandbox: No write access anywhere outside the test, see policy file.
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>> -----Original Message-----
>> From: Erick Erickson <er...@gmail.com>
>> Sent: Saturday, August 29, 2020 2:54 PM
>> To: dev@lucene.apache.org
>> Subject: Annoying but harmless exceptions due to filepermissions when running
>> tests
>>
>> These exceptions are handled in the code and don’t affect running tests, but
>> they can be a distraction when trying to figure out what’s causing a failure.
>> When CoreContainer is being initialized, these two paths get “Permission
>> Denied” errors since they try to create directories/files.
>>
>> java.security.AccessControlException: access denied ("java.io.FilePermission"
>> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
>> ore" "write”)
>>
>> java.security.AccessControlException: access denied ("java.io.FilePermission"
>> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
>> iles" "write”)
>>
>> In both cases, the code logs a warning like "Features requiring this directory
>> may fail”.
>>
>> “build” is permitted this way, so I guess gradle runs as some other user?
>>
>> drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
>>
>> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
>> think since they’re handled, but if it’s easy I’d like to not see them.
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
RE: Annoying but harmless exceptions due to filepermissions when running tests
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,
this is a bug in the test! It should never ever modify files outside its sandbox. It should not even modify files in build directory. It is fully valid to reject those writes - has nothing to do with users, it's just forbidden by the test framework. Modifying this file is harmful, as it may affect later tests.
So the correct way is to copy those files to the solr container running inside test's sandbox. That's one of the main advantages of the Test sandbox: No write access anywhere outside the test, see policy file.
Uwe
-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Erick Erickson <er...@gmail.com>
> Sent: Saturday, August 29, 2020 2:54 PM
> To: dev@lucene.apache.org
> Subject: Annoying but harmless exceptions due to filepermissions when running
> tests
>
> These exceptions are handled in the code and don’t affect running tests, but
> they can be a distraction when trying to figure out what’s causing a failure.
> When CoreContainer is being initialized, these two paths get “Permission
> Denied” errors since they try to create directories/files.
>
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/filest
> ore" "write”)
>
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/Users/Erick/apache/solrJiras/master/solr/core/build/resources/test/solr/userf
> iles" "write”)
>
> In both cases, the code logs a warning like "Features requiring this directory
> may fail”.
>
> “build” is permitted this way, so I guess gradle runs as some other user?
>
> drwxr-xr-x 11 Erick staff 352 Aug 28 09:30 build
>
> Any hints on an easy way to avoid these? It’s not worth much effort I don’t
> think since they’re handled, but if it’s easy I’d like to not see them.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org