You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Tim Ellison <t....@gmail.com> on 2006/03/16 16:37:49 UTC
Re: Tainting testing environment
Stepan Mishura wrote:
> Hi Tim,
>
> I've reviewed my update and found I did a mistake - I occasionally
> substituted in two places Security.setProperty with System.setProperty (Of
> course I'll provide a patch to fix this). But this slip made me thinks how
> to solve restoring testing environment in case of security properties.
>
> I think that nobody will object this is a good style for tests to clean up
> testing environment in the end of testing (I mean close all opened
> connections, remove created files, restore changed environment properties
> and so on). This helps to avoid conflicts between tests, when one test
> influences on results of another test.
>
> java.lang.System class provides a ways to set and unset system properties -
> it has corresponding API. But java.security.Security doesn't have such API
> to unset security properties. For example, if security property say 'a.b.c'
> was not set in environment (i.e. has null value) and a test during execution
> define this property then it is not possible to unset it (assign null value)
> because Security.setProperty(String key, String datum) throws NPE in case of
> null 'datum'. Setting value to empty string doesn't looks good. Are there
> any other solutions?
The two things that spring to mind are:
- create an internal API helper that we can inject on the bootclasspath
(or put in a bundle fragment) that will allow the test case to unset the
property. Of course, this will make the entire test Harmony-dependent
so it will have to live with 'implementation' tests rather than 'api' tests.
- move the test to a suite of tests that fork the VM on each test
invocation so you get a fresh environment every time (and therefore
don't have to tidy-up). This will make the test a pure API test, but
will have performance overhead.
Regards,
Tim
> Thanks,
> Stepan Mishura
> Intel Middleware Products Division
>
>> -----Original Message-----
>> From: Tim Ellison (JIRA) [mailto:jira@apache.org]
>> Sent: Wednesday, March 15, 2006 6:27 PM
>> To: harmony-commits@incubator.apache.org
>> Subject: [jira] Resolved: (HARMONY-200) 2 security tests must correctly
>> restore environment
>>
>> [ http://issues.apache.org/jira/browse/HARMONY-200?page=all ]
>>
>> Tim Ellison resolved HARMONY-200:
>> ---------------------------------
>>
>> Resolution: Fixed
>>
>> Stepan,
>>
>> Thanks for the patch, applied to SECURITY module tests at repo revision
>> 386061.
>>
>> Please check that the patch was applied as you expected.
>>
>>
>>> 2 security tests must correctly restore environment
>>> ---------------------------------------------------
>>>
>>> Key: HARMONY-200
>>> URL: http://issues.apache.org/jira/browse/HARMONY-200
>>> Project: Harmony
>>> Type: Bug
>>> Components: Classlib
>>> Reporter: Stepan Mishura
>>> Assignee: Tim Ellison
>>> Priority: Minor
>>> Attachments: fixHarmony200.txt
>>>
>>> The following security tests set different system properties during
>> execution:
>>> javax/security/auth/PolicyTest.java
>>>
>> org/apache/harmony/security/x/security/auth/login/DefaultConfigurationTest.
>> java
>>> To avoid conflicts pervious values of system properties must be correctly
>> restored by tests at the end of testing.
>>
>> --
>> This message is automatically generated by JIRA.
>> -
>> If you think it was sent incorrectly contact one of the administrators:
>> http://issues.apache.org/jira/secure/Administrators.jspa
>> -
>> For more information on JIRA, see:
>> http://www.atlassian.com/software/jira
>
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
Re: Tainting testing environment
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Stepan Mishura wrote:
> On 3/19/06, Geir Magnusson Jr wrote:
> <snip>
>
>>> - move the test to a suite of tests that fork the VM on each test
>>> invocation so you get a fresh environment every time (and therefore
>>> don't have to tidy-up). This will make the test a pure API test, but
>>> will have performance overhead.
>> I'd vote for the second option. Much cleaner and easy to understand -
>> therefore to maintain.
>
>
> Well, I can add that first option doesn't allow running tests in parallel
> (for example, if we switch to TestNG) because they share the same
> environment (BTW, this also relates to system properties). So the second
> option is more preferable way to solve the problem but ... how many
> different test suites we are going to create?
Who knows.
>
> What if I want to run a test in environment with all security properties
> unset? Should I create a new test suite to fork the VM with -
> Djava.security.properties==<empty file>? And what if I want to unset only
> selected security properties? Another test suite with custom security
> properties file? ... It doesn't look for me as elegant and lightweight
> solution and this makes me feel a little depressed.
As elegant and lightweight as what?
Have the abillity to neatly define the configuration scenarious you
suggest sounds ince.
geir
Re: Tainting testing environment
Posted by Stepan Mishura <st...@gmail.com>.
On 3/19/06, Geir Magnusson Jr wrote:
<snip>
> > - move the test to a suite of tests that fork the VM on each test
> > invocation so you get a fresh environment every time (and therefore
> > don't have to tidy-up). This will make the test a pure API test, but
> > will have performance overhead.
>
> I'd vote for the second option. Much cleaner and easy to understand -
> therefore to maintain.
Well, I can add that first option doesn't allow running tests in parallel
(for example, if we switch to TestNG) because they share the same
environment (BTW, this also relates to system properties). So the second
option is more preferable way to solve the problem but ... how many
different test suites we are going to create?
What if I want to run a test in environment with all security properties
unset? Should I create a new test suite to fork the VM with -
Djava.security.properties==<empty file>? And what if I want to unset only
selected security properties? Another test suite with custom security
properties file? ... It doesn't look for me as elegant and lightweight
solution and this makes me feel a little depressed.
Ok. I'll try to find other solutions and if there are no alternatives we'll
use this approach.
Thanks,
Stepan.
Do we really care about some subset of tests being sub-optimal? I
>
> Also - I'd like to see us do a "fresh fork" test run from time to time
> just to convince ourselves that there are no side effects that are
> making things pass...
>
> geir
>
> >
> > Regards,
> > Tim
> >
> >
> >> Thanks,
> >> Stepan Mishura
> >> Intel Middleware Products Division
> >>
> >>> -----Original Message-----
> >>> From: Tim Ellison (JIRA) [mailto:jira@apache.org]
> >>> Sent: Wednesday, March 15, 2006 6:27 PM
> >>> To: harmony-commits@incubator.apache.org
> >>> Subject: [jira] Resolved: (HARMONY-200) 2 security tests must
> correctly
> >>> restore environment
> >>>
> >>> [ http://issues.apache.org/jira/browse/HARMONY-200?page=all ]
> >>>
> >>> Tim Ellison resolved HARMONY-200:
> >>> ---------------------------------
> >>>
> >>> Resolution: Fixed
> >>>
> >>> Stepan,
> >>>
> >>> Thanks for the patch, applied to SECURITY module tests at repo
> revision
> >>> 386061.
> >>>
> >>> Please check that the patch was applied as you expected.
> >>>
> >>>
> >>>> 2 security tests must correctly restore environment
> >>>> ---------------------------------------------------
> >>>>
> >>>> Key: HARMONY-200
> >>>> URL: http://issues.apache.org/jira/browse/HARMONY-200
> >>>> Project: Harmony
> >>>> Type: Bug
> >>>> Components: Classlib
> >>>> Reporter: Stepan Mishura
> >>>> Assignee: Tim Ellison
> >>>> Priority: Minor
> >>>> Attachments: fixHarmony200.txt
> >>>>
> >>>> The following security tests set different system properties during
> >>> execution:
> >>>> javax/security/auth/PolicyTest.java
> >>>>
> >>>
> org/apache/harmony/security/x/security/auth/login/DefaultConfigurationTest.
> >>> java
> >>>> To avoid conflicts pervious values of system properties must be
> correctly
> >>> restored by tests at the end of testing.
> >>>
> >>> --
> >>> This message is automatically generated by JIRA.
> >>> -
> >>> If you think it was sent incorrectly contact one of the
> administrators:
> >>> http://issues.apache.org/jira/secure/Administrators.jspa
> >>> -
> >>> For more information on JIRA, see:
> >>> http://www.atlassian.com/software/jira
> >
>
--
Thanks,
Stepan Mishura
Intel Middleware Products Division
Re: Tainting testing environment
Posted by Geir Magnusson Jr <ge...@pobox.com>.
Tim Ellison wrote:
> Stepan Mishura wrote:
[SNIP]
>
> The two things that spring to mind are:
>
> - create an internal API helper that we can inject on the bootclasspath
> (or put in a bundle fragment) that will allow the test case to unset the
> property. Of course, this will make the entire test Harmony-dependent
> so it will have to live with 'implementation' tests rather than 'api' tests.
>
> - move the test to a suite of tests that fork the VM on each test
> invocation so you get a fresh environment every time (and therefore
> don't have to tidy-up). This will make the test a pure API test, but
> will have performance overhead.
I'd vote for the second option. Much cleaner and easy to understand -
therefore to maintain.
Do we really care about some subset of tests being sub-optimal? I
Also - I'd like to see us do a "fresh fork" test run from time to time
just to convince ourselves that there are no side effects that are
making things pass...
geir
>
> Regards,
> Tim
>
>
>> Thanks,
>> Stepan Mishura
>> Intel Middleware Products Division
>>
>>> -----Original Message-----
>>> From: Tim Ellison (JIRA) [mailto:jira@apache.org]
>>> Sent: Wednesday, March 15, 2006 6:27 PM
>>> To: harmony-commits@incubator.apache.org
>>> Subject: [jira] Resolved: (HARMONY-200) 2 security tests must correctly
>>> restore environment
>>>
>>> [ http://issues.apache.org/jira/browse/HARMONY-200?page=all ]
>>>
>>> Tim Ellison resolved HARMONY-200:
>>> ---------------------------------
>>>
>>> Resolution: Fixed
>>>
>>> Stepan,
>>>
>>> Thanks for the patch, applied to SECURITY module tests at repo revision
>>> 386061.
>>>
>>> Please check that the patch was applied as you expected.
>>>
>>>
>>>> 2 security tests must correctly restore environment
>>>> ---------------------------------------------------
>>>>
>>>> Key: HARMONY-200
>>>> URL: http://issues.apache.org/jira/browse/HARMONY-200
>>>> Project: Harmony
>>>> Type: Bug
>>>> Components: Classlib
>>>> Reporter: Stepan Mishura
>>>> Assignee: Tim Ellison
>>>> Priority: Minor
>>>> Attachments: fixHarmony200.txt
>>>>
>>>> The following security tests set different system properties during
>>> execution:
>>>> javax/security/auth/PolicyTest.java
>>>>
>>> org/apache/harmony/security/x/security/auth/login/DefaultConfigurationTest.
>>> java
>>>> To avoid conflicts pervious values of system properties must be correctly
>>> restored by tests at the end of testing.
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> -
>>> If you think it was sent incorrectly contact one of the administrators:
>>> http://issues.apache.org/jira/secure/Administrators.jspa
>>> -
>>> For more information on JIRA, see:
>>> http://www.atlassian.com/software/jira
>