You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Stefan Seifert <ss...@pro-vision.de> on 2016/11/28 10:40:45 UTC
integration test fail due to LoginAdminWhitelist
for some time the caconfig integration tests are failing because the org.apache.sling.junit.core is not whitelisted in LoginAdminWhitelist (see [1]).
but it is whitelisted via regexp (using the same merge-fix as robert for the launchpad tests - see [2]).
the resulting slingstart.txt in the build does contain the correct whitelist information see [3]
on my local machine the integration tests are running fine. on Jenkins they fail. has anyone an idea what is going wrong? can someone reproduce the failure locally?
stefan
[1] https://builds.apache.org/job/sling-contrib-extensions-contextaware-config-integration-tests-1.8/201/console
[2] https://github.com/apache/sling/blob/trunk/contrib/extensions/contextaware-config/integration-tests/src/main/provisioning/sling.txt
[3] https://builds.apache.org/job/sling-contrib-extensions-contextaware-config-integration-tests-1.8/ws/target/slingstart.txt
Re: integration test fail due to LoginAdminWhitelist
Posted by Julian Sedding <js...@gmail.com>.
Sorry, forget the last bit about tests passing. I fooled my self when
I only built the "package" phase.
The tests also don't pass with regexp=junit.
Regards
Julian
On Mon, Nov 28, 2016 at 12:36 PM, Julian Sedding <js...@gmail.com> wrote:
> Hi Stefan
>
> I just had a look and noticed the following:
>
> Your current configuration looks like this:
> whitelist.bundles.regexp="^org\.apache\.sling\.junit(\..*)?$"
>
> Inside target/slingstart.txt this becomes:
> whitelist.bundles.regexp="^org.apache.sling.junit(..*)?$"
>
> Note: all slashes have disappeared.
>
> When I escape the slashes, i.e.
> whitelist.bundles.regexp="^org\\.apache\\.sling\\.junit(\\..*)?$"
>
> target/slingstart.txt remains the same:
> whitelist.bundles.regexp="^org\\.apache\\.sling\\.junit(\\..*)?$"
>
> I believe there may be a bug in the escaping of provisioning models.
> Unless someone can advise how to preserve the single back-slashes from
> the original configuration.
>
> Note that if I set the regexp to "junit" the tests pass.
>
> Regards
> Julian
>
>
> On Mon, Nov 28, 2016 at 11:40 AM, Stefan Seifert <ss...@pro-vision.de> wrote:
>> for some time the caconfig integration tests are failing because the org.apache.sling.junit.core is not whitelisted in LoginAdminWhitelist (see [1]).
>>
>> but it is whitelisted via regexp (using the same merge-fix as robert for the launchpad tests - see [2]).
>>
>> the resulting slingstart.txt in the build does contain the correct whitelist information see [3]
>>
>> on my local machine the integration tests are running fine. on Jenkins they fail. has anyone an idea what is going wrong? can someone reproduce the failure locally?
>>
>> stefan
>>
>> [1] https://builds.apache.org/job/sling-contrib-extensions-contextaware-config-integration-tests-1.8/201/console
>>
>> [2] https://github.com/apache/sling/blob/trunk/contrib/extensions/contextaware-config/integration-tests/src/main/provisioning/sling.txt
>>
>> [3] https://builds.apache.org/job/sling-contrib-extensions-contextaware-config-integration-tests-1.8/ws/target/slingstart.txt
>>
>>
Re: integration test fail due to LoginAdminWhitelist
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Tue, Nov 29, 2016 at 10:51 AM, Julian Sedding <js...@gmail.com> wrote:
> ...We need to understand the root cause....
If debugging doesn't help (because it changes timing) I suggest adding
enough debug logs to be able to follow and understand what's
happening.
Those are useful for troubleshooting in general anyway, so it's not wasted work.
-Bertrand
Re: integration test fail due to LoginAdminWhitelist
Posted by Julian Sedding <js...@gmail.com>.
Setting whitelist.bypass=true exhibits the same failures (bundle NOT
whitelisted)[0]. So issues with the regexp can be ruled out as the
main cause.
I wonder if we should install the startupfilter(disabler) bundles?
That might delay the TestClient to wait until the system is (more)
ready.
Ultimately I agree with Bertrand though. We need to understand the root cause.
Regards
Julian
[0] https://builds.apache.org/job/sling-contrib-extensions-contextaware-config-integration-tests-1.8/214
On Tue, Nov 29, 2016 at 10:25 AM, Julian Sedding <js...@gmail.com> wrote:
> I mostly agree with Bertrand's interpretation.
>
> However, I a config can be applied early (i.e. registered with config
> admin), but components are still lazily instantiated. Does anyone know
> if there is a guarantee that components are configured (activated)
> before they become visible?
>
> If there is no such guarantee, there may be a time-window when they
> are visible with default config before they are properly configured.
>
> Regards
> Julian
>
> PS: I just committed an experimental change to set
> whitelist.bypaass=true in order to rule out any regexp issues.
>
> On Tue, Nov 29, 2016 at 9:36 AM, Bertrand Delacretaz
> <bd...@apache.org> wrote:
>> On Mon, Nov 28, 2016 at 10:47 PM, Robert Munteanu <ro...@apache.org> wrote:
>>> ...That would be SLING-6305 [1] I guess. ...
>>
>> I haven't debugged deep down yet but intuitively it should be possible
>> to apply very early in the startup phase all the configs that are
>> supplied by the provisioning model.
>>
>> If someone has time to look at why this is not the case that might
>> help solve a number of similar issues + avoid multiple component
>> restarts when configs appear late.
>>
>> -Bertrand
Re: integration test fail due to LoginAdminWhitelist
Posted by Julian Sedding <js...@gmail.com>.
I mostly agree with Bertrand's interpretation.
However, I a config can be applied early (i.e. registered with config
admin), but components are still lazily instantiated. Does anyone know
if there is a guarantee that components are configured (activated)
before they become visible?
If there is no such guarantee, there may be a time-window when they
are visible with default config before they are properly configured.
Regards
Julian
PS: I just committed an experimental change to set
whitelist.bypaass=true in order to rule out any regexp issues.
On Tue, Nov 29, 2016 at 9:36 AM, Bertrand Delacretaz
<bd...@apache.org> wrote:
> On Mon, Nov 28, 2016 at 10:47 PM, Robert Munteanu <ro...@apache.org> wrote:
>> ...That would be SLING-6305 [1] I guess. ...
>
> I haven't debugged deep down yet but intuitively it should be possible
> to apply very early in the startup phase all the configs that are
> supplied by the provisioning model.
>
> If someone has time to look at why this is not the case that might
> help solve a number of similar issues + avoid multiple component
> restarts when configs appear late.
>
> -Bertrand
Re: integration test fail due to LoginAdminWhitelist
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Mon, Nov 28, 2016 at 10:47 PM, Robert Munteanu <ro...@apache.org> wrote:
> ...That would be SLING-6305 [1] I guess. ...
I haven't debugged deep down yet but intuitively it should be possible
to apply very early in the startup phase all the configs that are
supplied by the provisioning model.
If someone has time to look at why this is not the case that might
help solve a number of similar issues + avoid multiple component
restarts when configs appear late.
-Bertrand
Re: integration test fail due to LoginAdminWhitelist
Posted by Robert Munteanu <ro...@apache.org>.
On Mon, 2016-11-28 at 21:23 +0000, Stefan Seifert wrote:
> perhaps there is a timing issue that loads the OSGi config for
> LoginAdminWhitelist too late - and for some reasons only on unix
> environments?!
That would be SLING-6305 [1] I guess.�
Any more data points about narrowing the cause down would be welcome.
For me it's Linux + Java 8 ( Java 7 works fine ) but for some reason (
different setup? faster machine? ) the launchpad/testing job works on
Jenkins.
Robert
[1]: https://issues.apache.org/jira/browse/SLING-6305
RE: integration test fail due to LoginAdminWhitelist
Posted by Stefan Seifert <ss...@pro-vision.de>.
thanks, great!
unfortuantely the integration tests are now failing again. but your changes to org.apache.sling.junit.core do not seem to be the reason - even when rolling back to the latest snapshot before it's failing. same problem as before - "Bundle org.apache.sling.junit.core is NOT whitelisted", although it's whitelisted and the regex is excaped correctly. seemed to be pure luck that it worked 2 times during this day, all other runs just failed on jenkins.
locally i can reproduce:
- always runs fine on windows 10/java 8
- always fails in a vagrant box with ubuntu/java 8
perhaps there is a timing issue that loads the OSGi config for LoginAdminWhitelist too late - and for some reasons only on unix environments?!
stefan
>-----Original Message-----
>From: Julian Sedding [mailto:jsedding@gmail.com]
>Sent: Monday, November 28, 2016 6:45 PM
>To: Sling Developers List
>Subject: Re: integration test fail due to LoginAdminWhitelist
>
>Hi Stefan
>
>When analyzing the issue I noticed that the test was waiting for the
>ResourceResolverFactory service to become available. However, it is
>possible that the test starts before even all bundles in the
>repository are active (SLING-6334[0]). Thus, especially on slow
>systems, the likelihood of a timeout (10s by default) when waiting for
>a service is increased. Furthermore, the system was polled for the
>presence of the service every 50ms. In my local tests that seemed to
>slow down the service becoming available by up to 3 seconds. Waiting
>for the service can be done with a ServiceTracker instead of polling
>(SLING-6335[1]).
>
>After applying my changes, the time from when the test starts (after
>all bundles are active) and the time the ResourceResolverFactory
>becomes available went down from 10+ to ~3 seconds.
>
>Regards
>Julian
>
>[0] https://issues.apache.org/jira/browse/SLING-6334
>[1] https://issues.apache.org/jira/browse/SLING-6335
>
>
>On Mon, Nov 28, 2016 at 2:02 PM, Stefan Seifert <ss...@pro-vision.de>
>wrote:
>> this fix of my problem was finally quite simple: i just had to double the
>backslashes to escape them [1], and then it works, and the generated
>slingstart.txt contains the same string as before.
>>
>> what is still strange that on my system (windows 10 machine with java 8)
>it worked without proper escaping, whereas it failed on Jenkins and on
>julians machine.
>>
>> stefan
>>
>> [1]
>https://github.com/apache/sling/blob/trunk/contrib/extensions/contextaware-
>config/integration-tests/src/main/provisioning/sling.txt#L26
>>
>>
>>
>>>-----Original Message-----
>>>From: Konrad Windszus [mailto:konrad_w@gmx.de]
>>>Sent: Monday, November 28, 2016 1:06 PM
>>>To: dev@sling.apache.org
>>>Subject: Re: integration test fail due to LoginAdminWhitelist
>>>
>>>One last addendum:
>>>Since quoted-string in RFC 2616 has some known bugs
>>>(http://stackoverflow.com/questions/7886782/what-is-the-exact-syntax-and-
>>>semantics-of-a-quoted-string-in-the-http1-1-rfc2616) we should only refer
>>>to https://tools.ietf.org/html/rfc7230#section-3.2.6 instead (in the
>>>documentation).
>>>
>>>> On 28 Nov 2016, at 12:45, Stefan Seifert <ss...@pro-vision.de>
>wrote:
>>>>
>>>>
>>>>> Your current configuration looks like this:
>>>>> whitelist.bundles.regexp="^org\.apache\.sling\.junit(\..*)?$"
>>>>>
>>>>> Inside target/slingstart.txt this becomes:
>>>>> whitelist.bundles.regexp="^org.apache.sling.junit(..*)?$"
>>>>>
>>>>> Note: all slashes have disappeared.
>>>>
>>>> this is strange, yes.
>>>> the same escaping problem happens on my local machine - but there the
>>>integration test passes.
>>>> you could reproduce the problem on your machine?
>>>>
>>>> as a workaround i will simplify the regex and remove the backslashes.
>>>>
>>>> i think we have currently no precise documentation of the sling
>>>provisioning file format esp. regarding escaping rules. on [1] only some
>>>details of the embedded osgi configuration format are documented.
>>>>
>>>> stefan
>>>>
>>>> [1] https://sling.apache.org/documentation/development/slingstart.html
>>>>
>>>
>>
>>
Re: integration test fail due to LoginAdminWhitelist
Posted by Julian Sedding <js...@gmail.com>.
Hi Stefan
When analyzing the issue I noticed that the test was waiting for the
ResourceResolverFactory service to become available. However, it is
possible that the test starts before even all bundles in the
repository are active (SLING-6334[0]). Thus, especially on slow
systems, the likelihood of a timeout (10s by default) when waiting for
a service is increased. Furthermore, the system was polled for the
presence of the service every 50ms. In my local tests that seemed to
slow down the service becoming available by up to 3 seconds. Waiting
for the service can be done with a ServiceTracker instead of polling
(SLING-6335[1]).
After applying my changes, the time from when the test starts (after
all bundles are active) and the time the ResourceResolverFactory
becomes available went down from 10+ to ~3 seconds.
Regards
Julian
[0] https://issues.apache.org/jira/browse/SLING-6334
[1] https://issues.apache.org/jira/browse/SLING-6335
On Mon, Nov 28, 2016 at 2:02 PM, Stefan Seifert <ss...@pro-vision.de> wrote:
> this fix of my problem was finally quite simple: i just had to double the backslashes to escape them [1], and then it works, and the generated slingstart.txt contains the same string as before.
>
> what is still strange that on my system (windows 10 machine with java 8) it worked without proper escaping, whereas it failed on Jenkins and on julians machine.
>
> stefan
>
> [1] https://github.com/apache/sling/blob/trunk/contrib/extensions/contextaware-config/integration-tests/src/main/provisioning/sling.txt#L26
>
>
>
>>-----Original Message-----
>>From: Konrad Windszus [mailto:konrad_w@gmx.de]
>>Sent: Monday, November 28, 2016 1:06 PM
>>To: dev@sling.apache.org
>>Subject: Re: integration test fail due to LoginAdminWhitelist
>>
>>One last addendum:
>>Since quoted-string in RFC 2616 has some known bugs
>>(http://stackoverflow.com/questions/7886782/what-is-the-exact-syntax-and-
>>semantics-of-a-quoted-string-in-the-http1-1-rfc2616) we should only refer
>>to https://tools.ietf.org/html/rfc7230#section-3.2.6 instead (in the
>>documentation).
>>
>>> On 28 Nov 2016, at 12:45, Stefan Seifert <ss...@pro-vision.de> wrote:
>>>
>>>
>>>> Your current configuration looks like this:
>>>> whitelist.bundles.regexp="^org\.apache\.sling\.junit(\..*)?$"
>>>>
>>>> Inside target/slingstart.txt this becomes:
>>>> whitelist.bundles.regexp="^org.apache.sling.junit(..*)?$"
>>>>
>>>> Note: all slashes have disappeared.
>>>
>>> this is strange, yes.
>>> the same escaping problem happens on my local machine - but there the
>>integration test passes.
>>> you could reproduce the problem on your machine?
>>>
>>> as a workaround i will simplify the regex and remove the backslashes.
>>>
>>> i think we have currently no precise documentation of the sling
>>provisioning file format esp. regarding escaping rules. on [1] only some
>>details of the embedded osgi configuration format are documented.
>>>
>>> stefan
>>>
>>> [1] https://sling.apache.org/documentation/development/slingstart.html
>>>
>>
>
>
RE: integration test fail due to LoginAdminWhitelist
Posted by Stefan Seifert <ss...@pro-vision.de>.
this fix of my problem was finally quite simple: i just had to double the backslashes to escape them [1], and then it works, and the generated slingstart.txt contains the same string as before.
what is still strange that on my system (windows 10 machine with java 8) it worked without proper escaping, whereas it failed on Jenkins and on julians machine.
stefan
[1] https://github.com/apache/sling/blob/trunk/contrib/extensions/contextaware-config/integration-tests/src/main/provisioning/sling.txt#L26
>-----Original Message-----
>From: Konrad Windszus [mailto:konrad_w@gmx.de]
>Sent: Monday, November 28, 2016 1:06 PM
>To: dev@sling.apache.org
>Subject: Re: integration test fail due to LoginAdminWhitelist
>
>One last addendum:
>Since quoted-string in RFC 2616 has some known bugs
>(http://stackoverflow.com/questions/7886782/what-is-the-exact-syntax-and-
>semantics-of-a-quoted-string-in-the-http1-1-rfc2616) we should only refer
>to https://tools.ietf.org/html/rfc7230#section-3.2.6 instead (in the
>documentation).
>
>> On 28 Nov 2016, at 12:45, Stefan Seifert <ss...@pro-vision.de> wrote:
>>
>>
>>> Your current configuration looks like this:
>>> whitelist.bundles.regexp="^org\.apache\.sling\.junit(\..*)?$"
>>>
>>> Inside target/slingstart.txt this becomes:
>>> whitelist.bundles.regexp="^org.apache.sling.junit(..*)?$"
>>>
>>> Note: all slashes have disappeared.
>>
>> this is strange, yes.
>> the same escaping problem happens on my local machine - but there the
>integration test passes.
>> you could reproduce the problem on your machine?
>>
>> as a workaround i will simplify the regex and remove the backslashes.
>>
>> i think we have currently no precise documentation of the sling
>provisioning file format esp. regarding escaping rules. on [1] only some
>details of the embedded osgi configuration format are documented.
>>
>> stefan
>>
>> [1] https://sling.apache.org/documentation/development/slingstart.html
>>
>
Re: integration test fail due to LoginAdminWhitelist
Posted by Konrad Windszus <ko...@gmx.de>.
One last addendum:
Since quoted-string in RFC 2616 has some known bugs (http://stackoverflow.com/questions/7886782/what-is-the-exact-syntax-and-semantics-of-a-quoted-string-in-the-http1-1-rfc2616) we should only refer to https://tools.ietf.org/html/rfc7230#section-3.2.6 instead (in the documentation).
> On 28 Nov 2016, at 12:45, Stefan Seifert <ss...@pro-vision.de> wrote:
>
>
>> Your current configuration looks like this:
>> whitelist.bundles.regexp="^org\.apache\.sling\.junit(\..*)?$"
>>
>> Inside target/slingstart.txt this becomes:
>> whitelist.bundles.regexp="^org.apache.sling.junit(..*)?$"
>>
>> Note: all slashes have disappeared.
>
> this is strange, yes.
> the same escaping problem happens on my local machine - but there the integration test passes.
> you could reproduce the problem on your machine?
>
> as a workaround i will simplify the regex and remove the backslashes.
>
> i think we have currently no precise documentation of the sling provisioning file format esp. regarding escaping rules. on [1] only some details of the embedded osgi configuration format are documented.
>
> stefan
>
> [1] https://sling.apache.org/documentation/development/slingstart.html
>
Re: integration test fail due to LoginAdminWhitelist
Posted by Konrad Windszus <ko...@gmx.de>.
To figure out how the escaping exactly works, you can leverage the JCR Installer write back feature (which gives out .config files in the repository)
> On 28 Nov 2016, at 12:54, Konrad Windszus <ko...@gmx.de> wrote:
>
> Currently we have more or less the same documentation at https://sling.apache.org/documentation/bundles/configuration-installer-factory.html#configuration-files-config and https://sling.apache.org/documentation/development/slingstart.html#default-configuration-format. I would be in favor of getting rid of the description in the latter page and just link to the former.
> In the former page the escaping rules should be further clarified.
> Konrad
>
>> On 28 Nov 2016, at 12:45, Stefan Seifert <ss...@pro-vision.de> wrote:
>>
>>
>>> Your current configuration looks like this:
>>> whitelist.bundles.regexp="^org\.apache\.sling\.junit(\..*)?$"
>>>
>>> Inside target/slingstart.txt this becomes:
>>> whitelist.bundles.regexp="^org.apache.sling.junit(..*)?$"
>>>
>>> Note: all slashes have disappeared.
>>
>> this is strange, yes.
>> the same escaping problem happens on my local machine - but there the integration test passes.
>> you could reproduce the problem on your machine?
>>
>> as a workaround i will simplify the regex and remove the backslashes.
>>
>> i think we have currently no precise documentation of the sling provisioning file format esp. regarding escaping rules. on [1] only some details of the embedded osgi configuration format are documented.
>>
>> stefan
>>
>> [1] https://sling.apache.org/documentation/development/slingstart.html
>>
>
Re: integration test fail due to LoginAdminWhitelist
Posted by Konrad Windszus <ko...@gmx.de>.
Currently we have more or less the same documentation at https://sling.apache.org/documentation/bundles/configuration-installer-factory.html#configuration-files-config and https://sling.apache.org/documentation/development/slingstart.html#default-configuration-format. I would be in favor of getting rid of the description in the latter page and just link to the former.
In the former page the escaping rules should be further clarified.
Konrad
> On 28 Nov 2016, at 12:45, Stefan Seifert <ss...@pro-vision.de> wrote:
>
>
>> Your current configuration looks like this:
>> whitelist.bundles.regexp="^org\.apache\.sling\.junit(\..*)?$"
>>
>> Inside target/slingstart.txt this becomes:
>> whitelist.bundles.regexp="^org.apache.sling.junit(..*)?$"
>>
>> Note: all slashes have disappeared.
>
> this is strange, yes.
> the same escaping problem happens on my local machine - but there the integration test passes.
> you could reproduce the problem on your machine?
>
> as a workaround i will simplify the regex and remove the backslashes.
>
> i think we have currently no precise documentation of the sling provisioning file format esp. regarding escaping rules. on [1] only some details of the embedded osgi configuration format are documented.
>
> stefan
>
> [1] https://sling.apache.org/documentation/development/slingstart.html
>
RE: integration test fail due to LoginAdminWhitelist
Posted by Stefan Seifert <ss...@pro-vision.de>.
>Your current configuration looks like this:
>whitelist.bundles.regexp="^org\.apache\.sling\.junit(\..*)?$"
>
>Inside target/slingstart.txt this becomes:
>whitelist.bundles.regexp="^org.apache.sling.junit(..*)?$"
>
>Note: all slashes have disappeared.
this is strange, yes.
the same escaping problem happens on my local machine - but there the integration test passes.
you could reproduce the problem on your machine?
as a workaround i will simplify the regex and remove the backslashes.
i think we have currently no precise documentation of the sling provisioning file format esp. regarding escaping rules. on [1] only some details of the embedded osgi configuration format are documented.
stefan
[1] https://sling.apache.org/documentation/development/slingstart.html
Re: integration test fail due to LoginAdminWhitelist
Posted by Julian Sedding <js...@gmail.com>.
Hi Stefan
I just had a look and noticed the following:
Your current configuration looks like this:
whitelist.bundles.regexp="^org\.apache\.sling\.junit(\..*)?$"
Inside target/slingstart.txt this becomes:
whitelist.bundles.regexp="^org.apache.sling.junit(..*)?$"
Note: all slashes have disappeared.
When I escape the slashes, i.e.
whitelist.bundles.regexp="^org\\.apache\\.sling\\.junit(\\..*)?$"
target/slingstart.txt remains the same:
whitelist.bundles.regexp="^org\\.apache\\.sling\\.junit(\\..*)?$"
I believe there may be a bug in the escaping of provisioning models.
Unless someone can advise how to preserve the single back-slashes from
the original configuration.
Note that if I set the regexp to "junit" the tests pass.
Regards
Julian
On Mon, Nov 28, 2016 at 11:40 AM, Stefan Seifert <ss...@pro-vision.de> wrote:
> for some time the caconfig integration tests are failing because the org.apache.sling.junit.core is not whitelisted in LoginAdminWhitelist (see [1]).
>
> but it is whitelisted via regexp (using the same merge-fix as robert for the launchpad tests - see [2]).
>
> the resulting slingstart.txt in the build does contain the correct whitelist information see [3]
>
> on my local machine the integration tests are running fine. on Jenkins they fail. has anyone an idea what is going wrong? can someone reproduce the failure locally?
>
> stefan
>
> [1] https://builds.apache.org/job/sling-contrib-extensions-contextaware-config-integration-tests-1.8/201/console
>
> [2] https://github.com/apache/sling/blob/trunk/contrib/extensions/contextaware-config/integration-tests/src/main/provisioning/sling.txt
>
> [3] https://builds.apache.org/job/sling-contrib-extensions-contextaware-config-integration-tests-1.8/ws/target/slingstart.txt
>
>