You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by ma...@apache.org on 2019/09/12 20:40:29 UTC
[tomcat] branch master updated (d7d4374 -> cae17a5)
This is an automated email from the ASF dual-hosted git repository.
markt pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git.
from d7d4374 Fix https://bz.apache.org/bugzilla/show_bug.cgi?id=63710
new 583a9f4 PropertySource: Add an environment variable based source
new cae17a5 Additional changes required to enable EnvironmentPropertySource
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../org/apache/tomcat/util/IntrospectionUtils.java | 49 ++++++++++++++++++++--
java/org/apache/tomcat/util/digester/Digester.java | 40 ++++++++++++++----
webapps/docs/changelog.xml | 6 +++
webapps/docs/config/systemprops.xml | 9 +++-
4 files changed, 91 insertions(+), 13 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [tomcat] 02/02: Additional changes required to enable
EnvironmentPropertySource
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 15.09.19 um 11:03 schrieb Mark Thomas:
> On 14/09/2019 20:01, Felix Schumacher wrote:
>> Am 12.09.19 um 22:40 schrieb markt@apache.org:
>>> This is an automated email from the ASF dual-hosted git repository.
>>>
>>> markt pushed a commit to branch master
>>> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>>>
>>> commit cae17a52598393680952aa21cee0e27b13a73455
>>> Author: Mark Thomas <ma...@apache.org>
>>> AuthorDate: Thu Sep 12 15:31:26 2019 +0100
>>>
>>> Additional changes required to enable EnvironmentPropertySource
>>> ---
>>> .../org/apache/tomcat/util/IntrospectionUtils.java | 49 ++++++++++++++++++++--
>>> java/org/apache/tomcat/util/digester/Digester.java | 33 ++++++++++-----
>>> webapps/docs/changelog.xml | 4 +-
>>> 3 files changed, 69 insertions(+), 17 deletions(-)
>>>
>>> diff --git a/java/org/apache/tomcat/util/IntrospectionUtils.java b/java/org/apache/tomcat/util/IntrospectionUtils.java
>>> index 3ffa702..f6ac737 100644
>>> --- a/java/org/apache/tomcat/util/IntrospectionUtils.java
>>> +++ b/java/org/apache/tomcat/util/IntrospectionUtils.java
>>> @@ -476,9 +499,27 @@ public final class IntrospectionUtils {
>>> // This provides a layer of abstraction
>>>
>>> public static interface PropertySource {
>>> -
>>> public String getProperty(String key);
>>> -
>>> }
>>>
>>> +
>>> + public static interface PropertySourceSecure extends PropertySource {
>> I think a better name would be SecurePropertySource or
>> ClassloaderAwarePropertySource. The thing that it represents should be
>> at the end of the name IMHO.
> Fair enough. I prefer "SecurePropertySource" so I'll go with that before
> I tag.
>
>> At work I prototyped a similar approach and introduced a
>> NamespaceAwarePropertySource. It is basically an interface that has a
>> getNamespace() method that returns a prefix for the keys. I think that
>> it would be nice if these two approaches.
> Sorry, I'm not quite understanding how this works or the use case it is
> trying to address. Could you provide a simple example?
A namespaced PropertySource would look like this
interface NamespacedPropertySource extends PropertySource {
String getNamespace(); // or getPrefix()
}
Those PropertySources would be registered by the service loader approach
into a map<String, PropertySource> with their namespace as a key.
If a property is looked up with a key, for example "env.hostname", that
key would be split into the namespace (or prefix) and the actual key for
the source. The SecureProperty from the above map (found by the
namespace) would then be asked to resolve the property.
In this setup, the multiplexer that is looking up the source could check
with the security manager whether access to the key is allowed and thus
freeing the implementer of the NamespacedProperty of doing this work.
class MultiPropertySource implements SecurePropertySource { // not a
good name
Map<String, PropertySource> sources = findThemByServiceLoader();
String getProperty(String key, ClassLoader classLoader) {
String[] nameComponents = key.split(":", 2); // uses a colon for
separation as split uses a regex and a dot is a special char in that context
String namespace = nameComponents[0];
String realKey = nameComponents[1];
if (!checkSecurity(namespace, realKey, classLoader)) { // this would
do the security check
return null;
}
SecurePropertySource source = sources.get(namespace);
return source.getProperty(realKey);
}
}
Does this make sense to you?
Felix
>
>> My prototype didn't try to
>> call a security manager, but with this commit it would be easy to add.
>>
>> On the other hand it uses a ServiceLoader approach to automatically find
>> all NamespaceAwarePropertySources. Do you think this would be a good
>> addition for Tomcat?
> There is an entry in TOMCAT-NEXT around reducing the use of system
> properties. A ServiceLoader approach may be a good solution for some of
> those.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [tomcat] 02/02: Additional changes required to enable
EnvironmentPropertySource
Posted by Mark Thomas <ma...@apache.org>.
On 14/09/2019 20:01, Felix Schumacher wrote:
>
> Am 12.09.19 um 22:40 schrieb markt@apache.org:
>> This is an automated email from the ASF dual-hosted git repository.
>>
>> markt pushed a commit to branch master
>> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>>
>> commit cae17a52598393680952aa21cee0e27b13a73455
>> Author: Mark Thomas <ma...@apache.org>
>> AuthorDate: Thu Sep 12 15:31:26 2019 +0100
>>
>> Additional changes required to enable EnvironmentPropertySource
>> ---
>> .../org/apache/tomcat/util/IntrospectionUtils.java | 49 ++++++++++++++++++++--
>> java/org/apache/tomcat/util/digester/Digester.java | 33 ++++++++++-----
>> webapps/docs/changelog.xml | 4 +-
>> 3 files changed, 69 insertions(+), 17 deletions(-)
>>
>> diff --git a/java/org/apache/tomcat/util/IntrospectionUtils.java b/java/org/apache/tomcat/util/IntrospectionUtils.java
>> index 3ffa702..f6ac737 100644
>> --- a/java/org/apache/tomcat/util/IntrospectionUtils.java
>> +++ b/java/org/apache/tomcat/util/IntrospectionUtils.java
>> @@ -476,9 +499,27 @@ public final class IntrospectionUtils {
>> // This provides a layer of abstraction
>>
>> public static interface PropertySource {
>> -
>> public String getProperty(String key);
>> -
>> }
>>
>> +
>> + public static interface PropertySourceSecure extends PropertySource {
>
> I think a better name would be SecurePropertySource or
> ClassloaderAwarePropertySource. The thing that it represents should be
> at the end of the name IMHO.
Fair enough. I prefer "SecurePropertySource" so I'll go with that before
I tag.
> At work I prototyped a similar approach and introduced a
> NamespaceAwarePropertySource. It is basically an interface that has a
> getNamespace() method that returns a prefix for the keys. I think that
> it would be nice if these two approaches.
Sorry, I'm not quite understanding how this works or the use case it is
trying to address. Could you provide a simple example?
> My prototype didn't try to
> call a security manager, but with this commit it would be easy to add.
>
> On the other hand it uses a ServiceLoader approach to automatically find
> all NamespaceAwarePropertySources. Do you think this would be a good
> addition for Tomcat?
There is an entry in TOMCAT-NEXT around reducing the use of system
properties. A ServiceLoader approach may be a good solution for some of
those.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: [tomcat] 02/02: Additional changes required to enable
EnvironmentPropertySource
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 12.09.19 um 22:40 schrieb markt@apache.org:
> This is an automated email from the ASF dual-hosted git repository.
>
> markt pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>
> commit cae17a52598393680952aa21cee0e27b13a73455
> Author: Mark Thomas <ma...@apache.org>
> AuthorDate: Thu Sep 12 15:31:26 2019 +0100
>
> Additional changes required to enable EnvironmentPropertySource
> ---
> .../org/apache/tomcat/util/IntrospectionUtils.java | 49 ++++++++++++++++++++--
> java/org/apache/tomcat/util/digester/Digester.java | 33 ++++++++++-----
> webapps/docs/changelog.xml | 4 +-
> 3 files changed, 69 insertions(+), 17 deletions(-)
>
> diff --git a/java/org/apache/tomcat/util/IntrospectionUtils.java b/java/org/apache/tomcat/util/IntrospectionUtils.java
> index 3ffa702..f6ac737 100644
> --- a/java/org/apache/tomcat/util/IntrospectionUtils.java
> +++ b/java/org/apache/tomcat/util/IntrospectionUtils.java
> @@ -476,9 +499,27 @@ public final class IntrospectionUtils {
> // This provides a layer of abstraction
>
> public static interface PropertySource {
> -
> public String getProperty(String key);
> -
> }
>
> +
> + public static interface PropertySourceSecure extends PropertySource {
I think a better name would be SecurePropertySource or
ClassloaderAwarePropertySource. The thing that it represents should be
at the end of the name IMHO.
At work I prototyped a similar approach and introduced a
NamespaceAwarePropertySource. It is basically an interface that has a
getNamespace() method that returns a prefix for the keys. I think that
it would be nice if these two approaches. My prototype didn't try to
call a security manager, but with this commit it would be easy to add.
On the other hand it uses a ServiceLoader approach to automatically find
all NamespaceAwarePropertySources. Do you think this would be a good
addition for Tomcat?
Regards
Felix
> +
> + /**
> + * Obtain a property value, checking that code associated with the
> + * provided class loader has permission to access the property. If the
> + * {@code classLoader} is {@code null} or if {@code classLoader} does
> + * not implement {@link PermissionCheck} then the property value will be
> + * looked up <b>without</b> a call to
> + * {@link PermissionCheck#check(java.security.Permission)}
> + *
> + * @param key The key of the requested property
> + * @param classLoader The class loader associated with the code that
> + * trigger the property lookup
> + * @return The property value or {@code null} if it could not be found
> + * or if {@link PermissionCheck#check(java.security.Permission)}
> + * fails
> + */
> + public String getProperty(String key, ClassLoader classLoader);
> + }
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[tomcat] 02/02: Additional changes required to enable
EnvironmentPropertySource
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit cae17a52598393680952aa21cee0e27b13a73455
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Thu Sep 12 15:31:26 2019 +0100
Additional changes required to enable EnvironmentPropertySource
---
.../org/apache/tomcat/util/IntrospectionUtils.java | 49 ++++++++++++++++++++--
java/org/apache/tomcat/util/digester/Digester.java | 33 ++++++++++-----
webapps/docs/changelog.xml | 4 +-
3 files changed, 69 insertions(+), 17 deletions(-)
diff --git a/java/org/apache/tomcat/util/IntrospectionUtils.java b/java/org/apache/tomcat/util/IntrospectionUtils.java
index 3ffa702..f6ac737 100644
--- a/java/org/apache/tomcat/util/IntrospectionUtils.java
+++ b/java/org/apache/tomcat/util/IntrospectionUtils.java
@@ -26,6 +26,7 @@ import java.util.Hashtable;
import org.apache.juli.logging.Log;
import org.apache.juli.logging.LogFactory;
import org.apache.tomcat.util.res.StringManager;
+import org.apache.tomcat.util.security.PermissionCheck;
/**
* Utils for introspection and reflection
@@ -224,9 +225,27 @@ public final class IntrospectionUtils {
* @param staticProp Replacement properties
* @param dynamicProp Replacement properties
* @return the replacement value
+ * @deprecated Use {@link #replaceProperties(String, Hashtable, PropertySource[], ClassLoader)}
*/
+ @Deprecated
public static String replaceProperties(String value,
Hashtable<Object,Object> staticProp, PropertySource dynamicProp[]) {
+ return replaceProperties(value, staticProp, dynamicProp, null);
+ }
+
+ /**
+ * Replace ${NAME} with the property value.
+ * @param value The value
+ * @param staticProp Replacement properties
+ * @param dynamicProp Replacement properties
+ * @param classLoader Class loader associated with the code requesting the
+ * property
+ * @return the replacement value
+ */
+ public static String replaceProperties(String value,
+ Hashtable<Object,Object> staticProp, PropertySource dynamicProp[],
+ ClassLoader classLoader) {
+
if (value.indexOf('$') < 0) {
return value;
}
@@ -257,8 +276,12 @@ public final class IntrospectionUtils {
v = (String) staticProp.get(n);
}
if (v == null && dynamicProp != null) {
- for (int i = 0; i < dynamicProp.length; i++) {
- v = dynamicProp[i].getProperty(n);
+ for (PropertySource propertySource : dynamicProp) {
+ if (propertySource instanceof PropertySourceSecure) {
+ v = ((PropertySourceSecure) propertySource).getProperty(n, classLoader);
+ } else {
+ v = propertySource.getProperty(n);
+ }
if (v != null) {
break;
}
@@ -476,9 +499,27 @@ public final class IntrospectionUtils {
// This provides a layer of abstraction
public static interface PropertySource {
-
public String getProperty(String key);
-
}
+
+ public static interface PropertySourceSecure extends PropertySource {
+
+ /**
+ * Obtain a property value, checking that code associated with the
+ * provided class loader has permission to access the property. If the
+ * {@code classLoader} is {@code null} or if {@code classLoader} does
+ * not implement {@link PermissionCheck} then the property value will be
+ * looked up <b>without</b> a call to
+ * {@link PermissionCheck#check(java.security.Permission)}
+ *
+ * @param key The key of the requested property
+ * @param classLoader The class loader associated with the code that
+ * trigger the property lookup
+ * @return The property value or {@code null} if it could not be found
+ * or if {@link PermissionCheck#check(java.security.Permission)}
+ * fails
+ */
+ public String getProperty(String key, ClassLoader classLoader);
+ }
}
diff --git a/java/org/apache/tomcat/util/digester/Digester.java b/java/org/apache/tomcat/util/digester/Digester.java
index 810274b..610e83c 100644
--- a/java/org/apache/tomcat/util/digester/Digester.java
+++ b/java/org/apache/tomcat/util/digester/Digester.java
@@ -123,13 +123,19 @@ public class Digester extends DefaultHandler2 {
// --------------------------------------------------- Instance Variables
- private class SystemPropertySource implements IntrospectionUtils.PropertySource {
+ private static class SystemPropertySource implements IntrospectionUtils.PropertySourceSecure {
+
@Override
public String getProperty(String key) {
- ClassLoader cl = getClassLoader();
- if (cl instanceof PermissionCheck) {
+ // For backward compatibility
+ return getProperty(key, null);
+ }
+
+ @Override
+ public String getProperty(String key, ClassLoader classLoader) {
+ if (classLoader instanceof PermissionCheck) {
Permission p = new PropertyPermission(key, "read");
- if (!((PermissionCheck) cl).check(p)) {
+ if (!((PermissionCheck) classLoader).check(p)) {
return null;
}
}
@@ -138,13 +144,18 @@ public class Digester extends DefaultHandler2 {
}
- public class EnvironmentPropertySource implements IntrospectionUtils.PropertySource {
+ public static class EnvironmentPropertySource implements IntrospectionUtils.PropertySourceSecure {
+
@Override
public String getProperty(String key) {
- ClassLoader cl = getClassLoader();
- if (cl instanceof PermissionCheck) {
+ return null;
+ }
+
+ @Override
+ public String getProperty(String key, ClassLoader classLoader) {
+ if (classLoader instanceof PermissionCheck) {
Permission p = new RuntimePermission("getenv." + key, null);
- if (!((PermissionCheck) cl).check(p)) {
+ if (!((PermissionCheck) classLoader).check(p)) {
return null;
}
}
@@ -351,7 +362,7 @@ public class Digester extends DefaultHandler2 {
String value = System.getProperty(name);
if (value != null) {
try {
- String newValue = IntrospectionUtils.replaceProperties(value, null, propertySources);
+ String newValue = IntrospectionUtils.replaceProperties(value, null, propertySources, null);
if (!value.equals(newValue)) {
System.setProperty(name, newValue);
}
@@ -1933,7 +1944,7 @@ public class Digester extends DefaultHandler2 {
for (int i = 0; i < nAttributes; ++i) {
String value = newAttrs.getValue(i);
try {
- newAttrs.setValue(i, IntrospectionUtils.replaceProperties(value, null, source).intern());
+ newAttrs.setValue(i, IntrospectionUtils.replaceProperties(value, null, source, getClassLoader()).intern());
} catch (Exception e) {
log.warn(sm.getString("digester.failedToUpdateAttributes", newAttrs.getLocalName(i), value), e);
}
@@ -1952,7 +1963,7 @@ public class Digester extends DefaultHandler2 {
String in = bodyText.toString();
String out;
try {
- out = IntrospectionUtils.replaceProperties(in, null, source);
+ out = IntrospectionUtils.replaceProperties(in, null, source, getClassLoader());
} catch (Exception e) {
return bodyText; // return unchanged data
}
diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
index e89ca90..bec625b 100644
--- a/webapps/docs/changelog.xml
+++ b/webapps/docs/changelog.xml
@@ -69,8 +69,8 @@
<add>
Add a new <code>PropertySource</code> implementation,
<code>EnvironmentPropertySource</code>, that can be used to do property
- replacement in configuration files with environment variables. Pull
- request provided by Thomas Meyer. (markt)
+ replacement in configuration files with environment variables. Based on
+ a pull request provided by Thomas Meyer. (markt)
</add>
</changelog>
</subsection>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[tomcat] 01/02: PropertySource: Add an environment variable based
source
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 583a9f40ed033807db11827596e999f007264adb
Author: Thomas Meyer <th...@m3y3r.de>
AuthorDate: Sat Jul 20 22:22:42 2019 +0200
PropertySource: Add an environment variable based source
When tomcat runs in an Openshift based container a Secret containing passwords
can be map as environment variables (with an additional prefix).
An webapp containing an embedded context.xml which defines JDBC datasources and
placeholder variables can be used with this new PropertySource to easily inject
configuration from a Secret or ConfigMap.
---
java/org/apache/tomcat/util/digester/Digester.java | 15 +++++++++++++++
webapps/docs/changelog.xml | 6 ++++++
webapps/docs/config/systemprops.xml | 9 +++++++--
3 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/java/org/apache/tomcat/util/digester/Digester.java b/java/org/apache/tomcat/util/digester/Digester.java
index e3ed881..810274b 100644
--- a/java/org/apache/tomcat/util/digester/Digester.java
+++ b/java/org/apache/tomcat/util/digester/Digester.java
@@ -138,6 +138,21 @@ public class Digester extends DefaultHandler2 {
}
+ public class EnvironmentPropertySource implements IntrospectionUtils.PropertySource {
+ @Override
+ public String getProperty(String key) {
+ ClassLoader cl = getClassLoader();
+ if (cl instanceof PermissionCheck) {
+ Permission p = new RuntimePermission("getenv." + key, null);
+ if (!((PermissionCheck) cl).check(p)) {
+ return null;
+ }
+ }
+ return System.getenv(key);
+ }
+ }
+
+
protected IntrospectionUtils.PropertySource source[] = new IntrospectionUtils.PropertySource[] {
new SystemPropertySource() };
diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
index 7bdf0e7..e89ca90 100644
--- a/webapps/docs/changelog.xml
+++ b/webapps/docs/changelog.xml
@@ -66,6 +66,12 @@
Service is embedded directly (i.e. with no Server) in an applciation
and JNDI is enabled. Patch provided by S. Ali Tokmen. (markt)
</fix>
+ <add>
+ Add a new <code>PropertySource</code> implementation,
+ <code>EnvironmentPropertySource</code>, that can be used to do property
+ replacement in configuration files with environment variables. Pull
+ request provided by Thomas Meyer. (markt)
+ </add>
</changelog>
</subsection>
<subsection name="Coyote">
diff --git a/webapps/docs/config/systemprops.xml b/webapps/docs/config/systemprops.xml
index a17687f..d49a67d 100644
--- a/webapps/docs/config/systemprops.xml
+++ b/webapps/docs/config/systemprops.xml
@@ -43,11 +43,16 @@
<p>Set this to a fully qualified name of a class that implements
<code>org.apache.tomcat.util.IntrospectionUtils.PropertySource</code>.
Required to have a public constructor with no arguments.</p>
- <p>Use this to add a property source, that will be invoked when <code>${parameter}</code>
- denoted parameters are found in the XML files that Tomcat parses.</p>
+ <p>Use this to add a property source, that will be invoked when
+ <code>${parameter}</code> denoted parameters are found in the XML files
+ that Tomcat parses.</p>
<p>Property replacement from the specified property source on the JVM
system properties can also be done using the
<code>REPLACE_SYSTEM_PROPERTIES</code> system property.</p>
+ <p><code>org.apache.tomcat.util.digester.Digester$EnvironmentPropertySource</code>
+ can be used to replace parameters from the process' environment
+ variables, e.g. injected ConfigMaps or Secret objects in container
+ based systems like OpenShift or Kubernetes.</p>
</property>
<property name="org.apache.tomcat.util.digester. REPLACE_SYSTEM_PROPERTIES">
<p>Set this boolean system property to <code>true</code> to cause
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org