You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by ep...@apache.org on 2021/01/21 18:33:01 UTC
[lucene-solr] branch master updated: SOLR-14067: v3 Create
/contrib/scripting module with ScriptingUpdateProcessor (#2215)
This is an automated email from the ASF dual-hosted git repository.
epugh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
The following commit(s) were added to refs/heads/master by this push:
new cf5db8d SOLR-14067: v3 Create /contrib/scripting module with ScriptingUpdateProcessor (#2215)
cf5db8d is described below
commit cf5db8d6513e0f3e556ab6ee1b9ad3a6472ad2f2
Author: Eric Pugh <ep...@opensourceconnections.com>
AuthorDate: Thu Jan 21 13:32:46 2021 -0500
SOLR-14067: v3 Create /contrib/scripting module with ScriptingUpdateProcessor (#2215)
* Creating Scripting contrib module to centralize the less secure code related to scripts.
* tweak the changelog and update notice to explain why the name changed and the security posture thinking
* the test script happens to be a currency.xml, which made me think we were doing something specific to currency types, but instead any xml formatted file will suffice for the test.
* drop the ing, and be more specific on the name of the ref guide page
* use the same name everywhere
Co-authored-by: David Smiley <ds...@apache.org>
---
gradle/maven/defaults-maven.gradle | 1 +
settings.gradle | 1 +
solr/CHANGES.txt | 3 +
solr/contrib/scripting/README.md | 14 +
.../scripting/build.gradle} | 13 +-
.../scripting/update}/ScriptEngineCustomizer.java | 2 +-
.../update/ScriptUpdateProcessorFactory.java} | 145 +++++------
.../solr/scripting/update/package-info.java} | 11 +-
.../scripting/src/java/overview.html} | 27 +-
.../collection1/conf/addfields.updateprocessor.js | 0
.../bad-solrconfig-bogus-scriptengine-name.xml | 2 +-
.../conf/bad-solrconfig-invalid-scriptfile.xml | 4 +-
.../conf/bad-solrconfig-missing-scriptfile.xml | 2 +-
.../conf/conditional.updateprocessor.js | 0
.../solr/collection1/conf/cross-compatible.js | 0
.../src/test-files/solr/collection1/conf/evil.js | 0
.../solr/collection1/conf/invalid.script.xml} | 20 +-
.../conf/missing.functions.updateprocessor.js | 0
.../missleading.extension.updateprocessor.js.txt | 0
.../test-files/solr/collection1/conf/schema.xml | 73 ++++++
.../conf/solrconfig-script-updateprocessor.xml | 30 +--
.../conf/solrconfig.snippet.randomindexconfig.xml | 51 ++++
...tateless-solrconfig-script-updateprocessor.xml} | 26 +-
.../conf/throw.error.on.add.updateprocessor.js | 0
.../collection1/conf/trivial.updateprocessor0.js | 0
.../collection1/conf/trivial.updateprocessor1.js | 0
.../solr/scripting/update}/ScriptEngineTest.java | 2 +-
.../update/ScriptUpdateProcessorFactoryTest.java} | 72 +++---
.../TestBadScriptingUpdateProcessorConfig.java | 49 ++++
.../missleading.extension.updateprocessor.js.txt | 23 --
.../upload/with-script-processor/solrconfig.xml | 10 +-
.../trivial.updateprocessor.js | 22 ++
.../test/org/apache/solr/core/TestBadConfig.java | 30 +--
solr/packaging/build.gradle | 1 +
.../conf/solrconfig.xml | 37 +--
.../conf/update-script.js | 4 +-
solr/solr-ref-guide/src/configsets-api.adoc | 2 +-
.../src/configuring-solrconfig-xml.adoc | 1 +
.../src/major-changes-in-solr-9.adoc | 3 +
.../src/script-update-processor.adoc | 286 +++++++++++++++++++++
.../src/update-request-processors.adoc | 2 +-
.../update/processor/UpdateProcessorTestBase.java | 4 +-
42 files changed, 704 insertions(+), 269 deletions(-)
diff --git a/gradle/maven/defaults-maven.gradle b/gradle/maven/defaults-maven.gradle
index 2662a69..443d690 100644
--- a/gradle/maven/defaults-maven.gradle
+++ b/gradle/maven/defaults-maven.gradle
@@ -66,6 +66,7 @@ configure(rootProject) {
":solr:contrib:langid",
":solr:contrib:jaegertracer-configurator",
":solr:contrib:prometheus-exporter",
+ ":solr:contrib:scripting",
":solr:test-framework",
]
diff --git a/settings.gradle b/settings.gradle
index be2c09c..494c710 100644
--- a/settings.gradle
+++ b/settings.gradle
@@ -65,6 +65,7 @@ include "solr:contrib:extraction"
include "solr:contrib:langid"
include "solr:contrib:jaegertracer-configurator"
include "solr:contrib:prometheus-exporter"
+include "solr:contrib:scripting"
include "solr:contrib:ltr"
include "solr:webapp"
include "solr:test-framework"
diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt
index 4ce0332..7a17a75 100644
--- a/solr/CHANGES.txt
+++ b/solr/CHANGES.txt
@@ -188,6 +188,9 @@ Other Changes
* SOLR-14297: Replace commons-codec Base64 with JDK8 Base64 (Andras Salamon via Houston Putman)
+* SOLR-14067: StatelessScriptUpdateProcessorFactory moved to it's own /contrib/scripting/ package instead
+ of shipping as part of Solr due to security concerns. Renamed to ScriptUpdateProcessorFactory for simpler name. (Eric Pugh)
+
Bug Fixes
---------------------
* SOLR-14546: Fix for a relatively hard to hit issue in OverseerTaskProcessor that could lead to out of order execution
diff --git a/solr/contrib/scripting/README.md b/solr/contrib/scripting/README.md
new file mode 100644
index 0000000..3436a65
--- /dev/null
+++ b/solr/contrib/scripting/README.md
@@ -0,0 +1,14 @@
+Welcome to Apache Solr Scripting!
+===============================
+
+# Introduction
+
+The Scripting contrib module pulls together various scripting related functions.
+
+Today, the ScriptUpdateProcessorFactory allows Java scripting engines to support scripts written in languages such as JavaScript, Ruby, Python, and Groovy to be used during Solr document update processing, allowing dramatic flexibility in expressing custom document processing before being indexed. It exposes hooks for commit, delete, etc, but add is the most common usage. It is implemented as an UpdateProcessor to be placed in an UpdateChain.
+
+## Getting Started
+
+For information on how to get started please see:
+ * [Solr Reference Guide's section on Update Request Processors](https://lucene.apache.org/solr/guide/update-request-processors.html)
+ * [Solr Reference Guide's section on ScriptUpdateProcessorFactory](https://lucene.apache.org/solr/guide/script-update-processor.html)
diff --git a/solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java b/solr/contrib/scripting/build.gradle
similarity index 75%
copy from solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java
copy to solr/contrib/scripting/build.gradle
index f230526..32f4e1e 100644
--- a/solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java
+++ b/solr/contrib/scripting/build.gradle
@@ -14,15 +14,12 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.solr.update.processor;
-import javax.script.ScriptEngine;
+apply plugin: 'java-library'
-/**
- * Enables customization of a script engine. Will mostly be used to register engine scoped variables.
- */
-public interface ScriptEngineCustomizer {
-
- void customize(ScriptEngine engine);
+description = 'Scripting Package'
+dependencies {
+ implementation project(':solr:core')
+ testImplementation project(':solr:test-framework')
}
diff --git a/solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java b/solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptEngineCustomizer.java
similarity index 96%
copy from solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java
copy to solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptEngineCustomizer.java
index f230526..05dd604 100644
--- a/solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java
+++ b/solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptEngineCustomizer.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.solr.update.processor;
+package org.apache.solr.scripting.update;
import javax.script.ScriptEngine;
diff --git a/solr/core/src/java/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactory.java b/solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.java
similarity index 89%
rename from solr/core/src/java/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactory.java
rename to solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.java
index d2f5a07..e0de01b 100644
--- a/solr/core/src/java/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactory.java
+++ b/solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.solr.update.processor;
+package org.apache.solr.scripting.update;
import org.apache.solr.common.SolrException;
import org.apache.solr.common.SolrException.ErrorCode;
@@ -26,6 +26,8 @@ import org.apache.solr.request.SolrQueryRequest;
import org.apache.solr.request.LocalSolrQueryRequest;
import org.apache.solr.response.SolrQueryResponse;
import org.apache.solr.update.*;
+import org.apache.solr.update.processor.UpdateRequestProcessor;
+import org.apache.solr.update.processor.UpdateRequestProcessorFactory;
import org.apache.solr.util.plugin.SolrCoreAware;
import org.apache.commons.io.IOUtils;
import org.apache.commons.io.FilenameUtils;
@@ -58,34 +60,33 @@ import org.slf4j.LoggerFactory;
/**
* <p>
- * An update request processor factory that enables the use of update
- * processors implemented as scripts which can be loaded by the
- * {@link SolrResourceLoader} (usually via the <code>conf</code> dir for
- * the SolrCore).
+ * An update request processor factory that enables the use of update
+ * processors implemented as scripts which can be loaded from the
+ * configSet. Previously known as the StatelessScriptUpdateProcessorFactory.
* </p>
* <p>
* This factory requires at least one configuration parameter named
- * <code>script</code> which may be the name of a script file as a string,
- * or an array of multiple script files. If multiple script files are
- * specified, they are executed sequentially in the order specified in the
+ * <code>script</code> which may be the name of a script file as a string,
+ * or an array of multiple script files. If multiple script files are
+ * specified, they are executed sequentially in the order specified in the
* configuration -- as if multiple factories were configured sequentially
* </p>
* <p>
- * Each script file is expected to declare functions with the same name
- * as each method in {@link UpdateRequestProcessor}, using the same
- * arguments. One slight deviation is in the optional return value from
- * these functions: If a script function has a <code>boolean</code> return
- * value, and that value is <code>false</code> then the processor will
- * cleanly terminate processing of the command and return, without forwarding
+ * Each script file is expected to declare functions with the same name
+ * as each method in {@link UpdateRequestProcessor}, using the same
+ * arguments. One slight deviation is in the optional return value from
+ * these functions: If a script function has a <code>boolean</code> return
+ * value, and that value is <code>false</code> then the processor will
+ * cleanly terminate processing of the command and return, without forwarding
* the command on to the next script or processor in the chain.
- * Due to limitations in the {@link ScriptEngine} API used by
+ * Due to limitations in the {@link ScriptEngine} API used by
* this factory, it can not enforce that all functions exist on initialization,
* so errors from missing functions will only be generated at runtime when
* the chain attempts to use them.
* </p>
* <p>
- * The factory may also be configured with an optional "params" argument,
- * which can be an {@link NamedList} (or array, or any other simple Java
+ * The factory may also be configured with an optional "params" argument,
+ * which can be an {@link NamedList} (or array, or any other simple Java
* object) which will be put into the global scope for each script.
* </p>
* <p>
@@ -97,40 +98,40 @@ import org.slf4j.LoggerFactory;
* <li>params - The "params" init argument in the factory configuration (if any)</li>
* </ul>
* <p>
- * Internally this update processor uses JDK 6 scripting engine support,
- * and any {@link Invocable} implementations of <code>ScriptEngine</code>
- * that can be loaded using the Solr Plugin ClassLoader may be used.
- * By default, the engine used for each script is determined by the filed
- * extension (ie: a *.js file will be treated as a JavaScript script) but
- * this can be overridden by specifying an explicit "engine" name init
- * param for the factory, which identifies a registered name of a
- * {@link ScriptEngineFactory}.
- * (This may be particularly useful if multiple engines are available for
- * the same scripting language, and you wish to force the usage of a
+ * Internally this update processor uses JDK 6 scripting engine support,
+ * and any {@link Invocable} implementations of <code>ScriptEngine</code>
+ * that can be loaded using the Solr Plugin ClassLoader may be used.
+ * By default, the engine used for each script is determined by the file
+ * extension (ie: a *.js file will be treated as a JavaScript script) but
+ * this can be overridden by specifying an explicit "engine" name init
+ * param for the factory, which identifies a registered name of a
+ * {@link ScriptEngineFactory}.
+ * (This may be particularly useful if multiple engines are available for
+ * the same scripting language, and you wish to force the usage of a
* particular engine because of known quirks)
* </p>
* <p>
- * A new {@link ScriptEngineManager} is created for each
- * <code>SolrQueryRequest</code> defining a "global" scope for the script(s)
- * which is request specific. Separate <code>ScriptEngine</code> instances
- * are then used to evaluate the script files, resulting in an "engine" scope
+ * A new {@link ScriptEngineManager} is created for each
+ * <code>SolrQueryRequest</code> defining a "global" scope for the script(s)
+ * which is request specific. Separate <code>ScriptEngine</code> instances
+ * are then used to evaluate the script files, resulting in an "engine" scope
* that is specific to each script.
* </p>
* <p>
* A simple example...
* </p>
* <pre class="prettyprint">
- * <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ * <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
* <str name="script">updateProcessor.js</str>
* </processor>
* </pre>
* <p>
- * A more complex example involving multiple scripts in different languages,
- * and a "params" <code>NamedList</code> that will be put into the global
+ * A more complex example involving multiple scripts in different languages,
+ * and a "params" <code>NamedList</code> that will be put into the global
* scope of each script...
* </p>
* <pre class="prettyprint">
- * <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ * <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
* <arr name="script">
* <str name="script">first-processor.js</str>
* <str name="script">second-processor.py</str>
@@ -142,11 +143,11 @@ import org.slf4j.LoggerFactory;
* </processor>
* </pre>
* <p>
- * An example where the script file extensions are ignored, and an
+ * An example where the script file extensions are ignored, and an
* explicit script engine is used....
* </p>
* <pre class="prettyprint">
- * <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ * <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
* <arr name="script">
* <str name="script">first-processor.txt</str>
* <str name="script">second-processor.txt</str>
@@ -154,10 +155,10 @@ import org.slf4j.LoggerFactory;
* <str name="engine">rhino</str>
* </processor>
* </pre>
- *
+ *
* @since 4.0.0
*/
-public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcessorFactory implements SolrCoreAware {
+public class ScriptUpdateProcessorFactory extends UpdateRequestProcessorFactory implements SolrCoreAware {
private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
@@ -182,8 +183,8 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
Collection<String> scripts =
args.removeConfigArgs(SCRIPT_ARG);
if (scripts.isEmpty()) {
- throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
- "StatelessScriptUpdateProcessorFactory must be " +
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
+ "ScriptUpdateProcessorFactory must be " +
"initialized with at least one " + SCRIPT_ARG);
}
scriptFiles = new ArrayList<>();
@@ -199,8 +200,8 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
engineName = (String)engine;
} else {
throw new SolrException
- (SolrException.ErrorCode.SERVER_ERROR,
- "'" + ENGINE_NAME_ARG + "' init param must be a String (found: " +
+ (SolrException.ErrorCode.SERVER_ERROR,
+ "'" + ENGINE_NAME_ARG + "' init param must be a String (found: " +
engine.getClass() + ")");
}
}
@@ -246,7 +247,7 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
req.close();
}
-
+
}
@@ -259,13 +260,13 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
* @param rsp The solr response
* @return The list of initialized script engines.
*/
- private List<EngineInfo> initEngines(SolrQueryRequest req,
- SolrQueryResponse rsp)
+ private List<EngineInfo> initEngines(SolrQueryRequest req,
+ SolrQueryResponse rsp)
throws SolrException {
-
+
List<EngineInfo> scriptEngines = new ArrayList<>();
- ScriptEngineManager scriptEngineManager
+ ScriptEngineManager scriptEngineManager
= new ScriptEngineManager(resourceLoader.getClassLoader());
scriptEngineManager.put("logger", log);
@@ -281,10 +282,10 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
engine = scriptEngineManager.getEngineByName(engineName);
if (engine == null) {
String details = getSupportedEngines(scriptEngineManager, false);
- throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"No ScriptEngine found by name: "
- + engineName +
- (null != details ?
+ + engineName +
+ (null != details ?
" -- supported names: " + details : ""));
}
} else {
@@ -292,18 +293,18 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
(scriptFile.getExtension());
if (engine == null) {
String details = getSupportedEngines(scriptEngineManager, true);
- throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"No ScriptEngine found by file extension: "
- + scriptFile.getFileName() +
- (null != details ?
+ + scriptFile.getFileName() +
+ (null != details ?
" -- supported extensions: " + details : ""));
-
+
}
}
if (! (engine instanceof Invocable)) {
- String msg =
- "Engine " + ((null != engineName) ? engineName :
+ String msg =
+ "Engine " + ((null != engineName) ? engineName :
("for script " + scriptFile.getFileName())) +
" does not support function invocation (via Invocable): " +
engine.getClass().toString() + " (" +
@@ -319,7 +320,7 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
scriptEngines.add(new EngineInfo((Invocable)engine, scriptFile));
try {
Reader scriptSrc = scriptFile.openReader(resourceLoader);
-
+
try {
try {
AccessController.doPrivileged(new PrivilegedExceptionAction<Void>() {
@@ -333,23 +334,23 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
throw (ScriptException) e.getException();
}
} catch (ScriptException e) {
- throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
- "Unable to evaluate script: " +
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
+ "Unable to evaluate script: " +
scriptFile.getFileName(), e);
} finally {
IOUtils.closeQuietly(scriptSrc);
}
} catch (IOException ioe) {
- throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
- "Unable to evaluate script: " +
- scriptFile.getFileName(), ioe);
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
+ "Unable to evaluate script: " +
+ scriptFile.getFileName(), ioe);
}
}
return scriptEngines;
}
/**
- * For error messages - returns null if there are any exceptions of any
+ * For error messages - returns null if there are any exceptions of any
* kind building the string (or of the list is empty for some unknown reason).
* @param ext - if true, list of extensions, otherwise a list of engine names
*/
@@ -403,7 +404,7 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
if (invokeFunction("processDelete", cmd)) {
super.processDelete(cmd);
}
-
+
}
@Override
@@ -435,9 +436,9 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
}
/**
- * returns true if processing should continue, or false if the
- * request should be ended now. Result value is computed from the return
- * value of the script function if: it exists, is non-null, and can be
+ * returns true if processing should continue, or false if the
+ * request should be ended now. Result value is computed from the return
+ * value of the script function if: it exists, is non-null, and can be
* cast to a java Boolean.
*/
private boolean invokeFunction(String name, Object... cmd) {
@@ -461,10 +462,10 @@ public class StatelessScriptUpdateProcessorFactory extends UpdateRequestProcesso
}
} catch (ScriptException | NoSuchMethodException e) {
- throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
- "Unable to invoke function " + name +
- " in script: " +
- engine.getScriptFile().getFileName() +
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
+ "Unable to invoke function " + name +
+ " in script: " +
+ engine.getScriptFile().getFileName() +
": " + e.getMessage(), e);
}
}
diff --git a/solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java b/solr/contrib/scripting/src/java/org/apache/solr/scripting/update/package-info.java
similarity index 75%
rename from solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java
rename to solr/contrib/scripting/src/java/org/apache/solr/scripting/update/package-info.java
index f230526..5bf1066 100644
--- a/solr/core/src/java/org/apache/solr/update/processor/ScriptEngineCustomizer.java
+++ b/solr/contrib/scripting/src/java/org/apache/solr/scripting/update/package-info.java
@@ -14,15 +14,8 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.solr.update.processor;
-
-import javax.script.ScriptEngine;
/**
- * Enables customization of a script engine. Will mostly be used to register engine scoped variables.
+ * Support for scripting during document updates.
*/
-public interface ScriptEngineCustomizer {
-
- void customize(ScriptEngine engine);
-
-}
+package org.apache.solr.scripting.update;
diff --git a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml b/solr/contrib/scripting/src/java/overview.html
similarity index 61%
copy from solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
copy to solr/contrib/scripting/src/java/overview.html
index d13ba0b..82eb0f8 100644
--- a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
+++ b/solr/contrib/scripting/src/java/overview.html
@@ -1,5 +1,3 @@
-<?xml version="1.0" ?>
-
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -16,18 +14,13 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
-
-<config>
-
- <luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
-
- <updateRequestProcessorChain name="force-script-engine" default="true">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
- <str name="script">a-file-name-that-does-not-exist.js</str>
- </processor>
- <processor class="solr.RunUpdateProcessorFactory" />
- </updateRequestProcessorChain>
-
- <schemaFactory class="ClassicIndexSchemaFactory"/>
-
-</config>
+<html>
+<body>
+Apache Solr Search Server: Scripting contrib
+
+<p>
+This package provides an Update Processor that allows for Java scripting engines
+to be used during the Solr document update processing.
+</p>
+</body>
+</html>
diff --git a/solr/core/src/test-files/solr/collection1/conf/addfields.updateprocessor.js b/solr/contrib/scripting/src/test-files/solr/collection1/conf/addfields.updateprocessor.js
similarity index 100%
rename from solr/core/src/test-files/solr/collection1/conf/addfields.updateprocessor.js
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/addfields.updateprocessor.js
diff --git a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-bogus-scriptengine-name.xml b/solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-bogus-scriptengine-name.xml
similarity index 93%
rename from solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-bogus-scriptengine-name.xml
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-bogus-scriptengine-name.xml
index ded7416..9bc5506 100644
--- a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-bogus-scriptengine-name.xml
+++ b/solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-bogus-scriptengine-name.xml
@@ -22,7 +22,7 @@
<luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
<updateRequestProcessorChain name="force-script-engine" default="true">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="engine">giberish</str>
<str name="script">missleading.extension.updateprocessor.js.txt</str>
</processor>
diff --git a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-invalid-scriptfile.xml b/solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-invalid-scriptfile.xml
similarity index 89%
rename from solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-invalid-scriptfile.xml
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-invalid-scriptfile.xml
index 709774d..3542779 100644
--- a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-invalid-scriptfile.xml
+++ b/solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-invalid-scriptfile.xml
@@ -22,10 +22,10 @@
<luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
<updateRequestProcessorChain name="force-script-engine" default="true">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="engine">javascript</str>
<!-- not parsable as javascript -->
- <str name="script">currency.xml</str>
+ <str name="script">invalid.script.xml</str>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
diff --git a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml b/solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
similarity index 93%
copy from solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
copy to solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
index d13ba0b..ed43d9f 100644
--- a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
+++ b/solr/contrib/scripting/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
@@ -22,7 +22,7 @@
<luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
<updateRequestProcessorChain name="force-script-engine" default="true">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">a-file-name-that-does-not-exist.js</str>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
diff --git a/solr/core/src/test-files/solr/collection1/conf/conditional.updateprocessor.js b/solr/contrib/scripting/src/test-files/solr/collection1/conf/conditional.updateprocessor.js
similarity index 100%
rename from solr/core/src/test-files/solr/collection1/conf/conditional.updateprocessor.js
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/conditional.updateprocessor.js
diff --git a/solr/core/src/test-files/solr/collection1/conf/cross-compatible.js b/solr/contrib/scripting/src/test-files/solr/collection1/conf/cross-compatible.js
similarity index 100%
rename from solr/core/src/test-files/solr/collection1/conf/cross-compatible.js
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/cross-compatible.js
diff --git a/solr/core/src/test-files/solr/collection1/conf/evil.js b/solr/contrib/scripting/src/test-files/solr/collection1/conf/evil.js
similarity index 100%
rename from solr/core/src/test-files/solr/collection1/conf/evil.js
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/evil.js
diff --git a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml b/solr/contrib/scripting/src/test-files/solr/collection1/conf/invalid.script.xml
similarity index 63%
rename from solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/invalid.script.xml
index d13ba0b..c8455af 100644
--- a/solr/core/src/test-files/solr/collection1/conf/bad-solrconfig-missing-scriptfile.xml
+++ b/solr/contrib/scripting/src/test-files/solr/collection1/conf/invalid.script.xml
@@ -1,5 +1,4 @@
<?xml version="1.0" ?>
-
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -17,17 +16,12 @@
limitations under the License.
-->
-<config>
-
- <luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
+<!-- This file is designed to test that loading a .xml file into
+ a script engine that is configured for JavaScript properly raises
+ an exception.
- <updateRequestProcessorChain name="force-script-engine" default="true">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
- <str name="script">a-file-name-that-does-not-exist.js</str>
- </processor>
- <processor class="solr.RunUpdateProcessorFactory" />
- </updateRequestProcessorChain>
-
- <schemaFactory class="ClassicIndexSchemaFactory"/>
+-->
-</config>
+<foo version="1.0">
+ <bar/>
+</foo>
diff --git a/solr/core/src/test-files/solr/collection1/conf/missing.functions.updateprocessor.js b/solr/contrib/scripting/src/test-files/solr/collection1/conf/missing.functions.updateprocessor.js
similarity index 100%
rename from solr/core/src/test-files/solr/collection1/conf/missing.functions.updateprocessor.js
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/missing.functions.updateprocessor.js
diff --git a/solr/core/src/test-files/solr/configsets/upload/with-script-processor/missleading.extension.updateprocessor.js.txt b/solr/contrib/scripting/src/test-files/solr/collection1/conf/missleading.extension.updateprocessor.js.txt
similarity index 100%
rename from solr/core/src/test-files/solr/configsets/upload/with-script-processor/missleading.extension.updateprocessor.js.txt
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/missleading.extension.updateprocessor.js.txt
diff --git a/solr/contrib/scripting/src/test-files/solr/collection1/conf/schema.xml b/solr/contrib/scripting/src/test-files/solr/collection1/conf/schema.xml
new file mode 100644
index 0000000..140f12b
--- /dev/null
+++ b/solr/contrib/scripting/src/test-files/solr/collection1/conf/schema.xml
@@ -0,0 +1,73 @@
+<?xml version="1.0" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<!-- The Solr schema file. This file should be named "schema.xml" and
+ should be located where the classloader for the Solr webapp can find it.
+
+ This schema is used for testing, and as such has everything and the
+ kitchen sink thrown in. See example/solr/conf/schema.xml for a
+ more concise example.
+
+ -->
+
+<schema name="test" version="1.6">
+
+ <fieldType name="int" class="${solr.tests.IntegerFieldType}" docValues="${solr.tests.numeric.dv}" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
+ <fieldType name="double" class="${solr.tests.DoubleFieldType}" docValues="${solr.tests.numeric.dv}" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
+
+ <fieldType name="string" class="solr.StrField" sortMissingLast="true"/>
+ <!-- solr.TextField allows the specification of custom
+ text analyzers specified as a tokenizer and a list
+ of token filters.
+ -->
+ <fieldType name="text" class="solr.TextField">
+ <analyzer>
+ <tokenizer class="solr.StandardTokenizerFactory"/>
+ <filter class="solr.LowerCaseFilterFactory"/>
+ <filter class="solr.StopFilterFactory"/>
+ <filter class="solr.PorterStemFilterFactory"/>
+ </analyzer>
+ </fieldType>
+
+ <fieldType name="nametext" class="solr.TextField">
+ <analyzer class="org.apache.lucene.analysis.core.WhitespaceAnalyzer"/>
+ </fieldType>
+
+
+ <field name="id" type="string" indexed="true" stored="true" multiValued="false" required="false"/>
+ <field name="name" type="nametext" indexed="true" stored="true"/>
+ <field name="subject" type="text" indexed="true" stored="true"/>
+
+
+ <!-- Dynamic field definitions. If a field name is not found, dynamicFields
+ will be used if the name matches any of the patterns.
+ RESTRICTION: the glob-like pattern in the name attribute must have
+ a "*" only at the start or the end.
+ EXAMPLE: name="*_i" will match any field ending in _i (like myid_i, z_i)
+ Longer patterns will be matched first. if equal size patterns
+ both match, the first appearing in the schema will be used.
+ -->
+ <dynamicField name="*_i" type="int" indexed="true" stored="true"/>
+ <dynamicField name="*_d" type="double" indexed="true" stored="true"/>
+ <dynamicField name="*_s" type="string" indexed="true" stored="true"/>
+ <dynamicField name="*_sm" type="string" indexed="true" stored="true" multiValued="true"/>
+
+ <uniqueKey>id</uniqueKey>
+
+
+</schema>
diff --git a/solr/core/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml b/solr/contrib/scripting/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml
similarity index 78%
copy from solr/core/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml
copy to solr/contrib/scripting/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml
index 74f00fd..afa5a7c 100644
--- a/solr/core/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml
+++ b/solr/contrib/scripting/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml
@@ -18,7 +18,7 @@
-->
<!--
- Test Config that for ScriptUpdateProcessor
+ Test Config for ScriptUpdateProcessorFactory
-->
<config>
@@ -29,7 +29,7 @@
<schemaFactory class="ClassicIndexSchemaFactory"/>
<updateRequestProcessorChain name="force-script-engine" default="true">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="engine">javascript</str>
<str name="script">missleading.extension.updateprocessor.js.txt</str>
</processor>
@@ -42,7 +42,7 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="single-script">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">trivial.updateprocessor0.js</str>
<lst name="params">
<bool name="boolValue">true</bool>
@@ -53,7 +53,7 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="dual-scripts-arr">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<arr name="script">
<str>trivial.updateprocessor0.js</str>
<str>trivial.updateprocessor1.js</str>
@@ -67,7 +67,7 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="dual-scripts-strs">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">trivial.updateprocessor0.js</str>
<str name="script">trivial.updateprocessor1.js</str>
<lst name="params">
@@ -79,46 +79,46 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="conditional-scripts">
- <!-- multiple scripts,
+ <!-- multiple scripts,
test that the first one can conditionally stop execution -->
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">conditional.updateprocessor.js</str>
<str name="script">addfields.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="conditional-script">
- <!-- single script, followed by another processor
- (that happens to be a script).
+ <!-- single script, followed by another processor
+ (that happens to be a script).
test that the first one can conditionally stop execution -->
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">conditional.updateprocessor.js</str>
</processor>
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">addfields.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="error-on-add">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">throw.error.on.add.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="missing-functions">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">missing.functions.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="javascript-compatibility">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">cross-compatible.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="evil">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
<str name="script">evil.js</str>
</processor>
</updateRequestProcessorChain>
diff --git a/solr/contrib/scripting/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml b/solr/contrib/scripting/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
new file mode 100644
index 0000000..de5c714
--- /dev/null
+++ b/solr/contrib/scripting/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
@@ -0,0 +1,51 @@
+<?xml version="1.0" ?>
+
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<!--
+
+A solrconfig.xml snippet containing indexConfig settings for randomized testing.
+
+-->
+<indexConfig>
+ <!-- this sys property is not set by SolrTestCaseJ4 because we ideally want to use
+ the RandomMergePolicy in all tests - but some tests expect very specific
+ Merge behavior, so those tests can set it as needed.
+ -->
+ <mergePolicyFactory class="${solr.tests.mergePolicyFactory:org.apache.solr.util.RandomMergePolicyFactory}" />
+
+ <useCompoundFile>${useCompoundFile:false}</useCompoundFile>
+
+ <maxBufferedDocs>${solr.tests.maxBufferedDocs}</maxBufferedDocs>
+ <ramBufferSizeMB>${solr.tests.ramBufferSizeMB}</ramBufferSizeMB>
+ <maxCommitMergeWaitTime>${solr.tests.maxCommitMergeWaitTime:-1}</maxCommitMergeWaitTime>
+ <ramPerThreadHardLimitMB>${solr.tests.ramPerThreadHardLimitMB}</ramPerThreadHardLimitMB>
+
+ <mergeScheduler class="${solr.tests.mergeScheduler}" />
+
+ <writeLockTimeout>1000</writeLockTimeout>
+ <commitLockTimeout>10000</commitLockTimeout>
+
+ <!-- this sys property is not set by SolrTestCaseJ4 because almost all tests should
+ use the single process lockType for speed - but tests that explicitly need
+ to vary the lockType canset it as needed.
+ -->
+ <lockType>${solr.tests.lockType:single}</lockType>
+
+ <infoStream>${solr.tests.infostream:false}</infoStream>
+
+</indexConfig>
diff --git a/solr/core/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml b/solr/contrib/scripting/src/test-files/solr/collection1/conf/stateless-solrconfig-script-updateprocessor.xml
similarity index 79%
rename from solr/core/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/stateless-solrconfig-script-updateprocessor.xml
index 74f00fd..58fbb86 100644
--- a/solr/core/src/test-files/solr/collection1/conf/solrconfig-script-updateprocessor.xml
+++ b/solr/contrib/scripting/src/test-files/solr/collection1/conf/stateless-solrconfig-script-updateprocessor.xml
@@ -18,7 +18,7 @@
-->
<!--
- Test Config that for ScriptUpdateProcessor
+ Test Config for ScriptUpdateProcessorFactory
-->
<config>
@@ -29,7 +29,7 @@
<schemaFactory class="ClassicIndexSchemaFactory"/>
<updateRequestProcessorChain name="force-script-engine" default="true">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="engine">javascript</str>
<str name="script">missleading.extension.updateprocessor.js.txt</str>
</processor>
@@ -42,7 +42,7 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="single-script">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">trivial.updateprocessor0.js</str>
<lst name="params">
<bool name="boolValue">true</bool>
@@ -53,7 +53,7 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="dual-scripts-arr">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<arr name="script">
<str>trivial.updateprocessor0.js</str>
<str>trivial.updateprocessor1.js</str>
@@ -67,7 +67,7 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="dual-scripts-strs">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">trivial.updateprocessor0.js</str>
<str name="script">trivial.updateprocessor1.js</str>
<lst name="params">
@@ -79,22 +79,22 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="conditional-scripts">
- <!-- multiple scripts,
+ <!-- multiple scripts,
test that the first one can conditionally stop execution -->
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">conditional.updateprocessor.js</str>
<str name="script">addfields.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="conditional-script">
- <!-- single script, followed by another processor
- (that happens to be a script).
+ <!-- single script, followed by another processor
+ (that happens to be a script).
test that the first one can conditionally stop execution -->
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">conditional.updateprocessor.js</str>
</processor>
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">addfields.updateprocessor.js</str>
</processor>
</updateRequestProcessorChain>
@@ -112,13 +112,13 @@
</updateRequestProcessorChain>
<updateRequestProcessorChain name="javascript-compatibility">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">cross-compatible.js</str>
</processor>
</updateRequestProcessorChain>
<updateRequestProcessorChain name="evil">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
+ <processor class="org.apache.solr.update.processor.scripting.StatelessScriptUpdateProcessorFactory">
<str name="script">evil.js</str>
</processor>
</updateRequestProcessorChain>
diff --git a/solr/core/src/test-files/solr/collection1/conf/throw.error.on.add.updateprocessor.js b/solr/contrib/scripting/src/test-files/solr/collection1/conf/throw.error.on.add.updateprocessor.js
similarity index 100%
rename from solr/core/src/test-files/solr/collection1/conf/throw.error.on.add.updateprocessor.js
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/throw.error.on.add.updateprocessor.js
diff --git a/solr/core/src/test-files/solr/collection1/conf/trivial.updateprocessor0.js b/solr/contrib/scripting/src/test-files/solr/collection1/conf/trivial.updateprocessor0.js
similarity index 100%
rename from solr/core/src/test-files/solr/collection1/conf/trivial.updateprocessor0.js
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/trivial.updateprocessor0.js
diff --git a/solr/core/src/test-files/solr/collection1/conf/trivial.updateprocessor1.js b/solr/contrib/scripting/src/test-files/solr/collection1/conf/trivial.updateprocessor1.js
similarity index 100%
rename from solr/core/src/test-files/solr/collection1/conf/trivial.updateprocessor1.js
rename to solr/contrib/scripting/src/test-files/solr/collection1/conf/trivial.updateprocessor1.js
diff --git a/solr/core/src/test/org/apache/solr/update/processor/ScriptEngineTest.java b/solr/contrib/scripting/src/test/org/apache/solr/scripting/update/ScriptEngineTest.java
similarity index 98%
rename from solr/core/src/test/org/apache/solr/update/processor/ScriptEngineTest.java
rename to solr/contrib/scripting/src/test/org/apache/solr/scripting/update/ScriptEngineTest.java
index 8a913e1..e0bd83b 100644
--- a/solr/core/src/test/org/apache/solr/update/processor/ScriptEngineTest.java
+++ b/solr/contrib/scripting/src/test/org/apache/solr/scripting/update/ScriptEngineTest.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.solr.update.processor;
+package org.apache.solr.scripting.update;
import org.apache.lucene.util.Constants;
diff --git a/solr/core/src/test/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactoryTest.java b/solr/contrib/scripting/src/test/org/apache/solr/scripting/update/ScriptUpdateProcessorFactoryTest.java
similarity index 87%
rename from solr/core/src/test/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactoryTest.java
rename to solr/contrib/scripting/src/test/org/apache/solr/scripting/update/ScriptUpdateProcessorFactoryTest.java
index 09dd783..0a53db0 100644
--- a/solr/core/src/test/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactoryTest.java
+++ b/solr/contrib/scripting/src/test/org/apache/solr/scripting/update/ScriptUpdateProcessorFactoryTest.java
@@ -14,7 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.solr.update.processor;
+package org.apache.solr.scripting.update;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
@@ -25,24 +25,26 @@ import org.apache.solr.common.SolrException;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.core.SolrCore;
+import org.apache.solr.update.processor.UpdateProcessorTestBase;
+import org.apache.solr.update.processor.UpdateRequestProcessorChain;
import org.junit.Assume;
import org.junit.BeforeClass;
/**
- * Tests {@link StatelessScriptUpdateProcessorFactory}.
+ * Tests {@link ScriptUpdateProcessorFactory}.
*
- * TODO: This test, to run from an IDE, requires a working directory of <path-to>/solr/core/src/test-files. Fix!
+ * TODO: This test, to run from an IDE, requires a working directory of <path-to>/solr/contrib/scripting/src/test-files. Fix!
*/
-public class StatelessScriptUpdateProcessorFactoryTest extends UpdateProcessorTestBase {
+public class ScriptUpdateProcessorFactoryTest extends UpdateProcessorTestBase {
@BeforeClass
public static void beforeClass() throws Exception {
Assume.assumeNotNull((new ScriptEngineManager()).getEngineByExtension("js"));
- initCore("solrconfig-script-updateprocessor.xml", "schema12.xml");
+ initCore("solrconfig-script-updateprocessor.xml", "schema.xml");
}
/**
- * simple test of a basic script processor chain using the full
+ * simple test of a basic script processor chain using the full
* RequestHandler + UpdateProcessorChain flow
*/
public void testFullRequestHandlerFlow() throws Exception {
@@ -62,13 +64,13 @@ public class StatelessScriptUpdateProcessorFactoryTest extends UpdateProcessorTe
// clean up
processDeleteById("run-no-scripts","4055");
processCommit("run-no-scripts");
-
+
}
public void testSingleScript() throws Exception {
SolrCore core = h.getCore();
UpdateRequestProcessorChain chained = core.getUpdateProcessingChain("single-script");
- final StatelessScriptUpdateProcessorFactory factory = ((StatelessScriptUpdateProcessorFactory) chained.getProcessors().get(0));
+ final ScriptUpdateProcessorFactory factory = ((ScriptUpdateProcessorFactory) chained.getProcessors().get(0));
final List<String> functionMessages = new ArrayList<>();
factory.setScriptEngineCustomizer(new ScriptEngineCustomizer() {
@Override
@@ -91,7 +93,7 @@ public class StatelessScriptUpdateProcessorFactoryTest extends UpdateProcessorTe
processDeleteById("single-script","1");
processCommit("single-script");
-
+
assertQ("found deleted doc",
req("q","id:1")
, "//result[@numFound=0]");
@@ -108,12 +110,12 @@ public class StatelessScriptUpdateProcessorFactoryTest extends UpdateProcessorTe
public void testMultipleScripts() throws Exception {
SolrCore core = h.getCore();
- for (final String chain : new String[] {"dual-scripts-arr",
+ for (final String chain : new String[] {"dual-scripts-arr",
"dual-scripts-strs"}) {
-
+
UpdateRequestProcessorChain chained = core.getUpdateProcessingChain(chain);
- final StatelessScriptUpdateProcessorFactory factory =
- ((StatelessScriptUpdateProcessorFactory) chained.getProcessors().get(0));
+ final ScriptUpdateProcessorFactory factory =
+ ((ScriptUpdateProcessorFactory) chained.getProcessors().get(0));
final List<String> functionMessages = new ArrayList<>();
ScriptEngineCustomizer customizer = new ScriptEngineCustomizer() {
@Override
@@ -128,12 +130,12 @@ public class StatelessScriptUpdateProcessorFactoryTest extends UpdateProcessorTe
doc(f("id", "2"),
f("name", " foo "),
f("subject", "bar")));
-
- assertEquals(chain + " didn't add Double field",
+
+ assertEquals(chain + " didn't add Double field",
42.3d, d.getFieldValue("script_added_d"));
assertEquals(chain + " didn't add integer field",
42, d.getFieldValue("script_added_i"));
-
+
processCommit("run-no-scripts");
assertQ(chain + ": couldn't find doc by id",
@@ -142,72 +144,72 @@ public class StatelessScriptUpdateProcessorFactoryTest extends UpdateProcessorTe
processDeleteById(chain, "2");
processCommit(chain);
-
+
assertEquals(chain, 6, functionMessages.size());
assertTrue(chain, functionMessages.contains("processAdd0"));
assertTrue(chain, functionMessages.contains("processAdd1"));
assertTrue(chain + ": script order doesn't match conf order",
- functionMessages.indexOf("processAdd0")
+ functionMessages.indexOf("processAdd0")
< functionMessages.indexOf("processAdd1"));
assertTrue(chain, functionMessages.contains("processDelete0"));
assertTrue(chain, functionMessages.contains("processDelete1"));
assertTrue(chain + ": script order doesn't match conf order",
- functionMessages.indexOf("processDelete0")
+ functionMessages.indexOf("processDelete0")
< functionMessages.indexOf("processDelete1"));
assertTrue(chain, functionMessages.contains("processCommit0"));
assertTrue(chain, functionMessages.contains("processCommit1"));
assertTrue(chain + ": script order doesn't match conf order",
- functionMessages.indexOf("processCommit0")
+ functionMessages.indexOf("processCommit0")
< functionMessages.indexOf("processCommit1"));
finish(chain);
-
+
assertEquals(chain, 8, functionMessages.size());
assertTrue(chain, functionMessages.contains("finish0"));
assertTrue(chain, functionMessages.contains("finish1"));
assertTrue(chain + ": script order doesn't match conf order",
- functionMessages.indexOf("finish0")
+ functionMessages.indexOf("finish0")
< functionMessages.indexOf("finish1"));
assertQ(chain + ": found deleted doc",
req("q","id:2")
, "//result[@numFound=0]");
-
+
}
}
public void testConditionalExecution() throws Exception {
- for (String chain : new String[] {"conditional-script",
+ for (String chain : new String[] {"conditional-script",
"conditional-scripts"}) {
ModifiableSolrParams reqParams = new ModifiableSolrParams();
-
+
SolrInputDocument d = processAdd(chain,
reqParams,
doc(f("id", "3"),
f("name", " foo "),
f("subject", "bar")));
-
- assertFalse(chain + " added String field despite condition",
+
+ assertFalse(chain + " added String field despite condition",
d.containsKey("script_added_s"));
- assertFalse(chain + " added Double field despite condition",
+ assertFalse(chain + " added Double field despite condition",
d.containsKey("script_added_d"));
-
+
reqParams.add("go-for-it", "true");
-
+
d = processAdd(chain,
reqParams,
doc(f("id", "4"),
f("name", " foo "),
f("subject", "bar")));
-
- assertEquals(chain + " didn't add String field",
+
+ assertEquals(chain + " didn't add String field",
"i went for it", d.getFieldValue("script_added_s"));
- assertEquals(chain +" didn't add Double field",
+ assertEquals(chain +" didn't add Double field",
42.3d, d.getFieldValue("script_added_d"));
assertEquals(chain + " didn't add integer field",
42, d.getFieldValue("script_added_i"));
@@ -222,8 +224,8 @@ public class StatelessScriptUpdateProcessorFactoryTest extends UpdateProcessorTe
doc(f("id", "5"),
f("name", " foo "),
f("subject", "bar")));
-
- assertEquals(chain +" didn't add Double field",
+
+ assertEquals(chain +" didn't add Double field",
42.3d, d.getFieldValue("script_added_d"));
assertEquals(chain + " didn't add integer field",
42, d.getFieldValue("script_added_i"));
diff --git a/solr/contrib/scripting/src/test/org/apache/solr/scripting/update/TestBadScriptingUpdateProcessorConfig.java b/solr/contrib/scripting/src/test/org/apache/solr/scripting/update/TestBadScriptingUpdateProcessorConfig.java
new file mode 100644
index 0000000..9d64e8d
--- /dev/null
+++ b/solr/contrib/scripting/src/test/org/apache/solr/scripting/update/TestBadScriptingUpdateProcessorConfig.java
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.scripting.update;
+
+import javax.script.ScriptEngineManager;
+
+import org.apache.solr.core.AbstractBadConfigTestBase;
+import org.junit.Assume;
+
+public class TestBadScriptingUpdateProcessorConfig extends AbstractBadConfigTestBase {
+
+
+ public void testBogusScriptEngine() throws Exception {
+ // sanity check
+ Assume.assumeTrue(null == (new ScriptEngineManager()).getEngineByName("giberish"));
+
+ assertConfigs("bad-solrconfig-bogus-scriptengine-name.xml",
+ "schema.xml","giberish");
+ }
+
+ public void testMissingScriptFile() throws Exception {
+ // sanity check
+ Assume.assumeNotNull((new ScriptEngineManager()).getEngineByExtension("js"));
+ assertConfigs("bad-solrconfig-missing-scriptfile.xml",
+ "schema.xml","a-file-name-that-does-not-exist.js");
+ }
+
+ public void testInvalidScriptFile() throws Exception {
+ // sanity check
+ Assume.assumeNotNull((new ScriptEngineManager()).getEngineByName("javascript"));
+ assertConfigs("bad-solrconfig-invalid-scriptfile.xml",
+ "schema.xml","invalid.script.xml");
+ }
+
+}
diff --git a/solr/core/src/test-files/solr/collection1/conf/missleading.extension.updateprocessor.js.txt b/solr/core/src/test-files/solr/collection1/conf/missleading.extension.updateprocessor.js.txt
deleted file mode 100644
index 984e1d8..0000000
--- a/solr/core/src/test-files/solr/collection1/conf/missleading.extension.updateprocessor.js.txt
+++ /dev/null
@@ -1,23 +0,0 @@
-function processAdd(cmd) {
- // Integer.valueOf is needed here to get a tru java object, because
- // all javascript numbers are floating point (ie: java.lang.Double)
- cmd.getSolrInputDocument().addField("script_added_i",
- java.lang.Integer.valueOf(42));
- cmd.getSolrInputDocument().addField("script_added_d", 42.3);
-
-}
-function processDelete() {
- // NOOP
-}
-function processCommit() {
- // NOOP
-}
-function processRollback() {
- // NOOP
-}
-function processMergeIndexes() {
- // NOOP
-}
-function finish() {
- // NOOP
-}
diff --git a/solr/core/src/test-files/solr/configsets/upload/with-script-processor/solrconfig.xml b/solr/core/src/test-files/solr/configsets/upload/with-script-processor/solrconfig.xml
index 1c62889..affccbf 100644
--- a/solr/core/src/test-files/solr/configsets/upload/with-script-processor/solrconfig.xml
+++ b/solr/core/src/test-files/solr/configsets/upload/with-script-processor/solrconfig.xml
@@ -37,12 +37,9 @@
<luceneMatchVersion>${tests.luceneMatchVersion:LATEST}</luceneMatchVersion>
- <updateHandler class="solr.DirectUpdateHandler2">
- <commitWithin>
- <softCommit>${solr.commitwithin.softcommit:true}</softCommit>
- </commitWithin>
+ <lib dir="${solr.install.dir:../../../../../}/contrib/scripting/build/libs/" regex="solr-scripting-\d.*\.jar" />
+
- </updateHandler>
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
@@ -55,11 +52,10 @@
<updateRequestProcessorChain name="force-script-engine" default="true">
<processor class="solr.StatelessScriptUpdateProcessorFactory">
<str name="engine">javascript</str>
- <str name="script">missleading.extension.updateprocessor.js.txt</str>
+ <str name="script">trivial.updateprocessor.js</str>
</processor>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
<requestHandler name="/update" class="solr.UpdateRequestHandler" />
</config>
-
diff --git a/solr/core/src/test-files/solr/configsets/upload/with-script-processor/trivial.updateprocessor.js b/solr/core/src/test-files/solr/configsets/upload/with-script-processor/trivial.updateprocessor.js
new file mode 100644
index 0000000..698b78c
--- /dev/null
+++ b/solr/core/src/test-files/solr/configsets/upload/with-script-processor/trivial.updateprocessor.js
@@ -0,0 +1,22 @@
+function processAdd(cmd) {
+ // Integer.valueOf is needed here to get a tru java object, because
+ // all javascript numbers are floating point (ie: java.lang.Double)
+ cmd.getSolrInputDocument().addField("script_added_i",
+ java.lang.Integer.valueOf(42));
+}
+
+function processDelete() {
+ // NOOP
+}
+function processCommit() {
+ // NOOP
+}
+function processRollback() {
+ // NOOP
+}
+function processMergeIndexes() {
+ // NOOP
+}
+function finish() {
+ // NOOP
+}
diff --git a/solr/core/src/test/org/apache/solr/core/TestBadConfig.java b/solr/core/src/test/org/apache/solr/core/TestBadConfig.java
index 1dfad85..91fd9ae 100644
--- a/solr/core/src/test/org/apache/solr/core/TestBadConfig.java
+++ b/solr/core/src/test/org/apache/solr/core/TestBadConfig.java
@@ -16,10 +16,6 @@
*/
package org.apache.solr.core;
-import javax.script.ScriptEngineManager;
-
-import org.junit.Assume;
-
public class TestBadConfig extends AbstractBadConfigTestBase {
public void testUnsetSysProperty() throws Exception {
@@ -44,7 +40,7 @@ public class TestBadConfig extends AbstractBadConfigTestBase {
}
public void testUpdateLogButNoVersionField() throws Exception {
-
+
System.setProperty("enable.update.log", "true");
try {
assertConfigs("solrconfig.xml", "schema12.xml", "_version_");
@@ -53,28 +49,6 @@ public class TestBadConfig extends AbstractBadConfigTestBase {
}
}
- public void testBogusScriptEngine() throws Exception {
- // sanity check
- Assume.assumeTrue(null == (new ScriptEngineManager()).getEngineByName("giberish"));
-
- assertConfigs("bad-solrconfig-bogus-scriptengine-name.xml",
- "schema.xml","giberish");
- }
-
- public void testMissingScriptFile() throws Exception {
- // sanity check
- Assume.assumeNotNull((new ScriptEngineManager()).getEngineByExtension("js"));
- assertConfigs("bad-solrconfig-missing-scriptfile.xml",
- "schema.xml","a-file-name-that-does-not-exist.js");
- }
-
- public void testInvalidScriptFile() throws Exception {
- // sanity check
- Assume.assumeNotNull((new ScriptEngineManager()).getEngineByName("javascript"));
- assertConfigs("bad-solrconfig-invalid-scriptfile.xml",
- "schema.xml","currency.xml");
- }
-
public void testBogusMergePolicy() throws Exception {
assertConfigs("bad-mpf-solrconfig.xml", "schema-minimal.xml",
"DummyMergePolicyFactory");
@@ -89,7 +63,7 @@ public class TestBadConfig extends AbstractBadConfigTestBase {
assertConfigs("bad-solrconfig-managed-schema-named-schema.xml.xml",
"schema-minimal.xml", "managedSchemaResourceName can't be 'schema.xml'");
}
-
+
public void testUnknownSchemaAttribute() throws Exception {
assertConfigs("bad-solrconfig-unexpected-schema-attribute.xml", "schema-minimal.xml",
"Unexpected arg(s): {bogusParam=bogusValue}");
diff --git a/solr/packaging/build.gradle b/solr/packaging/build.gradle
index 27e8e3a..3b242ad 100644
--- a/solr/packaging/build.gradle
+++ b/solr/packaging/build.gradle
@@ -52,6 +52,7 @@ dependencies {
":solr:contrib:langid",
":solr:contrib:ltr",
":solr:contrib:prometheus-exporter",
+ ":solr:contrib:scripting"
].each { contribName ->
distSolr project(contribName)
contrib project(path: contribName, configuration: "packaging")
diff --git a/solr/server/solr/configsets/sample_techproducts_configs/conf/solrconfig.xml b/solr/server/solr/configsets/sample_techproducts_configs/conf/solrconfig.xml
index 8fea9ca..14f094d 100644
--- a/solr/server/solr/configsets/sample_techproducts_configs/conf/solrconfig.xml
+++ b/solr/server/solr/configsets/sample_techproducts_configs/conf/solrconfig.xml
@@ -83,6 +83,8 @@
<lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-ltr-\d.*\.jar" />
+ <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-scripting-\d.*\.jar" />
+
<!-- an exact 'path' can be used instead of a 'dir' to specify a
specific jar file. This will cause a serious error to be logged
if it can't be loaded.
@@ -379,11 +381,11 @@
<query>
<!-- Maximum number of clauses allowed when parsing a boolean query string.
-
+
This limit only impacts boolean queries specified by a user as part of a query string,
and provides per-collection controls on how complex user specified boolean queries can
be. Query strings that specify more clauses then this will result in an error.
-
+
If this per-collection limit is greater then the global `maxBooleanClauses` limit
specified in `solr.xml`, it will have no effect, as that setting also limits the size
of user specified boolean queries.
@@ -656,6 +658,9 @@
enableRemoteStreaming - enables use of the stream.file
and stream.url parameters for specifying remote streams.
+ enableStreamBody - This attribute controls whether streaming
+ content from the HTTP parameter stream.body is allowed.
+
multipartUploadLimitInKB - specifies the max size (in KiB) of
Multipart File Uploads that Solr will allow in a Request.
@@ -674,12 +679,12 @@
*** WARNING ***
Before enabling remote streaming, you should make sure your
system has authentication enabled.
-
- <requestParsers enableRemoteStreaming="false"
+ -->
+ <requestParsers enableRemoteStreaming="true"
+ enableStreamBody="true"
multipartUploadLimitInKB="-1"
formdataUploadLimitInKB="-1"
addHttpRequestToContext="false"/>
- -->
<!-- HTTP Caching
@@ -1273,19 +1278,17 @@
This example hooks in an update processor implemented using JavaScript.
- See more about the script update processor at http://wiki.apache.org/solr/ScriptUpdateProcessor
- -->
- <!--
- <updateRequestProcessorChain name="script">
- <processor class="solr.StatelessScriptUpdateProcessorFactory">
- <str name="script">update-script.js</str>
- <lst name="params">
- <str name="config_param">example config parameter</str>
- </lst>
- </processor>
- <processor class="solr.RunUpdateProcessorFactory" />
- </updateRequestProcessorChain>
+ See more about script update processor at https://lucene.apache.org/solr/guide/script-update-processor.html
-->
+ <updateRequestProcessorChain name="script">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
+ <str name="script">update-script.js</str>
+ <lst name="params">
+ <str name="config_param">example config parameter</str>
+ </lst>
+ </processor>
+ <processor class="solr.RunUpdateProcessorFactory" />
+ </updateRequestProcessorChain>
<!-- Response Writers
diff --git a/solr/server/solr/configsets/sample_techproducts_configs/conf/update-script.js b/solr/server/solr/configsets/sample_techproducts_configs/conf/update-script.js
index 49b07f9..bd36118 100644
--- a/solr/server/solr/configsets/sample_techproducts_configs/conf/update-script.js
+++ b/solr/server/solr/configsets/sample_techproducts_configs/conf/update-script.js
@@ -4,14 +4,14 @@
In order for this to be executed, it must be properly wired into solrconfig.xml; by default it is commented out in
the example solrconfig.xml and must be uncommented to be enabled.
- See http://wiki.apache.org/solr/ScriptUpdateProcessor for more details.
+ See https://lucene.apache.org/solr/guide/script-update-processor.html for more details.
*/
function processAdd(cmd) {
doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument
id = doc.getFieldValue("id");
- logger.info("update-script#processAdd: id=" + id);
+ logger.warn("update-script#processAdd: id=" + id); // WARN level messages will show up in Solr Admin Logging UI
// Set a field value:
// doc.setField("foo_s", "whatever");
diff --git a/solr/solr-ref-guide/src/configsets-api.adoc b/solr/solr-ref-guide/src/configsets-api.adoc
index ead84f1..d5fd3dc 100644
--- a/solr/solr-ref-guide/src/configsets-api.adoc
+++ b/solr/solr-ref-guide/src/configsets-api.adoc
@@ -88,7 +88,7 @@ This functionality is enabled by default, but can be disabled via a runtime para
A configset is uploaded in a "trusted" mode if authentication is enabled and the upload operation is performed as an authenticated request. Without authentication, a configset is uploaded in an "untrusted" mode. Upon creation of a collection using an "untrusted" configset, the following functionality will not work:
* The XSLT transformer (`tr` parameter) cannot be used at request processing time.
-* If specified in the configset, the StatelessScriptUpdateProcessor will not initialize.
+* If specified in the configset, the ScriptUpdateProcessorFactory will not initialize.
* Collections won't initialize if <lib> directives are used in the configset. (Note: Libraries added to Solr's classpath don't need the <lib> directive)
If you use any of these parameters or features, you must have enabled security features in your Solr installation and you must upload the configset as an authenticated user.
diff --git a/solr/solr-ref-guide/src/configuring-solrconfig-xml.adoc b/solr/solr-ref-guide/src/configuring-solrconfig-xml.adoc
index f1f0fa2..4e83f0e 100644
--- a/solr/solr-ref-guide/src/configuring-solrconfig-xml.adoc
+++ b/solr/solr-ref-guide/src/configuring-solrconfig-xml.adoc
@@ -8,6 +8,7 @@
query-settings-in-solrconfig, \
requestdispatcher-in-solrconfig, \
update-request-processors, \
+ script-update-processor, \
codec-factory
// Licensed to the Apache Software Foundation (ASF) under one
diff --git a/solr/solr-ref-guide/src/major-changes-in-solr-9.adoc b/solr/solr-ref-guide/src/major-changes-in-solr-9.adoc
index 60ab785..e554c15 100644
--- a/solr/solr-ref-guide/src/major-changes-in-solr-9.adoc
+++ b/solr/solr-ref-guide/src/major-changes-in-solr-9.adoc
@@ -144,6 +144,9 @@ _(raw; not yet edited)_
* SOLR-14972: The default port of prometheus exporter has changed from 9983 to 8989, so you may need to adjust your configuration after upgrade.
+* SOLR-14067: StatelessScriptUpdateProcessorFactory moved to it's own /contrib/scripting/ package instead
+ of shipping as part of Solr due to security concerns. Renamed to ScriptUpdateProcessorFactory for simpler name.
+
=== Upgrade Prerequisites in Solr 9
* Upgrade all collections in stateFormat=1 to stateFormat=2 *before* upgrading to Solr 9, as Solr 9 does not support the
diff --git a/solr/solr-ref-guide/src/script-update-processor.adoc b/solr/solr-ref-guide/src/script-update-processor.adoc
new file mode 100644
index 0000000..149f201
--- /dev/null
+++ b/solr/solr-ref-guide/src/script-update-processor.adoc
@@ -0,0 +1,286 @@
+= Script Update Processor
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+The {solr-javadocs}/contrib/scripting/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.html[ScriptUpdateProcessorFactory] allows Java scripting engines to be used
+during Solr document update processing, allowing dramatic flexibility in
+expressing custom document processing logic before being indexed. It has hooks to the
+commit, delete, rollback, etc indexing actions, however add is the most common usage.
+It is implemented as an UpdateProcessor to be placed in an UpdateChain.
+
+TIP: This used to be known as the _StatelessScriptingUpdateProcessor_ and was renamed to clarify the key aspect of this update processor is it enables scripting.
+
+The script can be written in any scripting language supported by your JVM (such
+as JavaScript), and executed dynamically so no pre-compilation is necessary.
+
+WARNING: Being able to run a script of your choice as part of the indexing pipeline is a really powerful tool, that I sometimes call the
+_Get out of jail free_ card because you can solve some problems this way that you can't in any other way. However, you are introducing some
+potential security vulnerabilities.
+
+== Installing the ScriptingUpdateProcessor and Scripting Engines
+
+The scripting update processor lives in the contrib module `/contrib/scripting`, and you need to explicitly add it to your Solr setup.
+
+Java 11 and previous versions come with a JavaScript engine called Nashorn, but Java 12 will require you to add your own JavaScript engine. Other supported scripting engines like
+JRuby, Jython, Groovy, all require you to add JAR files.
+
+Learn more about adding the `dist/solr-scripting-*.jar` file, and any other needed JAR files (depending on your scripting engine) into Solr's <<libs.adoc#lib-directories,Lib Directories>>.
+
+== Configuration
+
+[source,xml]
+----
+<updateRequestProcessorChain name="script">
+ <processor class="org.apache.solr.scripting.update.ScriptUpdateProcessorFactory">
+ <str name="script">update-script.js</str>
+ </processor>
+ <!-- optional parameters passed to script
+ <lst name="params">
+ <str name="config_param">example config parameter</str>
+ </lst>
+ -->
+ <processor class="solr.LogUpdateProcessorFactory" />
+ <processor class="solr.RunUpdateProcessorFactory" />
+ </updateRequestProcessorChain>
+----
+
+NOTE: The processor supports the defaults/appends/invariants concept for its config.
+However, it is also possible to skip this level and configure the parameters directly underneath the `<processor>` tag.
+
+Below follows a list of each configuration parameters and their meaning:
+
+`script`::
+The script file name. The script file must be placed in the `conf/ directory.
+There can be one or more "script" parameters specified; multiple scripts are executed in the order specified.
+
+`engine`::
+Optionally specifies the scripting engine to use. This is only needed if the extension
+of the script file is not a standard mapping to the scripting engine. For example, if your
+script file was coded in JavaScript but the file name was called `update-script.foo`,
+use "javascript" as the engine name.
+
+`params`::
+Optional parameters that are passed into the script execution context. This is
+specified as a named list (`<lst>`) structure with nested typed parameters. If
+specified, the script context will get a "params" object, otherwise there will be no "params" object available.
+
+
+== Script execution context
+
+Every script has some variables provided to it.
+
+`logger`::
+Logger (org.slf4j.Logger) instance. This is useful for logging information from the script.
+
+`req`::
+{solr-javadocs}/core/org/apache/solr/request/SolrQueryRequest.html[SolrQueryRequest] instance.
+
+`rsp`::
+{solr-javadocs}/core/org/apache/solr/response/SolrQueryResponse.html[SolrQueryResponse] instance.
+
+`params`::
+The "params" object, if any specified, from the configuration.
+
+== Examples
+
+The `processAdd()` and the other script methods can return false to skip further
+processing of the document. All methods must be defined, though generally the
+`processAdd()` method is where the action is.
+
+Here's a URL that works with the techproducts example setup demonstrating specifying
+the "script" update chain: `http://localhost:8983/solr/techproducts/update?commit=true&stream.contentType=text/csv&fieldnames=id,description&stream.body=1,foo&update.chain=script`
+which logs the following:
+
+[source,text]
+----
+INFO: update-script#processAdd: id=1
+----
+
+You can see the message recorded in the Solr logging UI.
+
+=== Javascript
+
+Note: There is a JavaScript example `update-script.js` as part of the `techproducts` configset.
+Check `solrconfig.xml` and uncomment the update request processor definition to enable this feature.
+
+[source,javascript]
+----
+function processAdd(cmd) {
+
+ doc = cmd.solrDoc; // org.apache.solr.common.SolrInputDocument
+ id = doc.getFieldValue("id");
+ logger.info("update-script#processAdd: id=" + id);
+
+// Set a field value:
+// doc.setField("foo_s", "whatever");
+
+// Get a configuration parameter:
+// config_param = params.get('config_param'); // "params" only exists if processor configured with <lst name="params">
+
+// Get a request parameter:
+// some_param = req.getParams().get("some_param")
+
+// Add a field of field names that match a pattern:
+// - Potentially useful to determine the fields/attributes represented in a result set, via faceting on field_name_ss
+// field_names = doc.getFieldNames().toArray();
+// for(i=0; i < field_names.length; i++) {
+// field_name = field_names[i];
+// if (/attr_.*/.test(field_name)) { doc.addField("attribute_ss", field_names[i]); }
+// }
+
+}
+
+function processDelete(cmd) {
+ // no-op
+}
+
+function processMergeIndexes(cmd) {
+ // no-op
+}
+
+function processCommit(cmd) {
+ // no-op
+}
+
+function processRollback(cmd) {
+ // no-op
+}
+
+function finish() {
+ // no-op
+}
+----
+
+=== Ruby
+Ruby support is implemented via the https://www.jruby.org/[JRuby] project.
+To use JRuby as the scripting engine, add `jruby.jar` to Solr.
+
+Here's an example of a JRuby update processing script (note that all variables passed in require prefixing with `$`, such as `$logger`):
+
+[source,ruby]
+----
+def processAdd(cmd)
+ doc = cmd.solrDoc # org.apache.solr.common.SolrInputDocument
+ id = doc.getFieldValue('id')
+
+ $logger.info "update-script#processAdd: id=#{id}"
+
+ doc.setField('source_s', 'ruby')
+
+ $logger.info "update-script#processAdd: config_param=#{$params.get('config_param')}"
+end
+
+def processDelete(cmd)
+ # no-op
+end
+
+def processMergeIndexes(cmd)
+ # no-op
+end
+
+def processCommit(cmd)
+ # no-op
+end
+
+def processRollback(cmd)
+ # no-op
+end
+
+def finish()
+ # no-op
+end
+----
+
+==== Known issues
+
+The following in JRuby does not work as expected for some reason, though it does work properly in JavaScript:
+
+[source,ruby]
+----
+# $logger.info "update-script#processAdd: request_param=#{$req.params.get('request_param')}"
+# $rsp.add('script_processed',id)
+----
+
+=== Groovy
+
+Add JARs from a Groovy distro's `lib/` directory to Solr. All JARs from
+Groovy's distro probably aren't required, but more than just the main `groovy.jar`
+file is needed (at least when this was tested using Groovy 2.0.6)
+
+[source,groovy]
+----
+def processAdd(cmd) {
+ doc = cmd.solrDoc // org.apache.solr.common.SolrInputDocument
+ id = doc.getFieldValue('id')
+
+ logger.info "update-script#processAdd: id=" + id
+
+ doc.setField('source_s', 'groovy')
+
+ logger.info "update-script#processAdd: config_param=" + params.get('config_param')
+
+ logger.info "update-script#processAdd: request_param=" + req.params.get('request_param')
+ rsp.add('script_processed',id)
+}
+
+def processDelete(cmd) {
+ // no-op
+}
+
+def processMergeIndexes(cmd) {
+ // no-op
+}
+
+def processCommit(cmd) {
+ // no-op
+}
+
+def processRollback(cmd) {
+ // no-op
+}
+
+def finish() {
+ // no-op
+}
+----
+
+=== Python
+Python support is implemented via the https://www.jython.org/[Jython] project.
+Add the *standalone* `jython.jar` (the JAR that contains all the dependencies) into Solr.
+
+[source,python]
+----
+def processAdd(cmd):
+ doc = cmd.solrDoc
+ id = doc.getFieldValue("id")
+ logger.info("update-script#processAdd: id=" + id)
+
+def processDelete(cmd):
+ logger.info("update-script#processDelete")
+
+def processMergeIndexes(cmd):
+ logger.info("update-script#processMergeIndexes")
+
+def processCommit(cmd):
+ logger.info("update-script#processCommit")
+
+def processRollback(cmd):
+ logger.info("update-script#processRollback")
+
+def finish():
+ logger.info("update-script#finish")
+----
diff --git a/solr/solr-ref-guide/src/update-request-processors.adoc b/solr/solr-ref-guide/src/update-request-processors.adoc
index 7e9dce8..aeee353 100644
--- a/solr/solr-ref-guide/src/update-request-processors.adoc
+++ b/solr/solr-ref-guide/src/update-request-processors.adoc
@@ -283,7 +283,7 @@ What follows are brief descriptions of the currently available update request pr
{solr-javadocs}/core/org/apache/solr/update/processor/SignatureUpdateProcessorFactory.html[SignatureUpdateProcessorFactory]:: Uses a defined set of fields to generate a hash "signature" for the document. Useful for only indexing one copy of "similar" documents.
-{solr-javadocs}/core/org/apache/solr/update/processor/StatelessScriptUpdateProcessorFactory.html[StatelessScriptUpdateProcessorFactory]:: An update request processor factory that enables the use of update processors implemented as scripts.
+{solr-javadocs}/contrib/scripting/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.html[ScriptUpdateProcessorFactory]:: An processor that enables the use of update processors implemented as scripts. Learn more at the <<script-update-processor.adoc#script-update-processor,script update processor>> page.
{solr-javadocs}/core/org/apache/solr/update/processor/TemplateUpdateProcessorFactory.html[TemplateUpdateProcessorFactory]:: Allows adding new fields to documents based on a template pattern. This update processor can also be used at runtime (without defining it in `solrconfig.xml`), see the section <<templateupdateprocessorfactory>> below.
diff --git a/solr/test-framework/src/java/org/apache/solr/update/processor/UpdateProcessorTestBase.java b/solr/test-framework/src/java/org/apache/solr/update/processor/UpdateProcessorTestBase.java
index d3aa979..21c4972 100644
--- a/solr/test-framework/src/java/org/apache/solr/update/processor/UpdateProcessorTestBase.java
+++ b/solr/test-framework/src/java/org/apache/solr/update/processor/UpdateProcessorTestBase.java
@@ -140,7 +140,7 @@ public class UpdateProcessorTestBase extends SolrTestCaseJ4 {
/**
* Convenience method for building up SolrInputDocuments
*/
- final SolrInputDocument doc(SolrInputField... fields) {
+ protected final SolrInputDocument doc(SolrInputField... fields) {
SolrInputDocument d = new SolrInputDocument();
for (SolrInputField f : fields) {
d.put(f.getName(), f);
@@ -162,7 +162,7 @@ public class UpdateProcessorTestBase extends SolrTestCaseJ4 {
/**
* Convenience method for building up SolrInputFields with default boost
*/
- final SolrInputField f(String name, Object... values) {
+ protected final SolrInputField f(String name, Object... values) {
return field(name, values);
}
}