You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by sp...@apache.org on 2017/03/03 20:08:46 UTC
[33/46] tinkerpop git commit: TINKERPOP-1612 Re-write server docs
given new plugin model.
TINKERPOP-1612 Re-write server docs given new plugin model.
Project: http://git-wip-us.apache.org/repos/asf/tinkerpop/repo
Commit: http://git-wip-us.apache.org/repos/asf/tinkerpop/commit/29ed887e
Tree: http://git-wip-us.apache.org/repos/asf/tinkerpop/tree/29ed887e
Diff: http://git-wip-us.apache.org/repos/asf/tinkerpop/diff/29ed887e
Branch: refs/heads/master
Commit: 29ed887e63b93f151104f4a270a5c369b015ce4a
Parents: b4472c6
Author: Stephen Mallette <sp...@genoprime.com>
Authored: Mon Feb 27 19:20:42 2017 -0500
Committer: Stephen Mallette <sp...@genoprime.com>
Committed: Wed Mar 1 16:06:16 2017 -0500
----------------------------------------------------------------------
.../src/reference/gremlin-applications.asciidoc | 90 ++++++++++----------
1 file changed, 43 insertions(+), 47 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/tinkerpop/blob/29ed887e/docs/src/reference/gremlin-applications.asciidoc
----------------------------------------------------------------------
diff --git a/docs/src/reference/gremlin-applications.asciidoc b/docs/src/reference/gremlin-applications.asciidoc
index e620372..f3db936 100644
--- a/docs/src/reference/gremlin-applications.asciidoc
+++ b/docs/src/reference/gremlin-applications.asciidoc
@@ -406,12 +406,9 @@ $ bin/gremlin-server.sh conf/gremlin-server-modern.yaml
[INFO] MetricManager - Configured Metrics Slf4jReporter configured with interval=180000ms and loggerName=org.apache.tinkerpop.gremlin.server.Settings$Slf4jReporterMetrics
[INFO] GraphManager - Graph [graph] was successfully configured via [conf/tinkergraph-empty.properties].
[INFO] ServerGremlinExecutor - Initialized Gremlin thread pool. Threads in pool named with pattern gremlin-*
-[INFO] ScriptEngines - Loaded gremlin-groovy ScriptEngine
-[INFO] GremlinExecutor - Initialized gremlin-groovy ScriptEngine with scripts/generate-modern.groovy
[INFO] ServerGremlinExecutor - Initialized GremlinExecutor and configured ScriptEngines.
[INFO] ServerGremlinExecutor - A GraphTraversalSource is now bound to [g] with graphtraversalsource[tinkergraph[vertices:0 edges:0], standard]
[INFO] OpLoader - Adding the standard OpProcessor.
-[INFO] OpLoader - Adding the control OpProcessor.
[INFO] OpLoader - Adding the session OpProcessor.
[INFO] OpLoader - Adding the traversal OpProcessor.
[INFO] TraversalOpProcessor - Initialized cache for TraversalOpProcessor with size 1000 and expiration time of 600000 ms
@@ -420,7 +417,11 @@ $ bin/gremlin-server.sh conf/gremlin-server-modern.yaml
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo-lite with org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0
[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+gryo-stringd with org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0
-[INFO] GremlinServer$1 - Gremlin Server configured with worker thread pool of 1, gremlin pool of 8 and boss thread pool of 1.
+[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v1.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0
+[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v2.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0
+[INFO] AbstractChannelizer - Configured application/vnd.gremlin-v3.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0
+[INFO] AbstractChannelizer - Configured application/json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0
+[INFO] GremlinServer$1 - Gremlin Server configured with worker thread pool of 1, gremlin pool of 4 and boss thread pool of 1.
[INFO] GremlinServer$1 - Channel started at port 8182.
----
@@ -1141,7 +1142,6 @@ The following table describes the various YAML configuration options that Gremli
|metrics.jmxReporter.enabled |Turns on JMX reporting of metrics. |false
|metrics.slf4jReporter.enabled |Turns on SLF4j reporting of metrics. |false
|metrics.slf4jReporter.interval |Time in milliseconds between reports of metrics to SLF4j. |180000
-|plugins |A list of plugins that should be activated on server startup in the available script engines. It assumes that the plugins are in Gremlin Server's classpath. |_none_
|port |The port to bind the server to. |8182
|processors |A `List` of `Map` settings, where each `Map` represents a `OpProcessor` implementation to use along with its configuration. |_none_
|processors[X].className |The full class name of the `OpProcessor` implementation. |_none_
@@ -1404,7 +1404,7 @@ export JAVA_OPTIONS="$JAVA_OPTIONS -Djava.security.auth.login.config=conf/gremli
Protecting Script Execution
+++++++++++++++++++++++++++
-It is important to remember that Gremlin Server exposes a `ScriptEngine` instance that allows for remote execution
+It is important to remember that Gremlin Server exposes `GremlinScriptEngine` instances that allows for remote execution
of arbitrary code on the server. Obviously, this situation can represent a security risk or, more minimally, provide
ways for "bad" scripts to be inadvertently executed. A simple example of a "valid" Gremlin script that would cause
some problems would be, `while(true) {}`, which would consume a thread in the Gremlin pool indefinitely, thus
@@ -1412,30 +1412,33 @@ preventing it from serving other requests. Sending enough of these kinds of scr
available threads and Gremlin Server would stop responding.
Gremlin Server (more specifically the `GremlinGroovyScriptEngine`) provides methods to protect itself from these
-kinds of troublesome scripts. A user can configure the script engine with different `CompilerCustomizerProvider`
-implementations. Consider the basic configuration from the Gremlin Server YAML file:
+kinds of troublesome scripts. A user can configure the script engine with a `GroovyCompilerGremlinPlugin`
+implementation. Consider the basic configuration from the Gremlin Server YAML file:
[source,yaml]
+----
scriptEngines: {
gremlin-groovy: {
- imports: [java.lang.Math],
- staticImports: [java.lang.Math.PI],
- scripts: [scripts/empty-sample.groovy]}}
+ plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},
+ org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {},
+ org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
+ org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample.groovy]}}}}
+----
-This configuration can be extended to include a `config` key as follows:
+This configuration can be expaded to include a the `GroovyCompilerGremlinPlugin`:
[source,yaml]
+----
scriptEngines: {
gremlin-groovy: {
- imports: [java.lang.Math],
- staticImports: [java.lang.Math.PI],
- scripts: [scripts/empty-sample.groovy],
- config: {
- compilerCustomizerProviders: {
- "org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.TimedInterruptCustomizerProvider":[10000] }}}
-
-This configuration sets up the script engine with a `CompilerCustomizerProvider` implementation. The
-`TimedInterruptCustomizerProvider` injects checks that ensure that loops (like `while`) can only execute for `10000`
+ plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},
+ org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {}
+ org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
+ org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample-secure.groovy]},
+ org.apache.tinkerpop.gremlin.groovy.jsr223.GroovyCompilerGremlinPlugin: {timedInterrupt: 10000}}}}
+----
+
+This configuration sets up the script engine with to ensure that loops (like `while`) can only execute for `10000`
milliseconds. With this configuration in place, a remote execution as follows, now times out rather than consuming
the thread continuously:
@@ -1445,41 +1448,34 @@ gremlin> :remote connect tinkerpop.server conf/remote.yaml
gremlin> :> while(true) { }
Execution timed out after 10000 units. Start time: Fri Jul 24 11:04:52 EDT 2015
-There are a number of pre-packaged `CustomizerProvider` implementations:
+The `GroovyCompilerGremlinPlugin` has a number of configuration options:
[width="100%",cols="3,10a",options="header"]
|=========================================================
|Customizer |Description
-|`CompileStaticCustomizerProvider` |Applies `CompileStatic` annotations to incoming scripts thus removing dynamic dispatch. More information about static compilation can be found in the link:http://docs.groovy-lang.org/latest/html/documentation/#_static_compilation[Groovy Documentation]. It is possible to configure this `CustomizerProvider` by specifying a comma separated list of link:http://docs.groovy-lang.org/latest/html/documentation/#Typecheckingextensions-Workingwithextensions[type checking extensions] that can have the effect of securing calls to various methods.
-|`ConfigurationCustomizerProvider` |Allows configuration of the Groovy `CompilerConfiguration` object by taking a `Map` of key/value pairs where the "key" is a property to set on the `CompilerConfiguration`.
-|`ThreadInterruptCustomizerProvider` |Injects checks for thread interruption, thus allowing the thread to potentially respect calls to `Thread.interrupt()`
-|`TimedInterruptCustomizerProvider` |Injects checks into loops to interrupt them if they exceed the configured timeout in milliseconds.
-|`TypeCheckedCustomizerProvider` |Similar to the above mentioned, `CompileStaticCustomizerProvider`, the `TypeCheckedCustomizerProvider` injects `TypeChecked` annotations to incoming scripts. More information on the nature of this annotation can be found in the link:http://docs.groovy-lang.org/latest/html/documentation/#_the_code_typechecked_code_annotation[Groovy Documentation]. It too takes a comma separated list of link:http://docs.groovy-lang.org/latest/html/documentation/#Typecheckingextensions-Workingwithextensions[type checking extensions].
+|`compilation` |Allows for three configurations: `COMPILE_STATIC`, `TYPE_CHECKED` or `NONE` (default). When configured with `COMPILE_STATIC` or `TYPE_CHECKED` it applies `CompileStatic` or `TypeChecked` annotations (respectively) to incoming scripts thus removing dynamic dispatch. More information about static compilation can be found link:http://docs.groovy-lang.org/latest/html/documentation/#_static_compilation[here] and additional information on `TypeChecked` usage can be found link:http://docs.groovy-lang.org/latest/html/documentation/#_the_code_typechecked_code_annotation[here].
+|`compilerConfigurationOptions` |Allows configuration of the Groovy `CompilerConfiguration` object by taking a `Map` of key/value pairs where the "key" is a property to set on the `CompilerConfiguration`.
+|`enableThreadInterrupt` |Injects checks for thread interruption, thus allowing the thread to potentially respect calls to `Thread.interrupt()`
+|`extensions` | This setting is for use when `compilation` is configured with `COMPILE_STATIC` or `TYPE_CHECKED` and accepts a comma separated list of link:http://docs.groovy-lang.org/latest/html/documentation/#Typecheckingextensions-Workingwithextensions[type checking extensions] that can have the effect of securing calls to various methods.
+|`timedInterrupt` |Injects checks into loops to interrupt them if they exceed the configured timeout in milliseconds.
|=========================================================
+NOTE: Consult the latest link:http://docs.groovy-lang.org/latest/html/documentation/#_typing[Groovy Documentation]
+for information on the differences. It is important to understand the impact that these configuration will have on
+submitted scripts before enabling this feature.
+
To provide some basic out-of-the-box protections against troublesome scripts, the following configuration can be used:
[source,yaml]
+----
scriptEngines: {
gremlin-groovy: {
- imports: [java.lang.Math],
- staticImports: [java.lang.Math.PI],
- scripts: [scripts/empty-sample.groovy],
- config: {
- compilerCustomizerProviders: {
- "org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.ThreadInterruptCustomizerProvider":[],
- "org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.TimedInterruptCustomizerProvider":[10000],
- "org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.CompileStaticCustomizerProvider":["org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.SimpleSandboxExtension"]}}}}
-
-NOTE: The above configuration could also use the `TypeCheckedCustomizerProvider` in place of the
-`CompileStaticCustomizerProvider`. The differences between `TypeChecked` and `CompileStatic` are beyond the scope of
-this documentation. Consult the latest link:http://docs.groovy-lang.org/latest/html/documentation/#_typing[Groovy Documentation]
-for information on the differences. It is important to understand the impact that these configuration will have on
-submitted scripts before enabling this feature.
-
-NOTE: The import of classes to the script engine is handled by the `ImportCustomizerProvider`. As the concept of
-"imports" is a first-class citizen (i.e. has its own configuration options), it is not recommended that the
-`ImportCustomizerProvider` be used as a configuration option to `compilerCustomizerProviders`.
+ plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},
+ org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {}
+ org.apache.tinkerpop.gremlin.groovy.jsr223.GroovyCompilerGremlinPlugin: {enableThreadInterrupt: true, timedInterrupt: 10000, compilation: COMPILE_STATIC, extensions: org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.SimpleSandboxExtension},
+ org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
+ org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample-secure.groovy]}}}}
+----
This configuration uses the `SimpleSandboxExtension`, which blacklists calls to methods on the `System` class,
thereby preventing someone from remotely killing the server:
@@ -1561,8 +1557,8 @@ The `FileSandboxExtension` provides for a basic configurable security function i
sandboxing implementations can be developed by using this white listing model and extending from the
`AbstractSandboxExtension`.
-A final thought on the topic of `CompilerCustomizerProvider` implementations is that they are not just for
-"security" (though they are demonstrated in that capacity here). They can be used for a variety of features that
+A final thought on the topic of `GroovyCompilerGremlinPlugin` implementation is that it is not just for
+"security" (though is is demonstrated in that capacity here). It can be used for a variety of features that
can fine tune the Groovy compilation process. Read more about compilation customization in the
link:http://docs.groovy-lang.org/latest/html/documentation/#compilation-customizers[Groovy Documentation].