You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by ke...@apache.org on 2024/11/03 23:50:27 UTC
(tinkerpop) branch master-http-final updated (ed4f85c8e2 -> 048ec39e3b)
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a change to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
discard ed4f85c8e2 Make gremlin-python feature tests use gremlin-lang
discard 6c78d2b1e7 CTR remove unsupported types in GraphSONv4 in gremlin-python
discard ea26e79c5a update bigdecimal in statics to be consistent with server definition (#2875)
discard 4fa8a353ef change from streaming deserialization to aggregating data and make graphson class naming consistent (#2873)
discard 3481074001 Add signal that bytes were read in HttpGremlinResponseDecoder CTR
discard 95a21aef37 Remove unsupported types in GraphSONv4 from core. (#2874)
discard 21878616e0 Changed connection pooling to reflect having switched from long lived web socket connections to short lived http connections: (#2860)
discard 178f4f08f7 Update GraphSONV4 extended types, element, and datetime in gremlin-python (#2854)
discard 6a77d09392 Re-enable feature test of GraphSONv4 CTR
discard a4e5f8f9bd Remove Traverser serializers for GraphSON/GraphBinary V4 (#2862)
discard 37c246fb9c Update BulkSet GraphSONv4 serialization. (#2861)
discard 46a8dae9f2 CTR temporarily pin aiohttp to 3.8-supported version
discard 7381039095 Added request interceptor to gremlin-python. (#2849)
discard 7db715989c enable basic GraphSONV4 connectivity in python
discard 1215525523 Update GraphSONv4 RequestMessage documentation CTR
discard 3c489451fb Re-enable GraphSON message serializer for debugging. (#2836)
discard f7f9099d47 Rename GraphSONv4 g:OffsetDateTime to g:DateTime CTR
discard 5f3a726734 Update g:DateTime definition in GraphSONv4 documentation CTR.
discard 2ad520f7c1 CTR pin aiohttp's dependency yarl to a python3.8-supported version before we upgrade
new dfaccc7b33 Update g:DateTime definition in GraphSONv4 documentation CTR.
new ae76c17e1e Rename GraphSONv4 g:OffsetDateTime to g:DateTime CTR
new 55f7e180c9 Re-enable GraphSON message serializer for debugging. (#2836)
new 22e94ac224 Update GraphSONv4 RequestMessage documentation CTR
new 2d73a08546 enable basic GraphSONV4 connectivity in python
new f7b4f56768 Added request interceptor to gremlin-python. (#2849)
new b856745e7b Update BulkSet GraphSONv4 serialization. (#2861)
new 191849d7e2 Remove Traverser serializers for GraphSON/GraphBinary V4 (#2862)
new 5cea695c45 Re-enable feature test of GraphSONv4 CTR
new 7b3caefcf0 Update GraphSONV4 extended types, element, and datetime in gremlin-python (#2854)
new 68c3bd02d6 Changed connection pooling to reflect having switched from long lived web socket connections to short lived http connections: (#2860)
new 6ddd11431b Remove unsupported types in GraphSONv4 from core. (#2874)
new 774ef0e076 Add signal that bytes were read in HttpGremlinResponseDecoder CTR
new 6c7a31fd6d change from streaming deserialization to aggregating data and make graphson class naming consistent (#2873)
new c3e29aad63 update bigdecimal in statics to be consistent with server definition (#2875)
new 0d1635b194 CTR remove unsupported types in GraphSONv4 in gremlin-python
new 048ec39e3b Make gremlin-python feature tests use gremlin-lang
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ed4f85c8e2)
\
N -- N -- N refs/heads/master-http-final (048ec39e3b)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 17 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
gremlin-python/src/main/python/setup.py | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
(tinkerpop) 15/17: update bigdecimal in statics to be consistent with server definition (#2875)
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit c3e29aad63dc98989b00bec993a0aeb5e24e561e
Author: Yang Xia <55...@users.noreply.github.com>
AuthorDate: Thu Oct 31 17:49:06 2024 -0700
update bigdecimal in statics to be consistent with server definition (#2875)
---
.../python/gremlin_python/process/traversal.py | 6 +---
.../src/main/python/gremlin_python/statics.py | 27 +++++++++++++++--
.../gremlin_python/structure/io/graphsonV4.py | 27 ++++-------------
.../tests/structure/io/test_functionalityio.py | 1 -
.../tests/structure/io/test_graphbinaryv4model.py | 8 ++---
.../python/tests/structure/io/test_graphsonV4.py | 34 ++++------------------
.../tests/structure/io/test_graphsonv4model.py | 14 ++++-----
.../src/main/python/tests/test_statics.py | 18 ++++++++++++
8 files changed, 63 insertions(+), 72 deletions(-)
diff --git a/gremlin-python/src/main/python/gremlin_python/process/traversal.py b/gremlin-python/src/main/python/gremlin_python/process/traversal.py
index 8eac3e8349..bc37dc0506 100644
--- a/gremlin-python/src/main/python/gremlin_python/process/traversal.py
+++ b/gremlin-python/src/main/python/gremlin_python/process/traversal.py
@@ -841,11 +841,7 @@ class GremlinLang(object):
else:
return f'{arg}D'
if isinstance(arg, BigDecimal):
- # TODO double check implementation when revisiting types definition.
- if arg.scale > 0:
- return f'{arg.unscaled_value:.{arg.scale}f}M'
- else:
- return f'{arg.unscaled_value}M'
+ return f'{arg.value}M'
if isinstance(arg, datetime):
return f'datetime("{arg.isoformat()}")'
diff --git a/gremlin-python/src/main/python/gremlin_python/statics.py b/gremlin-python/src/main/python/gremlin_python/statics.py
index ac05fe2448..b5a83fbc32 100644
--- a/gremlin-python/src/main/python/gremlin_python/statics.py
+++ b/gremlin-python/src/main/python/gremlin_python/statics.py
@@ -16,7 +16,7 @@
# specific language governing permissions and limitations
# under the License.
#
-
+import decimal
from types import FunctionType
from aenum import Enum
@@ -74,12 +74,35 @@ class GremlinType(object):
self.gremlin_type = gremlin_type
-# TODO scale and unscaled value implementations are not following definition, need to revisit
class BigDecimal(object):
+ """
+ Provides a way to represent a BigDecimal for Gremlin.
+ """
def __init__(self, scale, unscaled_value):
self.scale = scale
self.unscaled_value = unscaled_value
+ @property
+ def value(self):
+ self._as_decimal = decimal.Decimal(self.unscaled_value)
+ precision = len(self._as_decimal.as_tuple().digits)
+ with decimal.localcontext(decimal.Context(prec=precision)):
+ return self._as_decimal.scaleb(-self.scale)
+
+"""
+Create a BigDecimal from a number that can be converted to a Decimal. Note precision may be lost during the conversion.
+"""
+def to_bigdecimal(value):
+ try:
+ decimal_value = value if isinstance(value, decimal.Decimal) else decimal.Decimal(str(value))
+ scale = -decimal_value.as_tuple().exponent
+ unscaled_value = int("".join(map(str, decimal_value.as_tuple().digits)))
+ except TypeError:
+ raise ValueError("BigDecimal does not support NaN, Infinity or -Infinity")
+ except Exception as err:
+ raise ValueError(f'Encountered error: {err}. Value must be able to convert to a Decimal.')
+ return BigDecimal(scale, unscaled_value if decimal_value >= 0 else -unscaled_value)
+
staticMethods = {}
staticEnums = {}
diff --git a/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py b/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
index 7a63c8bee7..d20d3b4e31 100644
--- a/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
+++ b/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
@@ -29,7 +29,7 @@ from aenum import Enum
from isodate import parse_duration, duration_isoformat
from gremlin_python import statics
-from gremlin_python.statics import FloatType, BigDecimal, ShortType, IntType, LongType, TypeType, DictType, ListType, SetType, SingleByte, SingleChar
+from gremlin_python.statics import FloatType, BigDecimal, ShortType, IntType, LongType, TypeType, DictType, ListType, SetType, SingleByte, SingleChar, to_bigdecimal
from gremlin_python.process.traversal import Direction, P, TextP, Traversal, Traverser, TraversalStrategy, T
from gremlin_python.structure.graph import Edge, Property, Vertex, VertexProperty, Path
from gremlin_python.structure.io.util import HashableDict, SymbolUtil
@@ -455,35 +455,18 @@ class FloatIO(_NumberIO):
return cls.python_type(v)
-class BigDecimalIO(_NumberIO):
- python_type = Decimal
+class BigDecimalIO(_GraphSONTypeIO):
+ python_type = BigDecimal
graphson_type = "g:BigDecimal"
graphson_base_type = "BigDecimal"
@classmethod
def dictify(cls, n, writer):
- if isinstance(n, bool): # because isinstance(False, int) and isinstance(True, int)
- return n
- elif math.isnan(n):
- return GraphSONUtil.typed_value(cls.graphson_base_type, "NaN", "g")
- elif math.isinf(n) and n > 0:
- return GraphSONUtil.typed_value(cls.graphson_base_type, "Infinity", "g")
- elif math.isinf(n) and n < 0:
- return GraphSONUtil.typed_value(cls.graphson_base_type, "-Infinity", "g")
- else:
- return GraphSONUtil.typed_value(cls.graphson_base_type, str(n), "g")
+ return GraphSONUtil.typed_value(cls.graphson_base_type, str(n.value), "g")
@classmethod
def objectify(cls, v, _):
- if isinstance(v, str):
- if v == 'NaN':
- return Decimal('nan')
- elif v == "Infinity":
- return Decimal('inf')
- elif v == "-Infinity":
- return Decimal('-inf')
-
- return Decimal(v)
+ return to_bigdecimal(v)
class DoubleIO(FloatIO):
diff --git a/gremlin-python/src/main/python/tests/structure/io/test_functionalityio.py b/gremlin-python/src/main/python/tests/structure/io/test_functionalityio.py
index be74a28710..8dfe37adab 100644
--- a/gremlin-python/src/main/python/tests/structure/io/test_functionalityio.py
+++ b/gremlin-python/src/main/python/tests/structure/io/test_functionalityio.py
@@ -157,7 +157,6 @@ def test_bigint_negative(remote_connection):
g.V(vid).drop().iterate()
-@pytest.mark.skip(reason="BigDecimal implementation needs revisiting")
def test_bigdecimal(remote_connection):
if not isinstance(remote_connection._client.response_serializer(), GraphBinarySerializersV4):
return
diff --git a/gremlin-python/src/main/python/tests/structure/io/test_graphbinaryv4model.py b/gremlin-python/src/main/python/tests/structure/io/test_graphbinaryv4model.py
index ab18c91443..63298d4bcd 100644
--- a/gremlin-python/src/main/python/tests/structure/io/test_graphbinaryv4model.py
+++ b/gremlin-python/src/main/python/tests/structure/io/test_graphbinaryv4model.py
@@ -48,22 +48,20 @@ def read_file_by_name(resource_name):
def test_pos_bigdecimal():
def decimal_cmp(x, y):
- if x.scale == y.scale and x.unscaled_value == y.unscaled_value:
+ if x.scale == y.scale and x.unscaled_value == y.unscaled_value and x.value == y.value:
return True
else:
return False
- # gremlin-python adds an extra 0 byte to the value.
run_writeread("pos-bigdecimal", decimal_cmp)
def test_neg_bigdecimal():
def decimal_cmp(x, y):
- if x.scale == y.scale and x.unscaled_value == y.unscaled_value:
+ if x.scale == y.scale and x.unscaled_value == y.unscaled_value and x.value == y.value:
return True
else:
return False
-
- # gremlin-python adds an extra 0 byte to the value.
+
run_writeread("neg-bigdecimal", decimal_cmp)
def test_pos_biginteger():
diff --git a/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py b/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py
index 942fb0f0ea..44b2fa7187 100644
--- a/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py
+++ b/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py
@@ -152,36 +152,15 @@ class TestGraphSONReader:
"@type": "g:BigDecimal",
"@value": 31.2
}))
- assert isinstance(x, Decimal)
- assert Decimal(31.2) == x
+ assert isinstance(x, BigDecimal)
+ assert Decimal('31.2') == x.value
##
x = self.graphson_reader.read_object(json.dumps({
"@type": "g:BigDecimal",
"@value": 123456789987654321123456789987654321
}))
- assert isinstance(x, Decimal)
- assert Decimal('123456789987654321123456789987654321') == x
- ##
- x = self.graphson_reader.read_object(json.dumps({
- "@type": "g:BigDecimal",
- "@value": "NaN"
- }))
- assert isinstance(x, Decimal)
- assert math.isnan(x)
- ##
- x = self.graphson_reader.read_object(json.dumps({
- "@type": "g:BigDecimal",
- "@value": "Infinity"
- }))
- assert isinstance(x, Decimal)
- assert math.isinf(x) and x > 0
- ##
- x = self.graphson_reader.read_object(json.dumps({
- "@type": "g:BigDecimal",
- "@value": "-Infinity"
- }))
- assert isinstance(x, Decimal)
- assert math.isinf(x) and x < 0
+ assert isinstance(x, BigDecimal)
+ assert to_bigdecimal(123456789987654321123456789987654321).value == x.value
##
x = self.graphson_reader.read_object(json.dumps({
"@type": "g:BigInteger",
@@ -371,10 +350,7 @@ class TestGraphSONWriter:
assert {"@type": "g:Double", "@value": "NaN"} == json.loads(self.graphson_writer.write_object(float('nan')))
assert {"@type": "g:Double", "@value": "Infinity"} == json.loads(self.graphson_writer.write_object(float('inf')))
assert {"@type": "g:Double", "@value": "-Infinity"} == json.loads(self.graphson_writer.write_object(float('-inf')))
- assert {"@type": "g:BigDecimal", "@value": "123456789987654321123456789987654321"} == json.loads(self.graphson_writer.write_object(Decimal('123456789987654321123456789987654321')))
- assert {"@type": "g:BigDecimal", "@value": "NaN"} == json.loads(self.graphson_writer.write_object(Decimal('nan')))
- assert {"@type": "g:BigDecimal", "@value": "Infinity"} == json.loads(self.graphson_writer.write_object(Decimal('inf')))
- assert {"@type": "g:BigDecimal", "@value": "-Infinity"} == json.loads(self.graphson_writer.write_object(Decimal('-inf')))
+ assert {"@type": "g:BigDecimal", "@value": "123.456789987654321123456789987654321"} == json.loads(self.graphson_writer.write_object(to_bigdecimal("123.456789987654321123456789987654321")))
assert {"@type": "g:BigInteger", "@value": "123456789987654321123456789987654321"} == json.loads(self.graphson_writer.write_object(long(123456789987654321123456789987654321)))
assert {"@type": "g:BigInteger", "@value": "123456789987654321123456789987654321"} == json.loads(self.graphson_writer.write_object(123456789987654321123456789987654321))
assert """true""" == self.graphson_writer.write_object(True)
diff --git a/gremlin-python/src/main/python/tests/structure/io/test_graphsonv4model.py b/gremlin-python/src/main/python/tests/structure/io/test_graphsonv4model.py
index 683ff030a5..6f6886e860 100644
--- a/gremlin-python/src/main/python/tests/structure/io/test_graphsonv4model.py
+++ b/gremlin-python/src/main/python/tests/structure/io/test_graphsonv4model.py
@@ -47,27 +47,25 @@ def read_file_by_name(resource_name):
with open(full_name, 'r') as resource_file:
return json.dumps(json.load(resource_file), separators=(',', ':'))
-@pytest.mark.skip(reason="enable after big decimal implementation/serialization is unified in python (")
def test_pos_bigdecimal():
def decimal_cmp(x, y):
- if x.scale == y.scale and x.unscaled_value == y.unscaled_value:
+ # python json library can only read a BigDecimal into float during deser, precision will be lost with GraphSON
+ # so compare float only
+ if float(x.value) == float(y.value):
return True
else:
return False
- # gremlin-python adds an extra 0 byte to the value.
run_writeread("pos-bigdecimal", decimal_cmp)
-@pytest.mark.skip(reason="enable after big decimal implementation/serialization is unified in python")
def test_neg_bigdecimal():
def decimal_cmp(x, y):
- if x.scale == y.scale and x.unscaled_value == y.unscaled_value:
+ if float(x.value) == float(y.value):
return True
else:
return False
-
- # gremlin-python adds an extra 0 byte to the value.
- run_writeread("neg-bigdecimal", decimal_cmp())
+
+ run_writeread("neg-bigdecimal", decimal_cmp)
def test_pos_biginteger():
# gremlin-python adds an extra 0 byte to the value.
diff --git a/gremlin-python/src/main/python/tests/test_statics.py b/gremlin-python/src/main/python/tests/test_statics.py
index 2c3f864754..e22718e8e1 100644
--- a/gremlin-python/src/main/python/tests/test_statics.py
+++ b/gremlin-python/src/main/python/tests/test_statics.py
@@ -19,6 +19,8 @@
__author__ = 'Marko A. Rodriguez (http://markorodriguez.com)'
+from decimal import Decimal
+
from gremlin_python import statics
from gremlin_python.process.traversal import Cardinality
from gremlin_python.process.traversal import P
@@ -63,3 +65,19 @@ class TestStatics(object):
raise Exception("SingleChar should throw a value error if input is not a single character string")
except ValueError:
pass
+
+ def test_bigdecimal(self):
+ assert statics.to_bigdecimal(1.23456).value == statics.BigDecimal(5,123456).value
+ assert statics.to_bigdecimal(-1.23456).value == statics.BigDecimal(5,-123456).value
+ # make sure the precision isn't changed globally
+ assert Decimal("123456789").scaleb(-5) == Decimal('1234.56789')
+ try:
+ statics.to_bigdecimal('NaN')
+ raise Exception("to_bigdecimal should throw a value error with NaN, Infinity or -Infinity")
+ except ValueError:
+ pass
+ try:
+ statics.to_bigdecimal('abc')
+ raise Exception("to_bigdecimal should throw a value error if input is not a convertable to Decimal")
+ except ValueError:
+ pass
(tinkerpop) 12/17: Remove unsupported types in GraphSONv4 from core. (#2874)
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 6ddd11431b2ac9d527124c7fc11b1305fb2c1322
Author: kenhuuu <10...@users.noreply.github.com>
AuthorDate: Thu Oct 31 16:00:15 2024 -0700
Remove unsupported types in GraphSONv4 from core. (#2874)
---
.../structure/io/graphson/GraphSONMapper.java | 15 +-
.../structure/io/graphson/GraphSONModule.java | 127 +--------
.../io/graphson/GraphSONSerializersV4.java | 161 -----------
.../structure/io/graphson/GraphSONXModuleV4.java | 34 ---
.../io/graphson/JavaTimeSerializersV4.java | 198 --------------
.../io/graphson/TraversalSerializersV4.java | 297 ---------------------
.../graphson/GraphSONMapperEmbeddedTypeTest.java | 50 +++-
.../GraphSONMapperPartialEmbeddedTypeTest.java | 9 +-
.../structure/io/graphson/GraphSONMapperTest.java | 27 ++
.../server/GremlinServerHttpIntegrateTest.java | 8 +-
10 files changed, 86 insertions(+), 840 deletions(-)
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapper.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapper.java
index 7f16e8d1eb..2fb95a92eb 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapper.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapper.java
@@ -84,7 +84,9 @@ public class GraphSONMapper implements Mapper<ObjectMapper> {
@Override
public ObjectMapper createMapper() {
final ObjectMapper om = new ObjectMapper(JsonFactory.builder().streamReadConstraints(streamReadConstraints).build());
- om.disable(SerializationFeature.FAIL_ON_EMPTY_BEANS);
+ if (version != GraphSONVersion.V4_0) {
+ om.disable(SerializationFeature.FAIL_ON_EMPTY_BEANS);
+ }
final GraphSONModule graphSONModule = version.getBuilder().create(normalize, typeInfo);
om.registerModule(graphSONModule);
@@ -141,9 +143,14 @@ public class GraphSONMapper implements Mapper<ObjectMapper> {
throw new IllegalStateException("Unknown GraphSONVersion: " + version);
}
- // this provider toStrings all unknown classes and converts keys in Map objects that are Object to String.
- final DefaultSerializerProvider provider = new GraphSONSerializerProvider(version);
- om.setSerializerProvider(provider);
+ // Starting with GraphSONv4, only types that can be returned from the result of a traversal are supported. This
+ // differs to previous versions where a gremlin-groovy script could return any type. So if an unknown type is
+ // encountered, an error should be thrown.
+ if (version != GraphSONVersion.V4_0) {
+ // this provider toStrings all unknown classes and converts keys in Map objects that are Object to String.
+ final DefaultSerializerProvider provider = new GraphSONSerializerProvider(version);
+ om.setSerializerProvider(provider);
+ }
if (normalize)
om.enable(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS);
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java
index 175114d149..c18f3b05c5 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java
@@ -158,70 +158,15 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
put(Set.class, "Set");
// TinkerPop Graph objects
- put(Lambda.class, "Lambda");
put(Vertex.class, "Vertex");
put(Edge.class, "Edge");
put(Property.class, "Property");
put(Path.class, "Path");
put(VertexProperty.class, "VertexProperty");
- put(Metrics.class, "Metrics");
- put(TraversalMetrics.class, "TraversalMetrics");
- put(TraversalExplanation.class, "TraversalExplanation");
put(Tree.class, "Tree");
- put(AndP.class, "P");
- put(OrP.class, "P");
- put(P.class, "P");
- put(TextP.class, "TextP");
Stream.of(
- VertexProperty.Cardinality.class,
- Column.class,
Direction.class,
- DT.class,
- Merge.class,
- Operator.class,
- Order.class,
- Pop.class,
- SackFunctions.Barrier.class,
- Pick.class,
- Scope.class,
T.class).forEach(e -> put(e, e.getSimpleName()));
- Arrays.asList(
- ConnectiveStrategy.class,
- ElementIdStrategy.class,
- EventStrategy.class,
- HaltedTraverserStrategy.class,
- PartitionStrategy.class,
- SubgraphStrategy.class,
- SeedStrategy.class,
- LazyBarrierStrategy.class,
- MatchAlgorithmStrategy.class,
- AdjacentToIncidentStrategy.class,
- ByModulatorOptimizationStrategy.class,
- ProductiveByStrategy.class,
- CountStrategy.class,
- FilterRankingStrategy.class,
- IdentityRemovalStrategy.class,
- IncidentToAdjacentStrategy.class,
- InlineFilterStrategy.class,
- MatchPredicateStrategy.class,
- OrderLimitStrategy.class,
- OptionsStrategy.class,
- PathProcessorStrategy.class,
- PathRetractionStrategy.class,
- RepeatUnrollStrategy.class,
- ComputerVerificationStrategy.class,
- LambdaRestrictionStrategy.class,
- ReadOnlyStrategy.class,
- StandardVerificationStrategy.class,
- EarlyLimitStrategy.class,
- EdgeLabelVerificationStrategy.class,
- ReservedKeysVerificationStrategy.class,
- //
- GraphFilterStrategy.class,
- VertexProgramStrategy.class
- ).forEach(strategy -> put(strategy, strategy.getSimpleName()));
-
- GraphSONModule.tryLoadSparqlStrategy().ifPresent(s -> put(s, s.getSimpleName()));
}});
/**
@@ -237,9 +182,6 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
addSerializer(Vertex.class, new GraphSONSerializersV4.VertexJacksonSerializer(normalize, typeInfo));
addSerializer(VertexProperty.class, new GraphSONSerializersV4.VertexPropertyJacksonSerializer(normalize, true));
addSerializer(Property.class, new GraphSONSerializersV4.PropertyJacksonSerializer());
- addSerializer(Metrics.class, new GraphSONSerializersV4.MetricsJacksonSerializer());
- addSerializer(TraversalMetrics.class, new GraphSONSerializersV4.TraversalMetricsJacksonSerializer());
- addSerializer(TraversalExplanation.class, new GraphSONSerializersV4.TraversalExplanationJacksonSerializer());
addSerializer(Path.class, new GraphSONSerializersV4.PathJacksonSerializer());
addSerializer(DirectionalStarGraph.class, new StarGraphGraphSONSerializerV4(normalize));
addSerializer(Tree.class, new GraphSONSerializersV4.TreeJacksonSerializer());
@@ -258,22 +200,9 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
// traversal
addSerializer(BulkSet.class, new TraversalSerializersV4.BulkSetJacksonSerializer());
- addSerializer(Traversal.class, new TraversalSerializersV4.TraversalJacksonSerializer());
- Stream.of(VertexProperty.Cardinality.class,
- Column.class,
+ Stream.of(
Direction.class,
- DT.class,
- Merge.class,
- Operator.class,
- Order.class,
- Pop.class,
- SackFunctions.Barrier.class,
- Scope.class,
- Pick.class,
T.class).forEach(e -> addSerializer(e, new TraversalSerializersV4.EnumJacksonSerializer()));
- addSerializer(P.class, new TraversalSerializersV4.PJacksonSerializer());
- addSerializer(Lambda.class, new TraversalSerializersV4.LambdaJacksonSerializer());
- addSerializer(TraversalStrategy.class, new TraversalSerializersV4.TraversalStrategyJacksonSerializer());
/////////////////////// DESERIALIZERS ////////////////////////////
@@ -282,10 +211,7 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
addDeserializer(Edge.class, new GraphSONSerializersV4.EdgeJacksonDeserializer());
addDeserializer(Property.class, new GraphSONSerializersV4.PropertyJacksonDeserializer());
addDeserializer(Path.class, new GraphSONSerializersV4.PathJacksonDeserializer());
- addDeserializer(TraversalExplanation.class, new GraphSONSerializersV4.TraversalExplanationJacksonDeserializer());
addDeserializer(VertexProperty.class, new GraphSONSerializersV4.VertexPropertyJacksonDeserializer());
- addDeserializer(Metrics.class, new GraphSONSerializersV4.MetricsJacksonDeserializer());
- addDeserializer(TraversalMetrics.class, new GraphSONSerializersV4.TraversalMetricsJacksonDeserializer());
addDeserializer(Tree.class, new GraphSONSerializersV4.TreeJacksonDeserializer());
// java.util - use the standard jackson serializers for collections when types aren't embedded
@@ -300,58 +226,9 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
addDeserializer(Double.class, new GraphSONSerializersV4.DoubleJacksonDeserializer());
// traversal
- Stream.of(VertexProperty.Cardinality.values(),
- Column.values(),
+ Stream.of(
Direction.values(),
- DT.values(),
- Merge.values(),
- Operator.values(),
- Order.values(),
- Pop.values(),
- SackFunctions.Barrier.values(),
- Scope.values(),
- Pick.values(),
T.values()).flatMap(Stream::of).forEach(e -> addDeserializer(e.getClass(), new TraversalSerializersV4.EnumJacksonDeserializer(e.getDeclaringClass())));
- addDeserializer(P.class, new TraversalSerializersV4.PJacksonDeserializer());
- addDeserializer(TextP.class, new TraversalSerializersV4.TextPJacksonDeserializer());
- addDeserializer(Lambda.class, new TraversalSerializersV4.LambdaJacksonDeserializer());
- Arrays.asList(
- ConnectiveStrategy.class,
- ElementIdStrategy.class,
- EventStrategy.class,
- HaltedTraverserStrategy.class,
- PartitionStrategy.class,
- SubgraphStrategy.class,
- SeedStrategy.class,
- LazyBarrierStrategy.class,
- MatchAlgorithmStrategy.class,
- AdjacentToIncidentStrategy.class,
- ByModulatorOptimizationStrategy.class,
- ProductiveByStrategy.class,
- CountStrategy.class,
- FilterRankingStrategy.class,
- IdentityRemovalStrategy.class,
- IncidentToAdjacentStrategy.class,
- InlineFilterStrategy.class,
- MatchPredicateStrategy.class,
- OrderLimitStrategy.class,
- OptionsStrategy.class,
- PathProcessorStrategy.class,
- PathRetractionStrategy.class,
- RepeatUnrollStrategy.class,
- ComputerVerificationStrategy.class,
- LambdaRestrictionStrategy.class,
- ReadOnlyStrategy.class,
- StandardVerificationStrategy.class,
- EarlyLimitStrategy.class,
- EdgeLabelVerificationStrategy.class,
- ReservedKeysVerificationStrategy.class,
- //
- GraphFilterStrategy.class,
- VertexProgramStrategy.class
- ).forEach(strategy -> addDeserializer(strategy, new TraversalSerializersV4.TraversalStrategyProxyJacksonDeserializer(strategy)));
-
- GraphSONModule.tryLoadSparqlStrategy().ifPresent(s -> addDeserializer(s, new TraversalSerializersV4.TraversalStrategyProxyJacksonDeserializer(s)));
}
public static Builder build() {
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONSerializersV4.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONSerializersV4.java
index 0c6a06feef..f4625797c5 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONSerializersV4.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONSerializersV4.java
@@ -326,44 +326,6 @@ class GraphSONSerializersV4 {
}
}
- final static class TraversalExplanationJacksonSerializer extends StdScalarSerializer<TraversalExplanation> {
- public TraversalExplanationJacksonSerializer() {
- super(TraversalExplanation.class);
- }
-
- @Override
- public void serialize(final TraversalExplanation traversalExplanation, final JsonGenerator jsonGenerator,
- final SerializerProvider serializerProvider) throws IOException {
- final Map<String, Object> m = new HashMap<>();
- m.put(GraphSONTokens.ORIGINAL, getStepsAsList(traversalExplanation.getOriginalTraversal()));
-
- final List<Pair<TraversalStrategy, Traversal.Admin<?, ?>>> strategyTraversals = traversalExplanation.getStrategyTraversals();
-
- final List<Map<String, Object>> intermediates = new ArrayList<>();
- for (final Pair<TraversalStrategy, Traversal.Admin<?, ?>> pair : strategyTraversals) {
- final Map<String, Object> intermediate = new HashMap<>();
- intermediate.put(GraphSONTokens.STRATEGY, pair.getValue0().toString());
- intermediate.put(GraphSONTokens.CATEGORY, pair.getValue0().getTraversalCategory().getSimpleName());
- intermediate.put(GraphSONTokens.TRAVERSAL, getStepsAsList(pair.getValue1()));
- intermediates.add(intermediate);
- }
- m.put(GraphSONTokens.INTERMEDIATE, intermediates);
-
- if (strategyTraversals.isEmpty())
- m.put(GraphSONTokens.FINAL, getStepsAsList(traversalExplanation.getOriginalTraversal()));
- else
- m.put(GraphSONTokens.FINAL, getStepsAsList(strategyTraversals.get(strategyTraversals.size() - 1).getValue1()));
-
- jsonGenerator.writeObject(m);
- }
-
- private List<String> getStepsAsList(final Traversal.Admin<?, ?> t) {
- final List<String> steps = new ArrayList<>();
- t.getSteps().iterator().forEachRemaining(s -> steps.add(s.toString()));
- return steps;
- }
- }
-
final static class IntegerGraphSONSerializer extends StdScalarSerializer<Integer> {
public IntegerGraphSONSerializer() {
super(Integer.class);
@@ -388,52 +350,6 @@ class GraphSONSerializersV4 {
}
}
- final static class TraversalMetricsJacksonSerializer extends StdScalarSerializer<TraversalMetrics> {
- public TraversalMetricsJacksonSerializer() {
- super(TraversalMetrics.class);
- }
-
- @Override
- public void serialize(final TraversalMetrics traversalMetrics, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider)
- throws IOException {
- // creation of the map enables all the fields to be properly written with their type if required
- final Map<String, Object> m = new HashMap<>();
- m.put(GraphSONTokens.DURATION, traversalMetrics.getDuration(TimeUnit.NANOSECONDS) / 1000000d);
- final List<Metrics> metrics = new ArrayList<>();
- metrics.addAll(traversalMetrics.getMetrics());
- m.put(GraphSONTokens.METRICS, metrics);
-
- jsonGenerator.writeObject(m);
- }
- }
-
- final static class MetricsJacksonSerializer extends StdScalarSerializer<Metrics> {
- public MetricsJacksonSerializer() {
- super(Metrics.class);
- }
-
- @Override
- public void serialize(final Metrics metrics, final JsonGenerator jsonGenerator,
- final SerializerProvider serializerProvider) throws IOException {
- final Map<String, Object> m = new HashMap<>();
- m.put(GraphSONTokens.ID, metrics.getId());
- m.put(GraphSONTokens.NAME, metrics.getName());
- m.put(GraphSONTokens.COUNTS, metrics.getCounts());
- m.put(GraphSONTokens.DURATION, metrics.getDuration(TimeUnit.NANOSECONDS) / 1000000d);
-
- if (!metrics.getAnnotations().isEmpty()) {
- m.put(GraphSONTokens.ANNOTATIONS, metrics.getAnnotations());
- }
- if (!metrics.getNested().isEmpty()) {
- final List<Metrics> nested = new ArrayList<>();
- metrics.getNested().forEach(it -> nested.add(it));
- m.put(GraphSONTokens.METRICS, nested);
- }
- jsonGenerator.writeObject(m);
- }
- }
-
-
/**
* Maps in the JVM can have {@link Object} as a key, but in JSON they must be a {@link String}.
*/
@@ -651,83 +567,6 @@ class GraphSONSerializersV4 {
}
}
- static class TraversalExplanationJacksonDeserializer extends StdDeserializer<TraversalExplanation> {
- public TraversalExplanationJacksonDeserializer() {
- super(TraversalExplanation.class);
- }
-
- @Override
- public TraversalExplanation deserialize(final JsonParser jsonParser, final DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
- final Map<String, Object> explainData = deserializationContext.readValue(jsonParser, Map.class);
- final String originalTraversal = explainData.get(GraphSONTokens.ORIGINAL).toString();
- final List<Triplet<String, String, String>> intermediates = new ArrayList<>();
- final List<Map<String,Object>> listMap = (List<Map<String,Object>>) explainData.get(GraphSONTokens.INTERMEDIATE);
- for (Map<String,Object> m : listMap) {
- intermediates.add(Triplet.with(m.get(GraphSONTokens.STRATEGY).toString(),
- m.get(GraphSONTokens.CATEGORY).toString(),
- m.get(GraphSONTokens.TRAVERSAL).toString()));
- }
-
- return new ImmutableExplanation(originalTraversal, intermediates);
- }
-
- @Override
- public boolean isCachable() {
- return true;
- }
- }
-
- static class MetricsJacksonDeserializer extends StdDeserializer<Metrics> {
- public MetricsJacksonDeserializer() {
- super(Metrics.class);
- }
-
- @Override
- public Metrics deserialize(final JsonParser jsonParser, final DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
- final Map<String, Object> metricsData = deserializationContext.readValue(jsonParser, Map.class);
- final MutableMetrics m = new MutableMetrics((String)metricsData.get(GraphSONTokens.ID), (String)metricsData.get(GraphSONTokens.NAME));
-
- m.setDuration(Math.round((Double) metricsData.get(GraphSONTokens.DURATION) * 1000000), TimeUnit.NANOSECONDS);
- for (Map.Entry<String, Long> count : ((Map<String, Long>)metricsData.getOrDefault(GraphSONTokens.COUNTS, new LinkedHashMap<>(0))).entrySet()) {
- m.setCount(count.getKey(), count.getValue());
- }
- for (Map.Entry<String, Long> count : ((Map<String, Long>) metricsData.getOrDefault(GraphSONTokens.ANNOTATIONS, new LinkedHashMap<>(0))).entrySet()) {
- m.setAnnotation(count.getKey(), count.getValue());
- }
- for (MutableMetrics nested : (List<MutableMetrics>)metricsData.getOrDefault(GraphSONTokens.METRICS, new ArrayList<>(0))) {
- m.addNested(nested);
- }
- return m;
- }
-
- @Override
- public boolean isCachable() {
- return true;
- }
- }
-
- static class TraversalMetricsJacksonDeserializer extends StdDeserializer<TraversalMetrics> {
-
- public TraversalMetricsJacksonDeserializer() {
- super(TraversalMetrics.class);
- }
-
- @Override
- public TraversalMetrics deserialize(final JsonParser jsonParser, final DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
- final Map<String, Object> traversalMetricsData = deserializationContext.readValue(jsonParser, Map.class);
-
- return new DefaultTraversalMetrics(
- Math.round((Double) traversalMetricsData.get(GraphSONTokens.DURATION) * 1000000),
- (List<MutableMetrics>) traversalMetricsData.get(GraphSONTokens.METRICS)
- );
- }
-
- @Override
- public boolean isCachable() {
- return true;
- }
- }
-
static class TreeJacksonDeserializer extends StdDeserializer<Tree> {
public TreeJacksonDeserializer() {
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONXModuleV4.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONXModuleV4.java
index aa7ad5e9ad..dfceb7a9ab 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONXModuleV4.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONXModuleV4.java
@@ -54,22 +54,10 @@ public final class GraphSONXModuleV4 extends GraphSONModule {
put(BigDecimal.class, "BigDecimal");
put(Byte.class, "Byte");
put(Character.class, "Char");
- put(InetAddress.class, "InetAddress");
// Time serializers/deserializers
put(Duration.class, "Duration");
- put(Instant.class, "Instant");
- put(LocalDate.class, "LocalDate");
- put(LocalDateTime.class, "LocalDateTime");
- put(LocalTime.class, "LocalTime");
- put(MonthDay.class, "MonthDay");
put(OffsetDateTime.class, "DateTime");
- put(OffsetTime.class, "OffsetTime");
- put(Period.class, "Period");
- put(Year.class, "Year");
- put(YearMonth.class, "YearMonth");
- put(ZonedDateTime.class, "ZonedDateTime");
- put(ZoneOffset.class, "ZoneOffset");
}});
/**
@@ -82,35 +70,13 @@ public final class GraphSONXModuleV4 extends GraphSONModule {
// java.time
addSerializer(Duration.class, new JavaTimeSerializersV4.DurationJacksonSerializer());
- addSerializer(Instant.class, new JavaTimeSerializersV4.InstantJacksonSerializer());
- addSerializer(LocalDate.class, new JavaTimeSerializersV4.LocalDateJacksonSerializer());
- addSerializer(LocalDateTime.class, new JavaTimeSerializersV4.LocalDateTimeJacksonSerializer());
- addSerializer(LocalTime.class, new JavaTimeSerializersV4.LocalTimeJacksonSerializer());
- addSerializer(MonthDay.class, new JavaTimeSerializersV4.MonthDayJacksonSerializer());
addSerializer(OffsetDateTime.class, new JavaTimeSerializersV4.OffsetDateTimeJacksonSerializer());
- addSerializer(OffsetTime.class, new JavaTimeSerializersV4.OffsetTimeJacksonSerializer());
- addSerializer(Period.class, new JavaTimeSerializersV4.PeriodJacksonSerializer());
- addSerializer(Year.class, new JavaTimeSerializersV4.YearJacksonSerializer());
- addSerializer(YearMonth.class, new JavaTimeSerializersV4.YearMonthJacksonSerializer());
- addSerializer(ZonedDateTime.class, new JavaTimeSerializersV4.ZonedDateTimeJacksonSerializer());
- addSerializer(ZoneOffset.class, new JavaTimeSerializersV4.ZoneOffsetJacksonSerializer());
/////////////////////// DESERIALIZERS ////////////////////////////
// java.time
addDeserializer(Duration.class, new JavaTimeSerializersV4.DurationJacksonDeserializer());
- addDeserializer(Instant.class, new JavaTimeSerializersV4.InstantJacksonDeserializer());
- addDeserializer(LocalDate.class, new JavaTimeSerializersV4.LocalDateJacksonDeserializer());
- addDeserializer(LocalDateTime.class, new JavaTimeSerializersV4.LocalDateTimeJacksonDeserializer());
- addDeserializer(LocalTime.class, new JavaTimeSerializersV4.LocalTimeJacksonDeserializer());
- addDeserializer(MonthDay.class, new JavaTimeSerializersV4.MonthDayJacksonDeserializer());
addDeserializer(OffsetDateTime.class, new JavaTimeSerializersV4.OffsetDateTimeJacksonDeserializer());
- addDeserializer(OffsetTime.class, new JavaTimeSerializersV4.OffsetTimeJacksonDeserializer());
- addDeserializer(Period.class, new JavaTimeSerializersV4.PeriodJacksonDeserializer());
- addDeserializer(Year.class, new JavaTimeSerializersV4.YearJacksonDeserializer());
- addDeserializer(YearMonth.class, new JavaTimeSerializersV4.YearMonthJacksonDeserializer());
- addDeserializer(ZonedDateTime.class, new JavaTimeSerializersV4.ZonedDateTimeJacksonDeserializer());
- addDeserializer(ZoneOffset.class, new JavaTimeSerializersV4.ZoneOffsetJacksonDeserializer());
}
public static GraphSONModuleBuilder build() {
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/JavaTimeSerializersV4.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/JavaTimeSerializersV4.java
index 891aa667d7..1a1633d294 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/JavaTimeSerializersV4.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/JavaTimeSerializersV4.java
@@ -110,96 +110,6 @@ final class JavaTimeSerializersV4 {
}
}
- final static class InstantJacksonSerializer extends AbstractJavaTimeSerializer<Instant> {
-
- public InstantJacksonSerializer() {
- super(Instant.class);
- }
- }
-
- final static class InstantJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<Instant> {
- public InstantJacksonDeserializer() {
- super(Instant.class);
- }
-
- @Override
- public Instant parse(final String val) {
- return Instant.parse(val);
- }
- }
-
- final static class LocalDateJacksonSerializer extends AbstractJavaTimeSerializer<LocalDate> {
-
- public LocalDateJacksonSerializer() {
- super(LocalDate.class);
- }
- }
-
- final static class LocalDateJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<LocalDate> {
- public LocalDateJacksonDeserializer() {
- super(LocalDate.class);
- }
-
- @Override
- public LocalDate parse(final String val) {
- return LocalDate.parse(val);
- }
- }
-
- final static class LocalDateTimeJacksonSerializer extends AbstractJavaTimeSerializer<LocalDateTime> {
-
- public LocalDateTimeJacksonSerializer() {
- super(LocalDateTime.class);
- }
- }
-
- final static class LocalDateTimeJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<LocalDateTime> {
- public LocalDateTimeJacksonDeserializer() {
- super(LocalDateTime.class);
- }
-
- @Override
- public LocalDateTime parse(final String val) {
- return LocalDateTime.parse(val);
- }
- }
-
- final static class LocalTimeJacksonSerializer extends AbstractJavaTimeSerializer<LocalTime> {
-
- public LocalTimeJacksonSerializer() {
- super(LocalTime.class);
- }
- }
-
- final static class LocalTimeJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<LocalTime> {
- public LocalTimeJacksonDeserializer() {
- super(LocalTime.class);
- }
-
- @Override
- public LocalTime parse(final String val) {
- return LocalTime.parse(val);
- }
- }
-
- final static class MonthDayJacksonSerializer extends AbstractJavaTimeSerializer<MonthDay> {
-
- public MonthDayJacksonSerializer() {
- super(MonthDay.class);
- }
- }
-
- final static class MonthDayJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<MonthDay> {
- public MonthDayJacksonDeserializer() {
- super(MonthDay.class);
- }
-
- @Override
- public MonthDay parse(final String val) {
- return MonthDay.parse(val);
- }
- }
-
final static class OffsetDateTimeJacksonSerializer extends AbstractJavaTimeSerializer<OffsetDateTime> {
public OffsetDateTimeJacksonSerializer() {
@@ -217,112 +127,4 @@ final class JavaTimeSerializersV4 {
return OffsetDateTime.parse(val);
}
}
-
- final static class OffsetTimeJacksonSerializer extends AbstractJavaTimeSerializer<OffsetTime> {
-
- public OffsetTimeJacksonSerializer() {
- super(OffsetTime.class);
- }
- }
-
- final static class OffsetTimeJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<OffsetTime> {
- public OffsetTimeJacksonDeserializer() {
- super(OffsetTime.class);
- }
-
- @Override
- public OffsetTime parse(final String val) {
- return OffsetTime.parse(val);
- }
- }
-
- final static class PeriodJacksonSerializer extends AbstractJavaTimeSerializer<Period> {
-
- public PeriodJacksonSerializer() {
- super(Period.class);
- }
- }
-
- final static class PeriodJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<Period> {
- public PeriodJacksonDeserializer() {
- super(Period.class);
- }
-
- @Override
- public Period parse(final String val) {
- return Period.parse(val);
- }
- }
-
- final static class YearJacksonSerializer extends AbstractJavaTimeSerializer<Year> {
-
- public YearJacksonSerializer() {
- super(Year.class);
- }
- }
-
- final static class YearJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<Year> {
- public YearJacksonDeserializer() {
- super(Year.class);
- }
-
- @Override
- public Year parse(final String val) {
- return Year.parse(val);
- }
- }
-
- final static class YearMonthJacksonSerializer extends AbstractJavaTimeSerializer<YearMonth> {
-
- public YearMonthJacksonSerializer() {
- super(YearMonth.class);
- }
- }
-
- final static class YearMonthJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<YearMonth> {
- public YearMonthJacksonDeserializer() {
- super(YearMonth.class);
- }
-
- @Override
- public YearMonth parse(final String val) {
- return YearMonth.parse(val);
- }
- }
-
- final static class ZonedDateTimeJacksonSerializer extends AbstractJavaTimeSerializer<ZonedDateTime> {
-
- public ZonedDateTimeJacksonSerializer() {
- super(ZonedDateTime.class);
- }
- }
-
- final static class ZonedDateTimeJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<ZonedDateTime> {
- public ZonedDateTimeJacksonDeserializer() {
- super(ZonedDateTime.class);
- }
-
- @Override
- public ZonedDateTime parse(final String val) {
- return ZonedDateTime.parse(val);
- }
- }
-
- final static class ZoneOffsetJacksonSerializer extends AbstractJavaTimeSerializer<ZoneOffset> {
-
- public ZoneOffsetJacksonSerializer() {
- super(ZoneOffset.class);
- }
- }
-
- final static class ZoneOffsetJacksonDeserializer extends AbstractJavaTimeJacksonDeserializer<ZoneOffset> {
- public ZoneOffsetJacksonDeserializer() {
- super(ZoneOffset.class);
- }
-
- @Override
- public ZoneOffset parse(final String val) {
- return ZoneOffset.of(val);
- }
- }
}
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java
index 77a905cc35..1754ee6bd8 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java
@@ -19,37 +19,16 @@
package org.apache.tinkerpop.gremlin.structure.io.graphson;
-import org.apache.commons.configuration2.ConfigurationConverter;
-import org.apache.commons.configuration2.MapConfiguration;
-import org.apache.tinkerpop.gremlin.process.traversal.P;
-import org.apache.tinkerpop.gremlin.process.traversal.TextP;
-import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
-import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
import org.apache.tinkerpop.gremlin.process.traversal.step.util.BulkSet;
-import org.apache.tinkerpop.gremlin.process.traversal.strategy.TraversalStrategyProxy;
-import org.apache.tinkerpop.gremlin.process.traversal.util.AndP;
-import org.apache.tinkerpop.gremlin.process.traversal.util.ConnectiveP;
-import org.apache.tinkerpop.gremlin.process.traversal.util.OrP;
-import org.apache.tinkerpop.gremlin.util.function.Lambda;
import org.apache.tinkerpop.shaded.jackson.core.JsonGenerator;
import org.apache.tinkerpop.shaded.jackson.core.JsonParser;
import org.apache.tinkerpop.shaded.jackson.core.JsonProcessingException;
-import org.apache.tinkerpop.shaded.jackson.core.JsonToken;
import org.apache.tinkerpop.shaded.jackson.databind.DeserializationContext;
import org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider;
import org.apache.tinkerpop.shaded.jackson.databind.deser.std.StdDeserializer;
-import org.apache.tinkerpop.shaded.jackson.databind.jsontype.TypeSerializer;
import org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdScalarSerializer;
-import org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer;
import java.io.IOException;
-import java.lang.reflect.Method;
-import java.util.Collection;
-import java.util.List;
-import java.util.Map;
-import java.util.Objects;
-import java.util.Set;
-import java.util.concurrent.ConcurrentHashMap;
/**
* @author Marko A. Rodriguez (http://markorodriguez.com)
@@ -63,26 +42,6 @@ final class TraversalSerializersV4 {
// SERIALIZERS //
////////////////
- final static class TraversalJacksonSerializer extends StdSerializer<Traversal> {
-
- public TraversalJacksonSerializer() {
- super(Traversal.class);
- }
-
- @Override
- public void serialize(final Traversal traversal, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider)
- throws IOException {
- jsonGenerator.writeObject(traversal.asAdmin().getGremlinLang());
- }
-
- @Override
- public void serializeWithType(final Traversal traversal, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider, final TypeSerializer typeSerializer)
- throws IOException {
- serialize(traversal, jsonGenerator, serializerProvider);
- }
-
- }
-
static class EnumJacksonSerializer extends StdScalarSerializer<Enum> {
public EnumJacksonSerializer() {
@@ -97,49 +56,6 @@ final class TraversalSerializersV4 {
}
- final static class PJacksonSerializer extends StdScalarSerializer<P> {
-
- public PJacksonSerializer() {
- super(P.class);
- }
-
- @Override
- public void serialize(final P p, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider)
- throws IOException {
- jsonGenerator.writeStartObject();
- jsonGenerator.writeStringField(GraphSONTokens.PREDICATE, p.getPredicateName());
- if (p instanceof ConnectiveP) {
- jsonGenerator.writeArrayFieldStart(GraphSONTokens.VALUE);
- for (final P<?> predicate : ((ConnectiveP<?>) p).getPredicates()) {
- jsonGenerator.writeObject(predicate);
- }
- jsonGenerator.writeEndArray();
- } else
- jsonGenerator.writeObjectField(GraphSONTokens.VALUE, p.getValue());
-
- jsonGenerator.writeEndObject();
- }
-
- }
-
- final static class LambdaJacksonSerializer extends StdScalarSerializer<Lambda> {
-
- public LambdaJacksonSerializer() {
- super(Lambda.class);
- }
-
- @Override
- public void serialize(final Lambda lambda, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider)
- throws IOException {
- jsonGenerator.writeStartObject();
- jsonGenerator.writeStringField(GraphSONTokens.SCRIPT, lambda.getLambdaScript());
- jsonGenerator.writeStringField(GraphSONTokens.LANGUAGE, lambda.getLambdaLanguage());
- jsonGenerator.writeNumberField(GraphSONTokens.ARGUMENTS, lambda.getLambdaArguments());
- jsonGenerator.writeEndObject();
- }
-
- }
-
final static class BulkSetJacksonSerializer extends StdScalarSerializer<BulkSet> {
public BulkSetJacksonSerializer() {
@@ -157,23 +73,6 @@ final class TraversalSerializersV4 {
}
}
- final static class TraversalStrategyJacksonSerializer extends StdScalarSerializer<TraversalStrategy> {
-
- public TraversalStrategyJacksonSerializer() {
- super(TraversalStrategy.class);
- }
-
- @Override
- public void serialize(final TraversalStrategy traversalStrategy, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider)
- throws IOException {
- jsonGenerator.writeStartObject();
- for (final Map.Entry<Object, Object> entry : ConfigurationConverter.getMap(traversalStrategy.getConfiguration()).entrySet()) {
- jsonGenerator.writeObjectField((String) entry.getKey(), entry.getValue());
- }
- jsonGenerator.writeEndObject();
- }
- }
-
///////////////////
// DESERIALIZERS //
//////////////////
@@ -200,200 +99,4 @@ final class TraversalSerializersV4 {
return true;
}
}
-
- final static class PJacksonDeserializer extends AbstractReflectJacksonDeserializer<P> {
-
- public PJacksonDeserializer() {
- super(P.class);
- }
-
- @Override
- public P deserialize(final JsonParser jsonParser, final DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
- String predicate = null;
- Object value = null;
-
- while (jsonParser.nextToken() != JsonToken.END_OBJECT) {
- if (jsonParser.getCurrentName().equals(GraphSONTokens.PREDICATE)) {
- jsonParser.nextToken();
- predicate = jsonParser.getText();
- } else if (jsonParser.getCurrentName().equals(GraphSONTokens.VALUE)) {
- jsonParser.nextToken();
- value = deserializationContext.readValue(jsonParser, Object.class);
- }
- }
-
- if (predicate.equals(GraphSONTokens.AND) || predicate.equals(GraphSONTokens.OR)) {
- return predicate.equals(GraphSONTokens.AND) ? new AndP((List<P>) value) : new OrP((List<P>) value);
- } else if (predicate.equals(GraphSONTokens.NOT) && value instanceof P) {
- return P.not((P<?>) value);
- } else {
- try {
- if (value instanceof Collection) {
- if (predicate.equals("between"))
- return P.between(((List) value).get(0), ((List) value).get(1));
- else if (predicate.equals("inside"))
- return P.between(((List) value).get(0), ((List) value).get(1));
- else if (predicate.equals("outside"))
- return P.outside(((List) value).get(0), ((List) value).get(1));
- else if (predicate.equals("within"))
- return P.within((Collection) value);
- else if (predicate.equals("without"))
- return P.without((Collection) value);
- else
- return (P) tryFindMethod(P.class, predicate, Collection.class).invoke(null, (Collection) value);
- } else {
- try {
- return (P) tryFindMethod(P.class, predicate, Object.class).invoke(null, value);
- } catch (final NoSuchMethodException e) {
- return (P) tryFindMethod(P.class, predicate, Object[].class).invoke(null, (Object) new Object[]{value});
- }
- }
- } catch (final Exception e) {
- throw new IllegalStateException(e.getMessage(), e);
- }
- }
- }
-
- @Override
- public boolean isCachable() {
- return true;
- }
- }
-
- /**
- * Deserializers that make reflection calls can use this class as a base and thus cache reflected methods to avoid
- * future lookups.
- */
- static abstract class AbstractReflectJacksonDeserializer<T> extends StdDeserializer<T> {
- private final Map<CacheKey, Method> CACHE = new ConcurrentHashMap<>();
-
- public AbstractReflectJacksonDeserializer(final Class<? extends T> clazz) {
- super(clazz);
- }
-
- protected Method tryFindMethod(final Class<?> base, final String methodName, final Class<?> parameterType) throws Exception {
- return CACHE.computeIfAbsent(new CacheKey(methodName, parameterType),
- cacheKey -> {
- try {
- return base.getMethod(methodName, parameterType);
- } catch (Exception e) {
- throw new IllegalStateException(e.getMessage(), e);
- }
- });
- }
-
- private static class CacheKey {
- private final String predicate;
- private final Class<?> parameterType;
-
- public CacheKey(final String predicate, final Class<?> parameterType) {
- this.predicate = Objects.requireNonNull(predicate);
- this.parameterType = Objects.requireNonNull(parameterType);
- }
-
- @Override
- public boolean equals(final Object o) {
- if (this == o) return true;
- if (o == null || getClass() != o.getClass()) return false;
- final CacheKey cacheKey = (CacheKey) o;
- return predicate.equals(cacheKey.predicate) &&
- parameterType.equals(cacheKey.parameterType);
- }
-
- @Override
- public int hashCode() {
- return Objects.hash(predicate, parameterType);
- }
- }
- }
-
- final static class TextPJacksonDeserializer extends AbstractReflectJacksonDeserializer<TextP> {
-
- public TextPJacksonDeserializer() {
- super(TextP.class);
- }
-
- @Override
- public TextP deserialize(final JsonParser jsonParser, final DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
- String predicate = null;
- String value = null;
-
- while (jsonParser.nextToken() != JsonToken.END_OBJECT) {
- if (jsonParser.getCurrentName().equals(GraphSONTokens.PREDICATE)) {
- jsonParser.nextToken();
- predicate = jsonParser.getText();
- } else if (jsonParser.getCurrentName().equals(GraphSONTokens.VALUE)) {
- jsonParser.nextToken();
- value = deserializationContext.readValue(jsonParser, String.class);
- }
- }
-
- try {
- return (TextP) tryFindMethod(TextP.class, predicate, String.class).invoke(null, value);
- } catch (final Exception e) {
- throw new IllegalStateException(e.getMessage(), e);
- }
- }
-
- @Override
- public boolean isCachable() {
- return true;
- }
- }
-
- final static class LambdaJacksonDeserializer extends StdDeserializer<Lambda> {
-
- public LambdaJacksonDeserializer() {
- super(Lambda.class);
- }
-
- @Override
- public Lambda deserialize(final JsonParser jsonParser, final DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
- String script = null;
- String language = null;
- int arguments = -1;
-
- while (jsonParser.nextToken() != JsonToken.END_OBJECT) {
- if (jsonParser.getCurrentName().equals(GraphSONTokens.SCRIPT)) {
- jsonParser.nextToken();
- script = jsonParser.getText();
- } else if (jsonParser.getCurrentName().equals(GraphSONTokens.LANGUAGE)) {
- jsonParser.nextToken();
- language = jsonParser.getText();
- } else if (jsonParser.getCurrentName().equals(GraphSONTokens.ARGUMENTS)) {
- jsonParser.nextToken();
- arguments = jsonParser.getIntValue();
- }
- }
-
- if (-1 == arguments || arguments > 2)
- return new Lambda.UnknownArgLambda(script, language, arguments);
- else if (0 == arguments)
- return new Lambda.ZeroArgLambda<>(script, language);
- else if (1 == arguments)
- return new Lambda.OneArgLambda<>(script, language);
- else
- return new Lambda.TwoArgLambda<>(script, language);
- }
-
- @Override
- public boolean isCachable() {
- return true;
- }
- }
-
- final static class TraversalStrategyProxyJacksonDeserializer<T extends TraversalStrategy> extends AbstractObjectDeserializer<TraversalStrategyProxy> {
-
- private final Class<T> clazz;
-
- public TraversalStrategyProxyJacksonDeserializer(final Class<T> clazz) {
- super(TraversalStrategyProxy.class);
- this.clazz = clazz;
- }
-
- @Override
- public TraversalStrategyProxy<T> createObject(final Map<String, Object> data) {
- return new TraversalStrategyProxy<>(this.clazz, new MapConfiguration(data));
- }
- }
}
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java
index 872df275e2..b35423c70b 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java
@@ -109,7 +109,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandleTraversalExplanation() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final TraversalExplanation o = __().out().outV().outE().explain();
final TraversalExplanation deser = serializeDeserialize(mapper, o, TraversalExplanation.class);
@@ -195,7 +195,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandleBiFunctionLambda() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final Lambda o = (Lambda) Lambda.biFunction("x,y -> 'test'");
assertEquals(o, serializeDeserialize(mapper, o, Lambda.class));
@@ -203,7 +203,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandleComparatorLambda() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final Lambda o = (Lambda) Lambda.comparator("x,y -> x <=> y");
assertEquals(o, serializeDeserialize(mapper, o, Lambda.class));
@@ -211,7 +211,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandleConsumerLambda() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final Lambda o = (Lambda) Lambda.consumer("x -> x");
assertEquals(o, serializeDeserialize(mapper, o, Lambda.class));
@@ -219,7 +219,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandleFunctionLambda() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final Lambda o = (Lambda) Lambda.function("x -> x");
assertEquals(o, serializeDeserialize(mapper, o, Lambda.class));
@@ -227,7 +227,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandlePredicateLambda() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final Lambda o = (Lambda) Lambda.predicate("x -> true");
assertEquals(o, serializeDeserialize(mapper, o, Lambda.class));
@@ -235,7 +235,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandleSupplierLambda() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final Lambda o = (Lambda) Lambda.supplier("'test'");
assertEquals(o, serializeDeserialize(mapper, o, Lambda.class));
@@ -249,30 +249,40 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandleInstant() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final Instant o = Instant.ofEpochMilli(System.currentTimeMillis());
assertEquals(o, serializeDeserialize(mapper, o, Instant.class));
}
@Test
public void shouldHandleLocalDate() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final LocalDate o = LocalDate.now();
assertEquals(o, serializeDeserialize(mapper, o, LocalDate.class));
}
@Test
public void shouldHandleLocalDateTime() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final LocalDateTime o = LocalDateTime.now();
assertEquals(o, serializeDeserialize(mapper, o, LocalDateTime.class));
}
@Test
public void shouldHandleLocalTime() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final LocalTime o = LocalTime.now();
assertEquals(o, serializeDeserialize(mapper, o, LocalTime.class));
}
@Test
public void shouldHandleMonthDay() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final MonthDay o = MonthDay.now();
assertEquals(o, serializeDeserialize(mapper, o, MonthDay.class));
}
@@ -285,36 +295,48 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandleOffsetTime() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final OffsetTime o = OffsetTime.now();
assertEquals(o, serializeDeserialize(mapper, o, OffsetTime.class));
}
@Test
public void shouldHandlePeriod() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final Period o = Period.ofDays(3);
assertEquals(o, serializeDeserialize(mapper, o, Period.class));
}
@Test
public void shouldHandleYear() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final Year o = Year.now();
assertEquals(o, serializeDeserialize(mapper, o, Year.class));
}
@Test
public void shouldHandleYearMonth() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final YearMonth o = YearMonth.now();
assertEquals(o, serializeDeserialize(mapper, o, YearMonth.class));
}
@Test
public void shouldHandleZonedDateTime() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final ZonedDateTime o = ZonedDateTime.now();
assertEquals(o, serializeDeserialize(mapper, o, ZonedDateTime.class));
}
@Test
public void shouldHandleZonedOffset() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final ZoneOffset o = ZonedDateTime.now().getOffset();
assertEquals(o, serializeDeserialize(mapper, o, ZoneOffset.class));
}
@@ -365,7 +387,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandlePMultiValue() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final P o = P.within(1,2,3,null);
assertEquals(o, serializeDeserialize(mapper, o, P.class));
@@ -373,7 +395,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandlePSingleValue() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final P o = P.within(1);
assertEquals(o, serializeDeserialize(mapper, o, P.class));
@@ -381,7 +403,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandlePMultiValueAsList() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final P o = P.within(Arrays.asList(1,2,3,null));
assertEquals(o, serializeDeserialize(mapper, o, P.class));
@@ -389,7 +411,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandlePMultiValueAsSet() throws Exception {
- assumeThat(version, either(startsWith("v3")).or(startsWith("v4")));
+ assumeThat(version, startsWith("v3"));
final P o = P.within(new HashSet<>(Arrays.asList(1,2,3)));
assertEquals(o, serializeDeserialize(mapper, o, P.class));
@@ -397,7 +419,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandlePBetween() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final P o = P.between(1, 100);
assertEquals(o, serializeDeserialize(mapper, o, P.class));
@@ -407,7 +429,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
public void shouldReadPWithJsonArray() throws Exception {
// for some reason v3 is forgiving about the naked json array - leaving this here for backward compatibility,
// but should be a g:List (i think)
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final P o = P.within(Arrays.asList(1,2,3));
assertEquals(o, mapper.readValue("{\"@type\": \"g:P\", \"@value\": {\"predicate\": \"within\", \"value\": [{\"@type\": \"g:Int32\", \"@value\": 1},{\"@type\": \"g:Int32\", \"@value\": 2},{\"@type\": \"g:Int32\", \"@value\": 3}]}}", Object.class));
@@ -452,7 +474,7 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandlePExt() throws Exception {
- assumeThat(version, not(startsWith("v1")));
+ assumeThat(version, either(startsWith("v2")).or(startsWith("v3")));
final P o = PExt.mix("bah");
assertEquals(o, serializeDeserialize(mapper, o, P.class));
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperPartialEmbeddedTypeTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperPartialEmbeddedTypeTest.java
index 5315c1a9ec..99b45a8685 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperPartialEmbeddedTypeTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperPartialEmbeddedTypeTest.java
@@ -31,6 +31,7 @@ import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.time.Instant;
+import java.time.OffsetDateTime;
import java.time.ZoneOffset;
import java.time.ZonedDateTime;
import java.util.ArrayList;
@@ -145,7 +146,7 @@ public class GraphSONMapperPartialEmbeddedTypeTest extends AbstractGraphSONTest
@Test
public void shouldFailIfTypeSpecifiedIsNotSameTypeInPayload() {
- final ZoneOffset o = ZonedDateTime.now().getOffset();
+ final OffsetDateTime o = OffsetDateTime.now();
final ByteArrayOutputStream stream = new ByteArrayOutputStream();
try {
mapper.writeValue(stream, o);
@@ -155,9 +156,9 @@ public class GraphSONMapperPartialEmbeddedTypeTest extends AbstractGraphSONTest
fail("Should have failed decoding the value");
} catch (Exception e) {
if (version.startsWith("v4")) {
- assertThat(e.getMessage(), containsString("Could not deserialize the JSON value as required. Nested exception: java.lang.InstantiationException: Cannot deserialize the value with the detected type contained in the JSON ('" + GraphSONTokens.GREMLIN_TYPE_NAMESPACE + ":ZoneOffset') to the type specified in parameter to the object mapper (class java.time.Instant). Those types are incompatible."));
+ assertThat(e.getMessage(), containsString("Could not deserialize the JSON value as required. Nested exception: java.lang.InstantiationException: Cannot deserialize the value with the detected type contained in the JSON ('" + GraphSONTokens.GREMLIN_TYPE_NAMESPACE + ":DateTime') to the type specified in parameter to the object mapper (class java.time.Instant). Those types are incompatible."));
} else {
- assertThat(e.getMessage(), containsString("Could not deserialize the JSON value as required. Nested exception: java.lang.InstantiationException: Cannot deserialize the value with the detected type contained in the JSON ('" + GraphSONTokens.GREMLINX_TYPE_NAMESPACE + ":ZoneOffset') to the type specified in parameter to the object mapper (class java.time.Instant). Those types are incompatible."));
+ assertThat(e.getMessage(), containsString("Could not deserialize the JSON value as required. Nested exception: java.lang.InstantiationException: Cannot deserialize the value with the detected type contained in the JSON ('" + GraphSONTokens.GREMLINX_TYPE_NAMESPACE + ":OffsetDateTime') to the type specified in parameter to the object mapper (class java.time.Instant). Those types are incompatible."));
}
}
}
@@ -276,6 +277,8 @@ public class GraphSONMapperPartialEmbeddedTypeTest extends AbstractGraphSONTest
@Test
public void shouldHandleVariantsOfP() throws Exception {
+ if (version.startsWith("v4")) return;
+
final List<P> variantsOfP = Arrays.asList(
P.between(1,2),
P.eq(1),
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperTest.java
index 052e95b15b..27ad64a369 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperTest.java
@@ -53,7 +53,10 @@ import java.util.HashMap;
import java.util.HashSet;
import static org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.__.__;
+import static org.hamcrest.core.IsNot.not;
+import static org.hamcrest.core.StringStartsWith.startsWith;
import static org.junit.Assert.assertEquals;
+import static org.junit.Assume.assumeThat;
/**
* @author Stephen Mallette (http://stephen.genoprime.com)
@@ -172,6 +175,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleTraversalExplanation() throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final TraversalExplanation te = __().out().outV().outE().explain();
final String json = mapper.writeValueAsString(te);
assertEquals("{\"original\":[\"InjectStep([])\",\"VertexStep(OUT,vertex)\",\"EdgeVertexStep(OUT)\",\"VertexStep(OUT,edge)\"],\"intermediate\":[],\"final\":[\"InjectStep([])\",\"VertexStep(OUT,vertex)\",\"EdgeVertexStep(OUT)\",\"VertexStep(OUT,edge)\"]}", json);
@@ -186,6 +191,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleInstant()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final Instant o = Instant.ofEpochMilli(System.currentTimeMillis());
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -193,6 +200,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleLocalDate()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final LocalDate o = LocalDate.now();
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -200,6 +209,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleLocalDateTime()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final LocalDateTime o = LocalDateTime.now();
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -207,6 +218,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleLocalTime()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final LocalTime o = LocalTime.now();
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -214,6 +227,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleMonthDay()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final MonthDay o = MonthDay.now();
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -228,6 +243,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleOffsetTime()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final OffsetTime o = OffsetTime.now();
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -235,6 +252,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandlePeriod()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final Period o = Period.ofDays(3);
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -242,6 +261,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleYear()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final Year o = Year.now();
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -249,6 +270,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleYearMonth()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final YearMonth o = YearMonth.now();
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -256,6 +279,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleZonedDateTime()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final ZonedDateTime o = ZonedDateTime.now();
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
@@ -263,6 +288,8 @@ public class GraphSONMapperTest {
@Test
public void shouldHandleZoneOffset()throws Exception {
+ assumeThat(version, not(startsWith("v4")));
+
final ZoneOffset o = ZonedDateTime.now().getOffset();
final String json = mapper.writeValueAsString(o);
assertEquals("\"" + o.toString() + "\"", json);
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerHttpIntegrateTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerHttpIntegrateTest.java
index 0ebcc28473..13018eaf36 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerHttpIntegrateTest.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerHttpIntegrateTest.java
@@ -61,7 +61,7 @@ import javax.script.SimpleBindings;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Field;
-import java.time.Instant;
+import java.time.OffsetDateTime;
import java.util.Base64;
import java.util.HashMap;
import java.util.List;
@@ -478,14 +478,14 @@ public class GremlinServerHttpIntegrateTest extends AbstractGremlinServerIntegra
final CloseableHttpClient httpclient = HttpClients.createDefault();
final HttpPost httppost = new HttpPost(TestClientFactory.createURLString());
httppost.addHeader("Content-Type", "application/json");
- httppost.setEntity(new StringEntity("{\"gremlin\":\"java.time.Instant.MAX\"}", Consts.UTF_8));
+ httppost.setEntity(new StringEntity("{\"gremlin\":\"java.time.OffsetDateTime.MAX\"}", Consts.UTF_8));
try (final CloseableHttpResponse response = httpclient.execute(httppost)) {
assertEquals(200, response.getStatusLine().getStatusCode());
assertEquals("application/json", response.getEntity().getContentType().getValue());
final String json = EntityUtils.toString(response.getEntity());
final JsonNode node = mapper.readTree(json);
- assertEquals(Instant.MAX, Instant.parse(node.get("result").get(TOKEN_DATA).get(GraphSONTokens.VALUEPROP).get(0).get(GraphSONTokens.VALUEPROP).asText()));
+ assertEquals(OffsetDateTime.MAX, OffsetDateTime.parse(node.get("result").get(TOKEN_DATA).get(GraphSONTokens.VALUEPROP).get(0).get(GraphSONTokens.VALUEPROP).asText()));
}
}
@@ -836,7 +836,7 @@ public class GremlinServerHttpIntegrateTest extends AbstractGremlinServerIntegra
assertEquals(SerTokens.MIME_JSON, response.getEntity().getContentType().getValue());
final String json = EntityUtils.toString(response.getEntity());
final JsonNode node = mapper.readTree(json);
- assertThat(node.get("status").get("message").asText(), startsWith("Error during serialization: Could not find a type identifier for the class"));
+ assertThat(node.get("status").get("message").asText(), startsWith("Error during serialization: No serializer found"));
}
}
(tinkerpop) 16/17: CTR remove unsupported types in GraphSONv4 in gremlin-python
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 0d1635b194fefc648a18028ce764a6e68186d6db
Author: Yang Xia <55...@users.noreply.github.com>
AuthorDate: Thu Oct 31 18:36:29 2024 -0700
CTR remove unsupported types in GraphSONv4 in gremlin-python
---
.../gremlin_python/structure/io/graphsonV4.py | 135 +--------------------
.../python/tests/structure/io/test_graphsonV4.py | 59 ---------
2 files changed, 6 insertions(+), 188 deletions(-)
diff --git a/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py b/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
index d20d3b4e31..87183d420f 100644
--- a/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
+++ b/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
@@ -147,32 +147,6 @@ class _GraphSONTypeIO(object, metaclass=GraphSONTypeType):
raise NotImplementedError()
-class _BytecodeSerializer(_GraphSONTypeIO):
- @classmethod
- def _dictify_instructions(cls, instructions, writer):
- out = []
- for instruction in instructions:
- inst = [instruction[0]]
- inst.extend(writer.to_dict(arg) for arg in instruction[1:])
- out.append(inst)
- return out
-
- @classmethod
- def dictify(cls, bytecode, writer):
- if isinstance(bytecode, Traversal):
- bytecode = bytecode.bytecode
- out = {}
- if bytecode.source_instructions:
- out["source"] = cls._dictify_instructions(bytecode.source_instructions, writer)
- if bytecode.step_instructions:
- out["step"] = cls._dictify_instructions(bytecode.step_instructions, writer)
- return GraphSONUtil.typed_value("Bytecode", out)
-
-
-class TraversalSerializer(_BytecodeSerializer):
- python_type = Traversal
-
-
class VertexSerializer(_GraphSONTypeIO):
python_type = Vertex
graphson_type = "g:Vertex"
@@ -222,71 +196,6 @@ class PropertySerializer(_GraphSONTypeIO):
"value": writer.to_dict(property.value)})
-class TraversalStrategySerializer(_GraphSONTypeIO):
- python_type = TraversalStrategy
-
- @classmethod
- def dictify(cls, strategy, writer):
- configuration = {}
- for key in strategy.configuration:
- configuration[key] = writer.to_dict(strategy.configuration[key])
- return GraphSONUtil.typed_value(strategy.strategy_name, configuration)
-
-
-class TraverserIO(_GraphSONTypeIO):
- python_type = Traverser
- graphson_type = "g:Traverser"
-
- @classmethod
- def dictify(cls, traverser, writer):
- return GraphSONUtil.typed_value("Traverser", {"value": writer.to_dict(traverser.object),
- "bulk": writer.to_dict(traverser.bulk)})
-
- @classmethod
- def objectify(cls, d, reader):
- return Traverser(reader.to_object(d["value"]),
- reader.to_object(d["bulk"]))
-
-
-class EnumSerializer(_GraphSONTypeIO):
- python_type = Enum
-
- @classmethod
- def dictify(cls, enum, _):
- return GraphSONUtil.typed_value(SymbolUtil.to_camel_case(type(enum).__name__),
- SymbolUtil.to_camel_case(str(enum.name)))
-
-
-class PSerializer(_GraphSONTypeIO):
- python_type = P
-
- @classmethod
- def dictify(cls, p, writer):
- out = {"predicate": p.operator,
- "value": [writer.to_dict(p.value), writer.to_dict(p.other)] if p.other is not None else
- writer.to_dict(p.value)}
- return GraphSONUtil.typed_value("P", out)
-
-
-class TextPSerializer(_GraphSONTypeIO):
- python_type = TextP
-
- @classmethod
- def dictify(cls, p, writer):
- out = {"predicate": p.operator,
- "value": [writer.to_dict(p.value), writer.to_dict(p.other)] if p.other is not None else
- writer.to_dict(p.value)}
- return GraphSONUtil.typed_value("TextP", out)
-
-
-class TypeSerializer(_GraphSONTypeIO):
- python_type = TypeType
-
- @classmethod
- def dictify(cls, typ, writer):
- return writer.to_dict(typ())
-
-
class UUIDIO(_GraphSONTypeIO):
python_type = uuid.UUID
graphson_type = "g:UUID"
@@ -403,27 +312,6 @@ class MapType(_GraphSONTypeIO):
return new_dict
-class BulkSetIO(_GraphSONTypeIO):
- graphson_type = "g:BulkSet"
-
- @classmethod
- def objectify(cls, l, reader):
- new_list = []
-
- # this approach basically mimics what currently existed in 3.3.4 and prior versions where BulkSet is
- # basically just coerced to list. the limitation here is that if the value of a bulk exceeds the size of
- # a list (into the long space) then stuff won't work nice.
- if len(l) > 0:
- x = 0
- while x < len(l):
- obj = reader.to_object(l[x])
- bulk = reader.to_object(l[x + 1])
- for y in range(bulk):
- new_list.append(obj)
- x = x + 2
- return new_list
-
-
class FloatIO(_NumberIO):
python_type = FloatType
graphson_type = "g:Float"
@@ -664,9 +552,14 @@ class PathDeserializer(_GraphSONTypeIO):
return Path(reader.to_object(d["labels"]), reader.to_object(d["objects"]))
-class TDeserializer(_GraphSONTypeIO):
+class TIO(_GraphSONTypeIO):
graphson_type = "g:T"
+ graphson_base_type = "T"
+ python_type = T
+ @classmethod
+ def dictify(cls, t, writer):
+ return GraphSONUtil.typed_value(cls.graphson_base_type, t.name, "g")
@classmethod
def objectify(cls, d, reader):
return T[d]
@@ -684,19 +577,3 @@ class DirectionIO(_GraphSONTypeIO):
@classmethod
def objectify(cls, d, reader):
return Direction[d]
-
-
-class TraversalMetricsDeserializer(_GraphSONTypeIO):
- graphson_type = "g:TraversalMetrics"
-
- @classmethod
- def objectify(cls, d, reader):
- return reader.to_object(d)
-
-
-class MetricsDeserializer(_GraphSONTypeIO):
- graphson_type = "g:Metrics"
-
- @classmethod
- def objectify(cls, d, reader):
- return reader.to_object(d)
diff --git a/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py b/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py
index 44b2fa7187..7d88e73782 100644
--- a/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py
+++ b/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py
@@ -77,15 +77,6 @@ class TestGraphSONReader:
assert x['b'] == "marko"
assert len(x) == 2
- # BulkSet gets coerced to a List - both have the same behavior
- x = self.graphson_reader.read_object(
- json.dumps({"@type": "g:BulkSet",
- "@value": ["marko", {"@type": "g:Int64", "@value": 1}, "josh", {"@type": "g:Int64", "@value": 3}]}))
- assert isinstance(x, list)
- assert len(x) == 4
- assert x.count("marko") == 1
- assert x.count("josh") == 3
-
def test_number_input(self):
x = self.graphson_reader.read_object(json.dumps({
"@type": "g:Byte",
@@ -290,18 +281,6 @@ class TestGraphSONReader:
assert isinstance(prop, uuid.UUID)
assert str(prop) == '41d2e28a-20a4-4ab0-b379-d810dede3786'
- def test_metrics(self):
- prop = self.graphson_reader.read_object(
- json.dumps([{'@type': 'g:TraversalMetrics', '@value': {'dur': 1.468594, 'metrics': [
- {'@type': 'g:Metrics', '@value': {'dur': 1.380957, 'counts': {}, 'name': 'GraphStep(__.V())', 'annotations': {'percentDur': 94.03259171697556}, 'id': '4.0.0()'}},
- {'@type': 'g:Metrics', '@value': {'dur': 0.087637, 'counts': {}, 'name': 'ReferenceElementStep', 'annotations': {'percentDur': 5.967408283024444}, 'id': '3.0.0()'}}
- ]}}]))
- assert isinstance(prop, list)
- assert prop == [{'dur': 1.468594, 'metrics': [
- {'dur': 1.380957, 'counts': {}, 'name': 'GraphStep(__.V())', 'annotations': {'percentDur': 94.03259171697556}, 'id': '4.0.0()'},
- {'dur': 0.087637, 'counts': {}, 'name': 'ReferenceElementStep', 'annotations': {'percentDur': 5.967408283024444}, 'id': '3.0.0()'}
- ]}]
-
def test_binary(self):
bb = self.graphson_reader.read_object(
json.dumps({"@type": "g:Binary", "@value": "c29tZSBieXRlcyBmb3IgeW91"}))
@@ -356,47 +335,9 @@ class TestGraphSONWriter:
assert """true""" == self.graphson_writer.write_object(True)
def test_enum(self):
- assert {"@type": "g:Merge", "@value": "onMatch"} == json.loads(self.graphson_writer.write_object(Merge.on_match))
- assert {"@type": "g:Order", "@value": "shuffle"} == json.loads(self.graphson_writer.write_object(Order.shuffle))
- assert {"@type": "g:Barrier", "@value": "normSack"} == json.loads(self.graphson_writer.write_object(Barrier.norm_sack))
- assert {"@type": "g:Operator", "@value": "sum"} == json.loads(self.graphson_writer.write_object(Operator.sum_))
- assert {"@type": "g:Operator", "@value": "sumLong"} == json.loads(self.graphson_writer.write_object(Operator.sum_long))
assert {"@type": "g:Direction", "@value": "OUT"} == json.loads(self.graphson_writer.write_object(Direction.OUT))
assert {"@type": "g:Direction", "@value": "OUT"} == json.loads(self.graphson_writer.write_object(Direction.from_))
- def test_P(self):
- result = {'@type': 'g:P',
- '@value': {
- 'predicate': 'and',
- 'value': [{
- '@type': 'g:P',
- '@value': {
- 'predicate': 'or',
- 'value': [{
- '@type': 'g:P',
- '@value': {'predicate': 'lt', 'value': 'b'}
- },
- {'@type': 'g:P', '@value': {'predicate': 'gt', 'value': 'c'}}
- ]
- }
- },
- {'@type': 'g:P', '@value': {'predicate': 'neq', 'value': 'd'}}]}}
-
- assert result == json.loads(
- self.graphson_writer.write_object(P.lt("b").or_(P.gt("c")).and_(P.neq("d"))))
-
- result = {'@type': 'g:P', '@value': {'predicate': 'within', 'value': {'@type': 'g:List', '@value': [
- {"@type": "g:Int32", "@value": 1}, {"@type": "g:Int32", "@value": 2}]}}}
- assert result == json.loads(self.graphson_writer.write_object(P.within([1, 2])))
- assert result == json.loads(self.graphson_writer.write_object(P.within({1, 2})))
- assert result == json.loads(self.graphson_writer.write_object(P.within(1, 2)))
-
- result = {'@type': 'g:P', '@value': {'predicate': 'within', 'value': {'@type': 'g:List', '@value': [
- {"@type": "g:Int32", "@value": 1}]}}}
- assert result == json.loads(self.graphson_writer.write_object(P.within([1])))
- assert result == json.loads(self.graphson_writer.write_object(P.within({1})))
- assert result == json.loads(self.graphson_writer.write_object(P.within(1)))
-
def test_graph(self):
assert {"@type": "g:Vertex",
"@value": {"id": {"@type": "g:Int64", "@value": 12}, "label": ["person"]}} == json.loads(
(tinkerpop) 02/17: Rename GraphSONv4 g:OffsetDateTime to g:DateTime CTR
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit ae76c17e1eb38b8ef8a5345d20204e27246651f4
Author: Ken Hu <10...@users.noreply.github.com>
AuthorDate: Wed Oct 16 21:00:11 2024 -0700
Rename GraphSONv4 g:OffsetDateTime to g:DateTime CTR
In GraphSONv4, the newly created g:DateTime has the same underlying JSON
representation as the old g:OffsetDateTime; an ISO-8601 String.
---
.../gremlin/structure/io/graphson/GraphSONXModuleV4.java | 2 +-
.../gremlin/structure/io/graphson/max-offsetdatetime-v4.json | 2 +-
.../gremlin/structure/io/graphson/min-offsetdatetime-v4.json | 2 +-
.../tinkerpop/gremlin/structure/io/graphson/var-type-map-v4.json | 8 ++++----
.../gremlin/structure/io/AbstractTypedCompatibilityTest.java | 1 -
5 files changed, 7 insertions(+), 8 deletions(-)
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONXModuleV4.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONXModuleV4.java
index 2f9812c147..aa7ad5e9ad 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONXModuleV4.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONXModuleV4.java
@@ -63,7 +63,7 @@ public final class GraphSONXModuleV4 extends GraphSONModule {
put(LocalDateTime.class, "LocalDateTime");
put(LocalTime.class, "LocalTime");
put(MonthDay.class, "MonthDay");
- put(OffsetDateTime.class, "OffsetDateTime");
+ put(OffsetDateTime.class, "DateTime");
put(OffsetTime.class, "OffsetTime");
put(Period.class, "Period");
put(Year.class, "Year");
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/max-offsetdatetime-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/max-offsetdatetime-v4.json
index 6fc7c25fe6..022ea63ff8 100644
--- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/max-offsetdatetime-v4.json
+++ b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/max-offsetdatetime-v4.json
@@ -1,4 +1,4 @@
{
- "@type" : "g:OffsetDateTime",
+ "@type" : "g:DateTime",
"@value" : "+999999999-12-31T23:59:59.999999999-18:00"
}
\ No newline at end of file
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/min-offsetdatetime-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/min-offsetdatetime-v4.json
index d45954b5bf..2b73140486 100644
--- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/min-offsetdatetime-v4.json
+++ b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/min-offsetdatetime-v4.json
@@ -1,4 +1,4 @@
{
- "@type" : "g:OffsetDateTime",
+ "@type" : "g:DateTime",
"@value" : "-999999999-01-01T00:00+18:00"
}
\ No newline at end of file
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-type-map-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-type-map-v4.json
index 5f551eb5d4..e920264240 100644
--- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-type-map-v4.json
+++ b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-type-map-v4.json
@@ -13,13 +13,13 @@
"@value" : 3
} ]
}, {
- "@type" : "g:Date",
- "@value" : 1481295
+ "@type" : "g:DateTime",
+ "@value" : "1970-01-01T00:24:41.295Z"
}, "test", {
"@type" : "g:Int32",
"@value" : 123
}, {
- "@type" : "g:Date",
- "@value" : 1481295
+ "@type" : "g:DateTime",
+ "@value" : "1970-01-01T00:24:41.295Z"
}, "red" ]
}
\ No newline at end of file
diff --git a/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java b/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java
index e39869d474..3b7bdf9093 100644
--- a/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java
+++ b/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java
@@ -536,7 +536,6 @@ public abstract class AbstractTypedCompatibilityTest extends AbstractCompatibili
}
@Test
- @Ignore("re-enable after GraphSONV4 datetime is implemented")
public void shouldReadWriteMultiTypeMap() throws Exception {
final String resourceName = "var-type-map";
(tinkerpop) 03/17: Re-enable GraphSON message serializer for debugging. (#2836)
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 55f7e180c9284d29c2b136667aea7e0298f16ff1
Author: kenhuuu <10...@users.noreply.github.com>
AuthorDate: Thu Oct 17 12:47:38 2024 -0700
Re-enable GraphSON message serializer for debugging. (#2836)
The GraphSONv4 MessageSerializer doesn't support chunking so the
HttpObjectAggregator is added back and the
HttpGremlinResponseStreamDecoder is replaced with the original,
non-streaming version. This is mainly for debugging and testing purposes
so lowered performance is expected.
---
docs/src/reference/gremlin-variants.asciidoc | 2 +-
.../tinkerpop/gremlin/driver/Channelizer.java | 10 ++-
.../apache/tinkerpop/gremlin/driver/Cluster.java | 17 ++++-
.../driver/handler/HttpGremlinResponseDecoder.java | 87 ++++++++++++++++++++++
...tor.java => PayloadSerializingInterceptor.java} | 17 +++--
.../gremlin/driver/simple/SimpleHttpClient.java | 5 +-
.../tinkerpop/gremlin/driver/ClusterTest.java | 4 +-
.../gremlin/server/GremlinDriverIntegrateTest.java | 47 +++++++++++-
.../gremlin/server/GremlinServerIntegrateTest.java | 6 +-
.../gremlin/server/TestClientFactory.java | 5 ++
10 files changed, 178 insertions(+), 22 deletions(-)
diff --git a/docs/src/reference/gremlin-variants.asciidoc b/docs/src/reference/gremlin-variants.asciidoc
index d148a84ff8..9868582b4c 100644
--- a/docs/src/reference/gremlin-variants.asciidoc
+++ b/docs/src/reference/gremlin-variants.asciidoc
@@ -822,7 +822,7 @@ The following table describes the various configuration options for the Gremlin
|path |The URL path to the Gremlin Server. |_/gremlin_
|port |The port of the Gremlin Server to connect to. The same port will be applied for all hosts. |8192
|protocol |Sets the `AuthProperties.Property.PROTOCOL` properties for authentication to Gremlin Server. |_none_
-|serializer.className |The fully qualified class name of the `MessageSerializer` that will be used to communicate with the server. Note that the serializer configured on the client should be supported by the server configuration. |_none_
+|serializer.className |The fully qualified class name of the `MessageSerializer` that will be used to deserialize responses from the server. Note that the serializer configured on the client should be supported by the server configuration. |_none_
|serializer.config |A `Map` of configuration settings for the serializer. |_none_
|username |The username to submit on requests that require authentication. |_none_
|workerPoolSize |Size of the pool for handling background work. |available processors * 2
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Channelizer.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Channelizer.java
index c43b80c7d2..b64f969838 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Channelizer.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Channelizer.java
@@ -24,6 +24,7 @@ import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.channel.socket.SocketChannel;
import io.netty.handler.codec.http.HttpClientCodec;
+import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http.HttpResponseStatus;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslHandler;
@@ -31,7 +32,7 @@ import org.apache.tinkerpop.gremlin.driver.exception.ConnectionException;
import org.apache.tinkerpop.gremlin.driver.handler.GremlinResponseHandler;
import org.apache.tinkerpop.gremlin.driver.handler.HttpContentDecompressionHandler;
import org.apache.tinkerpop.gremlin.driver.handler.HttpGremlinRequestEncoder;
-import org.apache.tinkerpop.gremlin.driver.handler.HttpGremlinResponseStreamDecoder;
+import org.apache.tinkerpop.gremlin.driver.handler.HttpGremlinResponseDecoder;
import org.apache.tinkerpop.gremlin.driver.handler.SslCheckHandler;
import org.apache.tinkerpop.gremlin.util.message.ResponseMessage;
@@ -95,6 +96,7 @@ public interface Channelizer extends ChannelHandler {
public static final String PIPELINE_SSL_HANDLER = "gremlin-ssl-handler";
protected static final String PIPELINE_HTTP_CODEC = "http-codec";
+ protected static final String PIPELINE_HTTP_AGGREGATOR = "http-aggregator";
protected static final String PIPELINE_HTTP_ENCODER = "gremlin-encoder";
protected static final String PIPELINE_HTTP_DECODER = "gremlin-decoder";
protected static final String PIPELINE_HTTP_DECOMPRESSION_HANDLER = "http-decompression-handler";
@@ -182,7 +184,7 @@ public interface Channelizer extends ChannelHandler {
ResponseMessage.build().code(HttpResponseStatus.NO_CONTENT).result(Collections.emptyList()).create();
private HttpGremlinRequestEncoder gremlinRequestEncoder;
- private HttpGremlinResponseStreamDecoder gremlinResponseDecoder;
+ private HttpGremlinResponseDecoder gremlinResponseDecoder;
private HttpContentDecompressionHandler httpCompressionDecoder;
@@ -193,7 +195,7 @@ public interface Channelizer extends ChannelHandler {
httpCompressionDecoder = new HttpContentDecompressionHandler();
gremlinRequestEncoder = new HttpGremlinRequestEncoder(cluster.getSerializer(), cluster.getRequestInterceptors(),
cluster.isUserAgentOnConnectEnabled(), cluster.isBulkingEnabled(), connection.getUri());
- gremlinResponseDecoder = new HttpGremlinResponseStreamDecoder(cluster.getSerializer(), cluster.getMaxResponseContentLength());
+ gremlinResponseDecoder = new HttpGremlinResponseDecoder(cluster.getSerializer());
}
@Override
@@ -222,6 +224,8 @@ public interface Channelizer extends ChannelHandler {
DEFAULT_ALLOW_DUPLICATE_CONTENT_LENGTHS, false);
pipeline.addLast(PIPELINE_HTTP_CODEC, handler);
+ pipeline.addLast(PIPELINE_HTTP_AGGREGATOR, new HttpObjectAggregator(cluster.getMaxResponseContentLength() > 0
+ ? (int) cluster.getMaxResponseContentLength() : Integer.MAX_VALUE));
pipeline.addLast(PIPELINE_HTTP_ENCODER, gremlinRequestEncoder);
pipeline.addLast(PIPELINE_HTTP_DECOMPRESSION_HANDLER, httpCompressionDecoder);
pipeline.addLast(PIPELINE_HTTP_DECODER, gremlinResponseDecoder);
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Cluster.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Cluster.java
index 6f18c394e1..8d14d9cf53 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Cluster.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Cluster.java
@@ -31,9 +31,10 @@ import org.apache.commons.configuration2.Configuration;
import org.apache.commons.lang3.concurrent.BasicThreadFactory;
import org.apache.commons.lang3.tuple.Pair;
import org.apache.tinkerpop.gremlin.driver.auth.Auth;
-import org.apache.tinkerpop.gremlin.driver.interceptor.GraphBinarySerializationInterceptor;
+import org.apache.tinkerpop.gremlin.driver.interceptor.PayloadSerializingInterceptor;
import org.apache.tinkerpop.gremlin.util.MessageSerializer;
import org.apache.tinkerpop.gremlin.util.message.RequestMessage;
+import org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV4;
import org.apache.tinkerpop.gremlin.util.ser.Serializers;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -151,6 +152,10 @@ public final class Cluster {
return new Builder(address);
}
+ public static Builder build(final RequestInterceptor serializingInterceptor) {
+ return new Builder(serializingInterceptor);
+ }
+
public static Builder build(final File configurationFile) throws FileNotFoundException {
final Settings settings = Settings.read(new FileInputStream(configurationFile));
return getBuilderFromSettings(settings);
@@ -511,12 +516,18 @@ public final class Cluster {
private boolean enableBulkedResult = false;
private Builder() {
- addInterceptor(SERIALIZER_INTERCEPTOR_NAME, new GraphBinarySerializationInterceptor());
+ addInterceptor(SERIALIZER_INTERCEPTOR_NAME,
+ new PayloadSerializingInterceptor(new GraphBinaryMessageSerializerV4()));
}
private Builder(final String address) {
addContactPoint(address);
- addInterceptor(SERIALIZER_INTERCEPTOR_NAME, new GraphBinarySerializationInterceptor());
+ addInterceptor(SERIALIZER_INTERCEPTOR_NAME,
+ new PayloadSerializingInterceptor(new GraphBinaryMessageSerializerV4()));
+ }
+
+ private Builder(final RequestInterceptor bodySerializer) {
+ addInterceptor(SERIALIZER_INTERCEPTOR_NAME, bodySerializer);
}
/**
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinResponseDecoder.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinResponseDecoder.java
new file mode 100644
index 0000000000..3bd1719342
--- /dev/null
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinResponseDecoder.java
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.driver.handler;
+
+import io.netty.buffer.ByteBuf;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.handler.codec.MessageToMessageDecoder;
+import io.netty.handler.codec.TooLongFrameException;
+import io.netty.handler.codec.http.DefaultHttpObject;
+import io.netty.handler.codec.http.FullHttpResponse;
+import io.netty.handler.codec.http.HttpContent;
+import io.netty.handler.codec.http.HttpHeaderNames;
+import io.netty.handler.codec.http.HttpResponse;
+import io.netty.handler.codec.http.HttpResponseStatus;
+import io.netty.handler.codec.http.LastHttpContent;
+import io.netty.util.Attribute;
+import io.netty.util.AttributeKey;
+import io.netty.util.AttributeMap;
+import io.netty.util.CharsetUtil;
+import org.apache.tinkerpop.gremlin.util.MessageSerializer;
+import org.apache.tinkerpop.gremlin.util.message.ResponseMessage;
+import org.apache.tinkerpop.gremlin.util.ser.SerTokens;
+import org.apache.tinkerpop.gremlin.util.ser.SerializationException;
+import org.apache.tinkerpop.shaded.jackson.databind.JsonNode;
+import org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper;
+
+import java.util.List;
+
+import static org.apache.tinkerpop.gremlin.driver.Channelizer.HttpChannelizer.LAST_CONTENT_READ_RESPONSE;
+import static org.apache.tinkerpop.gremlin.driver.handler.HttpGremlinResponseStreamDecoder.IS_BULKED;
+
+public class HttpGremlinResponseDecoder extends MessageToMessageDecoder<FullHttpResponse> {
+ private static final String MESSAGE_NAME = "message";
+ private final MessageSerializer<?> serializer;
+ private final ObjectMapper mapper = new ObjectMapper();
+
+ public HttpGremlinResponseDecoder(final MessageSerializer<?> serializer) {
+ this.serializer = serializer;
+ }
+
+ @Override
+ protected void decode(ChannelHandlerContext ctx, FullHttpResponse msg, List<Object> out) throws Exception {
+ final ByteBuf content = msg.content();
+ ResponseMessage response;
+
+ try {
+ // no more chunks expected
+ if (isError(msg.status()) && !serializer.mimeTypesSupported()[0].equals(msg.headers().get(HttpHeaderNames.CONTENT_TYPE))) {
+ final String json = content.toString(CharsetUtil.UTF_8);
+ final JsonNode node = mapper.readTree(json);
+ final String message = node.has(MESSAGE_NAME) ? node.get(MESSAGE_NAME).asText() : "";
+ response = ResponseMessage.build()
+ .code(msg.status())
+ .statusMessage(message.isEmpty() ? msg.status().reasonPhrase() : message)
+ .create();
+ } else {
+ response = serializer.deserializeBinaryResponse(content);
+ }
+
+ ctx.channel().attr(IS_BULKED).set(response.getResult().isBulked());
+ out.add(response);
+ out.add(LAST_CONTENT_READ_RESPONSE);
+ } catch (SerializationException e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ private static boolean isError(final HttpResponseStatus status) {
+ return status != HttpResponseStatus.OK;
+ }
+}
\ No newline at end of file
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/interceptor/GraphBinarySerializationInterceptor.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/interceptor/PayloadSerializingInterceptor.java
similarity index 82%
rename from gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/interceptor/GraphBinarySerializationInterceptor.java
rename to gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/interceptor/PayloadSerializingInterceptor.java
index a6e656eea8..13616177c1 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/interceptor/GraphBinarySerializationInterceptor.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/interceptor/PayloadSerializingInterceptor.java
@@ -29,14 +29,21 @@ import org.apache.tinkerpop.gremlin.util.MessageSerializer;
import org.apache.tinkerpop.gremlin.util.message.RequestMessage;
import org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV4;
import org.apache.tinkerpop.gremlin.util.ser.SerializationException;
+import org.apache.tinkerpop.gremlin.util.ser.Serializers;
+
+import java.util.Map;
/**
- * A {@link RequestInterceptor} that serializes the request body to the {@code GraphBinary} format. This interceptor
- * should be run before other interceptors that need to calculate values based on the request body.
+ * A {@link RequestInterceptor} that serializes the request body usng the provided {@link MessageSerializer}. This
+ * interceptor should be run before other interceptors that need to calculate values based on the request body.
*/
-public class GraphBinarySerializationInterceptor implements RequestInterceptor {
- // Should be thread-safe as the GraphBinaryWriter doesn't maintain state.
- private static final MessageSerializer serializer = new GraphBinaryMessageSerializerV4();
+public class PayloadSerializingInterceptor implements RequestInterceptor {
+ // Should be thread-safe as the GraphBinaryWriter/GraphSONMessageSerializer doesn't maintain state.
+ private final MessageSerializer serializer;
+
+ public PayloadSerializingInterceptor(final MessageSerializer serializer) {
+ this.serializer = serializer;
+ }
@Override
public HttpRequest apply(HttpRequest httpRequest) {
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/simple/SimpleHttpClient.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/simple/SimpleHttpClient.java
index b2d932b2e2..232f94d1f4 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/simple/SimpleHttpClient.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/simple/SimpleHttpClient.java
@@ -28,7 +28,7 @@ import org.apache.tinkerpop.gremlin.driver.Channelizer;
import org.apache.tinkerpop.gremlin.driver.handler.HttpContentDecompressionHandler;
import org.apache.tinkerpop.gremlin.driver.handler.HttpGremlinResponseStreamDecoder;
import org.apache.tinkerpop.gremlin.driver.handler.HttpGremlinRequestEncoder;
-import org.apache.tinkerpop.gremlin.driver.interceptor.GraphBinarySerializationInterceptor;
+import org.apache.tinkerpop.gremlin.driver.interceptor.PayloadSerializingInterceptor;
import org.apache.tinkerpop.gremlin.util.MessageSerializer;
import org.apache.tinkerpop.gremlin.util.message.RequestMessage;
import io.netty.bootstrap.Bootstrap;
@@ -110,7 +110,8 @@ public class SimpleHttpClient extends AbstractClient {
new HttpGremlinResponseStreamDecoder(serializer, Integer.MAX_VALUE),
new HttpGremlinRequestEncoder(serializer,
Collections.singletonList(
- Pair.of("serializer", new GraphBinarySerializationInterceptor())),
+ Pair.of("serializer", new PayloadSerializingInterceptor(
+ new GraphBinaryMessageSerializerV4()))),
false, false, uri),
callbackResponseHandler);
}
diff --git a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ClusterTest.java b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ClusterTest.java
index fff9485a2a..4a56ed7f06 100644
--- a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ClusterTest.java
+++ b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ClusterTest.java
@@ -19,7 +19,7 @@
package org.apache.tinkerpop.gremlin.driver;
import org.apache.commons.lang3.tuple.Pair;
-import org.apache.tinkerpop.gremlin.driver.interceptor.GraphBinarySerializationInterceptor;
+import org.apache.tinkerpop.gremlin.driver.interceptor.PayloadSerializingInterceptor;
import org.junit.Test;
import java.util.List;
@@ -132,7 +132,7 @@ public class ClusterTest {
public void shouldContainBodySerializerByDefault() {
final List<Pair<String, ? extends RequestInterceptor>> interceptors = Cluster.build().create().getRequestInterceptors();
assertEquals(1, interceptors.size());
- assertTrue(interceptors.get(0).getRight() instanceof GraphBinarySerializationInterceptor);
+ assertTrue(interceptors.get(0).getRight() instanceof PayloadSerializingInterceptor);
}
@Test
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
index b14b49959a..6dc11483f1 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
@@ -33,6 +33,7 @@ import org.apache.tinkerpop.gremlin.driver.Result;
import org.apache.tinkerpop.gremlin.driver.ResultSet;
import org.apache.tinkerpop.gremlin.driver.exception.NoHostAvailableException;
import org.apache.tinkerpop.gremlin.driver.exception.ResponseException;
+import org.apache.tinkerpop.gremlin.driver.interceptor.PayloadSerializingInterceptor;
import org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin;
import org.apache.tinkerpop.gremlin.server.channel.HttpChannelizer;
import org.apache.tinkerpop.gremlin.structure.Vertex;
@@ -44,6 +45,7 @@ import org.apache.tinkerpop.gremlin.util.TimeUtil;
import org.apache.tinkerpop.gremlin.util.function.FunctionUtils;
import org.apache.tinkerpop.gremlin.util.message.RequestMessage;
import org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV4;
+import org.apache.tinkerpop.gremlin.util.ser.GraphSONMessageSerializerV4;
import org.apache.tinkerpop.gremlin.util.ser.Serializers;
import org.junit.AfterClass;
import org.junit.Before;
@@ -208,6 +210,45 @@ public class GremlinDriverIntegrateTest extends AbstractGremlinServerIntegration
}
}
+ @Test
+ public void shouldWorkWithGraphSONSerializer() throws Exception {
+ final Cluster cluster = TestClientFactory.build(new PayloadSerializingInterceptor(new GraphSONMessageSerializerV4()))
+ .serializer(Serializers.GRAPHSON_V4.simpleInstance()).create();
+
+ try {
+ final Client client = cluster.connect();
+ assertEquals(2, client.submit("g.inject(2)").all().get().get(0).getInt());
+ } finally {
+ cluster.close();
+ }
+ }
+
+ @Test
+ public void shouldWorkWithGraphSONRequestAndGraphBinaryResponse() throws Exception {
+ final Cluster cluster = TestClientFactory.build(new PayloadSerializingInterceptor(new GraphSONMessageSerializerV4()))
+ .serializer(Serializers.GRAPHBINARY_V4.simpleInstance()).create();
+
+ try {
+ final Client client = cluster.connect();
+ assertEquals(3, client.submit("g.inject(3)").all().get().get(0).getInt());
+ } finally {
+ cluster.close();
+ }
+ }
+
+ @Test
+ public void shouldWorkWithGraphBinaryRequestAndGraphSONResponse() throws Exception {
+ final Cluster cluster = TestClientFactory.build(new PayloadSerializingInterceptor(new GraphBinaryMessageSerializerV4()))
+ .serializer(Serializers.GRAPHSON_V4.simpleInstance()).create();
+
+ try {
+ final Client client = cluster.connect();
+ assertEquals(5, client.submit("g.inject(5)").all().get().get(0).getInt());
+ } finally {
+ cluster.close();
+ }
+ }
+
@Test
public void shouldInterceptRequestsWithHandshake() throws Exception {
final int requestsToMake = 32;
@@ -296,7 +337,7 @@ public class GremlinDriverIntegrateTest extends AbstractGremlinServerIntegration
public void shouldEventuallySucceedAfterChannelLevelError() {
final Cluster cluster = TestClientFactory.build()
.reconnectInterval(500)
- .maxResponseContentLength(64).create();
+ .maxResponseContentLength(32).create(); // Warning: compression can change the content length. Adjust as needed.
final Client client = cluster.connect();
try {
@@ -305,7 +346,7 @@ public class GremlinDriverIntegrateTest extends AbstractGremlinServerIntegration
fail("Request should have failed because it exceeded the max content length allowed");
} catch (Exception ex) {
final Throwable root = ExceptionHelper.getRootCause(ex);
- assertThat(root.getMessage(), containsString("Response exceeded 64 bytes."));
+ assertThat(root.getMessage(), containsString("Response entity too large"));
}
assertEquals(2, client.submit("1+1").all().join().get(0).getInt());
@@ -786,7 +827,7 @@ public class GremlinDriverIntegrateTest extends AbstractGremlinServerIntegration
fail("Should throw an exception.");
} catch (Exception re) {
final Throwable root = ExceptionHelper.getRootCause(re);
- assertTrue(root.getMessage().equals("Response exceeded 1 bytes."));
+ assertTrue(root.getMessage().contains("Response entity too large"));
} finally {
cluster.close();
}
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
index 14abcdf089..a8a17f3c3f 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
@@ -749,9 +749,9 @@ public class GremlinServerIntegrateTest extends AbstractGremlinServerIntegration
} catch (Exception re) {
final Throwable root = ExceptionHelper.getRootCause(re);
- // went with two possible error messages here as i think that there is some either non-deterministic
- // behavior around the error message or it's environmentally dependent (e.g. different jdk, versions, etc)
- assertThat(root.getMessage(), Matchers.anyOf(is("Connection to server is no longer active"), is("Connection reset by peer")));
+ // the server sends back a 413 Request Entity Too Large now in HTTP so detect that rather than the
+ // underlying connection closing due to error.
+ assertThat(root.getMessage(), Matchers.anyOf(is("Request Entity Too Large")));
// validate that we can still send messages to the server
assertEquals(2, client.submit("1+1").all().join().get(0).getInt());
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/TestClientFactory.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/TestClientFactory.java
index c8f1f4c6f1..ff7de898ca 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/TestClientFactory.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/TestClientFactory.java
@@ -19,6 +19,7 @@
package org.apache.tinkerpop.gremlin.server;
import org.apache.tinkerpop.gremlin.driver.Cluster;
+import org.apache.tinkerpop.gremlin.driver.RequestInterceptor;
import org.apache.tinkerpop.gremlin.driver.simple.SimpleHttpClient;
import java.net.URI;
@@ -42,6 +43,10 @@ public final class TestClientFactory {
return Cluster.build(address).port(PORT);
}
+ public static Cluster.Builder build(final RequestInterceptor serializingInterceptor) {
+ return Cluster.build(serializingInterceptor).port(PORT);
+ }
+
public static Cluster open() {
return build().create();
}
(tinkerpop) 01/17: Update g:DateTime definition in GraphSONv4 documentation CTR.
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit dfaccc7b33ad51018a537a0a23599db7f17484f1
Author: Ken Hu <10...@users.noreply.github.com>
AuthorDate: Wed Oct 16 21:13:48 2024 -0700
Update g:DateTime definition in GraphSONv4 documentation CTR.
Slightly reworded it to cover all ISO 8601 formats as offset is just one
of the valid formats.
---
docs/src/dev/io/graphson.asciidoc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/src/dev/io/graphson.asciidoc b/docs/src/dev/io/graphson.asciidoc
index db708ec925..60daf6b9a0 100644
--- a/docs/src/dev/io/graphson.asciidoc
+++ b/docs/src/dev/io/graphson.asciidoc
@@ -163,7 +163,7 @@ JSON Object with two required keys: "type" and "fields" +
==== DateTime
-JSON String representing ISO-8601 format for datetime with offset.
+JSON String representing a datetime in the ISO-8601 format.
[source,json]
----
(tinkerpop) 17/17: Make gremlin-python feature tests use gremlin-lang
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 048ec39e3b850121e7ff1552a3bbe4986caacfb2
Author: Ken Hu <10...@users.noreply.github.com>
AuthorDate: Sun Nov 3 00:00:45 2024 -0700
Make gremlin-python feature tests use gremlin-lang
---
.../src/main/python/radish/feature_steps.py | 34 ++++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)
diff --git a/gremlin-python/src/main/python/radish/feature_steps.py b/gremlin-python/src/main/python/radish/feature_steps.py
index 83d069645b..27b50911de 100644
--- a/gremlin-python/src/main/python/radish/feature_steps.py
+++ b/gremlin-python/src/main/python/radish/feature_steps.py
@@ -39,7 +39,37 @@ ignores = [
"g.withoutStrategies(CountStrategy).V().count()", # serialization issues with Class in GraphSON
"g.withoutStrategies(LazyBarrierStrategy).V().as(\"label\").aggregate(local,\"x\").select(\"x\").select(\"label\")",
"g.withSack(xx1, Operator.assign).V().local(__.out(\"knows\").barrier(Barrier.normSack)).in(\"knows\").barrier().sack()", # issues with BigInteger/BigDecimal - why do we carry BigDecimal? just use python Decimal module?
- "g.withSack(2).V().sack(Operator.div).by(__.constant(xx1)).sack()" # issues with BigInteger/BigDecimal - why do we carry BigDecimal? just use python Decimal module?
+ "g.withSack(2).V().sack(Operator.div).by(__.constant(xx1)).sack()", # issues with BigInteger/BigDecimal - why do we carry BigDecimal? just use python Decimal module?
+ ## The following section has queries that aren't supported by gremlin-lang parameters
+ 'g.V().branch(l1).option("a", __.values("age")).option("b", __.values("lang")).option("b", __.values("name"))',
+ 'g.V().choose(pred1, __.out("knows"), __.in("created")).values("name")',
+ 'g.V().repeat(__.both()).until(pred1).groupCount().by("name")',
+ 'g.V().both().properties("name").order().by(c1).dedup().value()',
+ 'g.V().filter(pred1)',
+ 'g.V(vid1).filter(pred1)',
+ 'g.V(vid2).filter(pred1)',
+ 'g.V(vid1).out().filter(pred1)',
+ 'g.E().filter(pred1)',
+ 'g.V().out("created").has("name", __.map(l1).is(P.gt(3))).values("name")',
+ 'g.V(vid1).map(l1)',
+ 'g.V(vid1).outE().label().map(l1)',
+ 'g.V(vid1).out().map(l1).map(l2)',
+ 'g.withPath().V().as("a").out().map(l1)',
+ 'g.withPath().V().as("a").out().out().map(l1)',
+ 'g.V().values("name").order().by(c1).by(c2)',
+ 'g.V().order().by("name", c1).by("name", c2).values("name")',
+ 'g.V().hasLabel("person").order().by(l1, Order.desc).values("name")',
+ 'g.V(v1).hasLabel("person").map(l1).order(Scope.local).by(Column.values, Order.desc).by(Column.keys, Order.asc)',
+ 'g.V().valueMap().unfold().map(l1)',
+ 'g.E(e11)',
+ 'g.E(e7,e11)',
+ 'g.E(xx1)',
+ 'g.withSideEffect("a", xx1).V().both().values("name").aggregate(Scope.local,"a").cap("a")',
+ 'g.V().group().by(l1).by(__.constant(1))',
+ 'g.V(vid1).out().values("name").inject("daniel").as("a").map(l1).path()',
+ 'g.V().group("a").by(l1).by(__.constant(1)).cap("a")',
+ 'g.withSideEffect("a", xx1).V().both().values("name").store("a").cap("a")'
+ ## section end
]
@@ -56,7 +86,7 @@ def choose_graph(step, graph_name):
return
step.context.graph_name = graph_name
- step.context.g = traversal().with_(step.context.remote_conn[graph_name])
+ step.context.g = traversal().with_(step.context.remote_conn[graph_name]).with_('language', 'gremlin-lang')
@given("the graph initializer of")
(tinkerpop) 05/17: enable basic GraphSONV4 connectivity in python
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 2d73a08546cf28be5ee0ad514ede9fb14e680b2d
Author: Yang Xia <55...@users.noreply.github.com>
AuthorDate: Tue Oct 22 15:30:06 2024 -0700
enable basic GraphSONV4 connectivity in python
---
.../python/gremlin_python/driver/serializer.py | 54 +-
.../gremlin_python/structure/io/graphsonV4.py | 730 +++++++++++++++++++++
.../tests/driver/test_driver_remote_connection.py | 24 +-
3 files changed, 806 insertions(+), 2 deletions(-)
diff --git a/gremlin-python/src/main/python/gremlin_python/driver/serializer.py b/gremlin-python/src/main/python/gremlin_python/driver/serializer.py
index 94099f4c5d..9aa4a7c0cc 100644
--- a/gremlin-python/src/main/python/gremlin_python/driver/serializer.py
+++ b/gremlin-python/src/main/python/gremlin_python/driver/serializer.py
@@ -31,11 +31,63 @@ try:
except ImportError:
import json
-from gremlin_python.structure.io import graphbinaryV4
+from gremlin_python.structure.io import graphbinaryV4, graphsonV4
from gremlin_python.structure.io.util import Marker
__author__ = 'David M. Brown (davebshow@gmail.com), Lyndon Bauto (lyndonb@bitquilltech.com)'
+"""
+GraphSONV4
+"""
+
+class GraphSONSerializerV4(object):
+ """
+ Message serializer for GraphSON. Allow users to pass custom reader,
+ writer, and version kwargs for custom serialization. Otherwise,
+ use current GraphSON version as default.
+ """
+
+ # KEEP TRACK OF CURRENT DEFAULTS
+ DEFAULT_READER_CLASS = graphsonV4.GraphSONReader
+ DEFAULT_WRITER_CLASS = graphsonV4.GraphSONWriter
+ DEFAULT_VERSION = b"application/vnd.gremlin-v4.0+json"
+
+ def __init__(self, reader=None, writer=None, version=None):
+ if not version:
+ version = self.DEFAULT_VERSION
+ self._version = version
+ if not reader:
+ reader = self.DEFAULT_READER_CLASS()
+ self._graphson_reader = reader
+ if not writer:
+ writer = self.DEFAULT_WRITER_CLASS()
+ self._graphson_writer = writer
+
+ @property
+ def version(self):
+ """Read only property"""
+ return self._version
+
+ def serialize_message(self, request_message):
+ message = self.build_message(request_message.fields, request_message.gremlin)
+ return message
+
+ def build_message(self, fields, gremlin):
+ message = fields
+ message['gremlin'] = gremlin
+ return self.finalize_message(message)
+
+ def finalize_message(self, message):
+ message = json.dumps(message)
+ return message
+
+ def deserialize_message(self, message, is_first_chunk=False):
+ if is_first_chunk:
+ msg = json.loads(message if isinstance(message, str) else message.decode('utf-8'))
+ return self._graphson_reader.to_object(msg)
+ else:
+ # graphSON does not stream, the first chunk contains all info
+ return ""
"""
GraphBinaryV4
diff --git a/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py b/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
new file mode 100644
index 0000000000..b4aa99bf8e
--- /dev/null
+++ b/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
@@ -0,0 +1,730 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import calendar
+import datetime
+import json
+import uuid
+import math
+from collections import OrderedDict
+from decimal import *
+import logging
+from datetime import timedelta
+
+from aenum import Enum
+from isodate import parse_duration, duration_isoformat
+
+from gremlin_python import statics
+from gremlin_python.statics import FloatType, FunctionType, ShortType, IntType, LongType, TypeType, DictType, ListType, SetType, SingleByte, ByteBufferType, SingleChar
+from gremlin_python.process.traversal import Direction, P, TextP, Traversal, Traverser, TraversalStrategy, T
+from gremlin_python.structure.graph import Edge, Property, Vertex, VertexProperty, Path
+from gremlin_python.structure.io.util import HashableDict, SymbolUtil
+
+log = logging.getLogger(__name__)
+
+# When we fall back to a superclass's serializer, we iterate over this map.
+# We want that iteration order to be consistent, so we use an OrderedDict,
+# not a dict.
+_serializers = OrderedDict()
+_deserializers = {}
+
+
+class GraphSONTypeType(type):
+ def __new__(mcs, name, bases, dct):
+ cls = super(GraphSONTypeType, mcs).__new__(mcs, name, bases, dct)
+ if not name.startswith('_'):
+ if cls.python_type:
+ _serializers[cls.python_type] = cls
+ if cls.graphson_type:
+ _deserializers[cls.graphson_type] = cls
+ return cls
+
+
+class GraphSONUtil(object):
+ TYPE_KEY = "@type"
+ VALUE_KEY = "@value"
+
+ @classmethod
+ def typed_value(cls, type_name, value, prefix="g"):
+ out = {cls.TYPE_KEY: cls.format_type(prefix, type_name)}
+ if value is not None:
+ out[cls.VALUE_KEY] = value
+ return out
+
+ @classmethod
+ def format_type(cls, prefix, type_name):
+ return "%s:%s" % (prefix, type_name)
+
+
+# Read/Write classes split to follow precedence of the Java API
+class GraphSONWriter(object):
+ def __init__(self, serializer_map=None):
+ """
+ :param serializer_map: map from Python type to serializer instance implementing `dictify`
+ """
+ self.serializers = _serializers.copy()
+ if serializer_map:
+ self.serializers.update(serializer_map)
+
+ def write_object(self, objectData):
+ # to JSON
+ return json.dumps(self.to_dict(objectData), separators=(',', ':'))
+
+ def to_dict(self, obj):
+ """
+ Encodes python objects in GraphSON type-tagged dict values
+ """
+ try:
+ return self.serializers[type(obj)].dictify(obj, self)
+ except KeyError:
+ for key, serializer in self.serializers.items():
+ if isinstance(obj, key):
+ return serializer.dictify(obj, self)
+
+ if isinstance(obj, dict):
+ return dict((self.to_dict(k), self.to_dict(v)) for k, v in obj.items())
+ elif isinstance(obj, set):
+ return set([self.to_dict(o) for o in obj])
+ elif isinstance(obj, list):
+ return [self.to_dict(o) for o in obj]
+ else:
+ return obj
+
+
+class GraphSONReader(object):
+ def __init__(self, deserializer_map=None):
+ """
+ :param deserializer_map: map from GraphSON type tag to deserializer instance implementing `objectify`
+ """
+ self.deserializers = _deserializers.copy()
+ if deserializer_map:
+ self.deserializers.update(deserializer_map)
+
+ def read_object(self, json_data):
+ # from JSON
+ return self.to_object(json.loads(json_data))
+
+ def to_object(self, obj):
+ """
+ Unpacks GraphSON type-tagged dict values into objects mapped in self.deserializers
+ """
+ if isinstance(obj, dict):
+ try:
+ return self.deserializers[obj[GraphSONUtil.TYPE_KEY]].objectify(obj[GraphSONUtil.VALUE_KEY], self)
+ except KeyError:
+ pass
+ return dict((self.to_object(k), self.to_object(v)) for k, v in obj.items())
+ elif isinstance(obj, set):
+ return set([self.to_object(o) for o in obj])
+ elif isinstance(obj, list):
+ return [self.to_object(o) for o in obj]
+ else:
+ return obj
+
+
+class _GraphSONTypeIO(object, metaclass=GraphSONTypeType):
+ python_type = None
+ graphson_type = None
+
+ def dictify(self, obj, writer):
+ raise NotImplementedError()
+
+ def objectify(self, d, reader):
+ raise NotImplementedError()
+
+
+class _BytecodeSerializer(_GraphSONTypeIO):
+ @classmethod
+ def _dictify_instructions(cls, instructions, writer):
+ out = []
+ for instruction in instructions:
+ inst = [instruction[0]]
+ inst.extend(writer.to_dict(arg) for arg in instruction[1:])
+ out.append(inst)
+ return out
+
+ @classmethod
+ def dictify(cls, bytecode, writer):
+ if isinstance(bytecode, Traversal):
+ bytecode = bytecode.bytecode
+ out = {}
+ if bytecode.source_instructions:
+ out["source"] = cls._dictify_instructions(bytecode.source_instructions, writer)
+ if bytecode.step_instructions:
+ out["step"] = cls._dictify_instructions(bytecode.step_instructions, writer)
+ return GraphSONUtil.typed_value("Bytecode", out)
+
+
+class TraversalSerializer(_BytecodeSerializer):
+ python_type = Traversal
+
+
+class VertexSerializer(_GraphSONTypeIO):
+ python_type = Vertex
+ graphson_type = "g:Vertex"
+
+ @classmethod
+ def dictify(cls, vertex, writer):
+ return GraphSONUtil.typed_value("Vertex", {"id": writer.to_dict(vertex.id),
+ "label": writer.to_dict([vertex.label])})
+
+
+class EdgeSerializer(_GraphSONTypeIO):
+ python_type = Edge
+ graphson_type = "g:Edge"
+
+ @classmethod
+ def dictify(cls, edge, writer):
+ return GraphSONUtil.typed_value("Edge", {"id": writer.to_dict(edge.id),
+ "label": writer.to_dict([edge.label]),
+ "inV": writer.to_dict(edge.inV),
+ "outV": writer.to_dict(edge.outV)})
+
+ # "outV": writer.to_dict(edge.outV.id),
+ # "outVLabel": writer.to_dict([edge.outV.label]),
+ # "inV": writer.to_dict(edge.inV.id),
+ # "inVLabel": writer.to_dict([edge.inV.label])})
+
+
+class VertexPropertySerializer(_GraphSONTypeIO):
+ python_type = VertexProperty
+ graphson_type = "g:VertexProperty"
+
+ @classmethod
+ def dictify(cls, vertex_property, writer):
+ return GraphSONUtil.typed_value("VertexProperty", {"id": writer.to_dict(vertex_property.id),
+ "label": writer.to_dict([vertex_property.label]),
+ "value": writer.to_dict(vertex_property.value),
+ "vertex": writer.to_dict(vertex_property.vertex.id)})
+
+
+class PropertySerializer(_GraphSONTypeIO):
+ python_type = Property
+ graphson_type = "g:Property"
+
+ @classmethod
+ def dictify(cls, property, writer):
+ elementDict = writer.to_dict(property.element)
+ if elementDict is not None:
+ valueDict = elementDict["@value"]
+ if "outVLabel" in valueDict:
+ del valueDict["outVLabel"]
+ if "inVLabel" in valueDict:
+ del valueDict["inVLabel"]
+ if "properties" in valueDict:
+ del valueDict["properties"]
+ if "value" in valueDict:
+ del valueDict["value"]
+ return GraphSONUtil.typed_value("Property", {"key": writer.to_dict(property.key),
+ "value": writer.to_dict(property.value),
+ "element": elementDict})
+
+
+class TraversalStrategySerializer(_GraphSONTypeIO):
+ python_type = TraversalStrategy
+
+ @classmethod
+ def dictify(cls, strategy, writer):
+ configuration = {}
+ for key in strategy.configuration:
+ configuration[key] = writer.to_dict(strategy.configuration[key])
+ return GraphSONUtil.typed_value(strategy.strategy_name, configuration)
+
+
+class TraverserIO(_GraphSONTypeIO):
+ python_type = Traverser
+ graphson_type = "g:Traverser"
+
+ @classmethod
+ def dictify(cls, traverser, writer):
+ return GraphSONUtil.typed_value("Traverser", {"value": writer.to_dict(traverser.object),
+ "bulk": writer.to_dict(traverser.bulk)})
+
+ @classmethod
+ def objectify(cls, d, reader):
+ return Traverser(reader.to_object(d["value"]),
+ reader.to_object(d["bulk"]))
+
+
+class EnumSerializer(_GraphSONTypeIO):
+ python_type = Enum
+
+ @classmethod
+ def dictify(cls, enum, _):
+ return GraphSONUtil.typed_value(SymbolUtil.to_camel_case(type(enum).__name__),
+ SymbolUtil.to_camel_case(str(enum.name)))
+
+
+class PSerializer(_GraphSONTypeIO):
+ python_type = P
+
+ @classmethod
+ def dictify(cls, p, writer):
+ out = {"predicate": p.operator,
+ "value": [writer.to_dict(p.value), writer.to_dict(p.other)] if p.other is not None else
+ writer.to_dict(p.value)}
+ return GraphSONUtil.typed_value("P", out)
+
+
+class TextPSerializer(_GraphSONTypeIO):
+ python_type = TextP
+
+ @classmethod
+ def dictify(cls, p, writer):
+ out = {"predicate": p.operator,
+ "value": [writer.to_dict(p.value), writer.to_dict(p.other)] if p.other is not None else
+ writer.to_dict(p.value)}
+ return GraphSONUtil.typed_value("TextP", out)
+
+
+class TypeSerializer(_GraphSONTypeIO):
+ python_type = TypeType
+
+ @classmethod
+ def dictify(cls, typ, writer):
+ return writer.to_dict(typ())
+
+
+class UUIDIO(_GraphSONTypeIO):
+ python_type = uuid.UUID
+ graphson_type = "g:UUID"
+ graphson_base_type = "UUID"
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ return GraphSONUtil.typed_value(cls.graphson_base_type, str(obj))
+
+ @classmethod
+ def objectify(cls, d, reader):
+ return cls.python_type(d)
+
+
+class DateIO(_GraphSONTypeIO):
+ python_type = datetime.datetime
+ graphson_type = "g:Date"
+ graphson_base_type = "Date"
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ try:
+ timestamp_seconds = calendar.timegm(obj.utctimetuple())
+ pts = timestamp_seconds * 1e3 + getattr(obj, 'microsecond', 0) / 1e3
+ except AttributeError:
+ pts = calendar.timegm(obj.timetuple()) * 1e3
+
+ ts = int(round(pts))
+ return GraphSONUtil.typed_value(cls.graphson_base_type, ts)
+
+ @classmethod
+ def objectify(cls, ts, reader):
+ # Python timestamp expects seconds
+ return datetime.datetime.utcfromtimestamp(ts / 1000.0)
+
+
+class _NumberIO(_GraphSONTypeIO):
+ @classmethod
+ def dictify(cls, n, writer):
+ if isinstance(n, bool): # because isinstance(False, int) and isinstance(True, int)
+ return n
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n)
+
+ @classmethod
+ def objectify(cls, v, _):
+ return cls.python_type(v)
+
+
+class ListIO(_GraphSONTypeIO):
+ python_type = ListType
+ graphson_type = "g:List"
+
+ @classmethod
+ def dictify(cls, l, writer):
+ new_list = []
+ for obj in l:
+ new_list.append(writer.to_dict(obj))
+ return GraphSONUtil.typed_value("List", new_list)
+
+ @classmethod
+ def objectify(cls, l, reader):
+ new_list = []
+ for obj in l:
+ new_list.append(reader.to_object(obj))
+ return new_list
+
+
+class SetIO(_GraphSONTypeIO):
+ python_type = SetType
+ graphson_type = "g:Set"
+
+ @classmethod
+ def dictify(cls, s, writer):
+ new_list = []
+ for obj in s:
+ new_list.append(writer.to_dict(obj))
+ return GraphSONUtil.typed_value("Set", new_list)
+
+ @classmethod
+ def objectify(cls, s, reader):
+ """
+ By default, returns a python set
+
+ In case Java returns numeric values of different types which
+ python don't recognize, coerce and return a list.
+ See comments of TINKERPOP-1844 for more details
+ """
+ new_list = [reader.to_object(obj) for obj in s]
+ new_set = set(new_list)
+ if len(new_list) != len(new_set):
+ log.warning("Coercing g:Set to list due to java numeric values. "
+ "See TINKERPOP-1844 for more details.")
+ return new_list
+
+ return new_set
+
+
+class MapType(_GraphSONTypeIO):
+ python_type = DictType
+ graphson_type = "g:Map"
+
+ @classmethod
+ def dictify(cls, d, writer):
+ l = []
+ for key in d:
+ l.append(writer.to_dict(key))
+ l.append(writer.to_dict(d[key]))
+ return GraphSONUtil.typed_value("Map", l)
+
+ @classmethod
+ def objectify(cls, l, reader):
+ new_dict = {}
+ if len(l) > 0:
+ x = 0
+ while x < len(l):
+ new_dict[HashableDict.of(reader.to_object(l[x]))] = reader.to_object(l[x + 1])
+ x = x + 2
+ return new_dict
+
+
+class BulkSetIO(_GraphSONTypeIO):
+ graphson_type = "g:BulkSet"
+
+ @classmethod
+ def objectify(cls, l, reader):
+ new_list = []
+
+ # this approach basically mimics what currently existed in 3.3.4 and prior versions where BulkSet is
+ # basically just coerced to list. the limitation here is that if the value of a bulk exceeds the size of
+ # a list (into the long space) then stuff won't work nice.
+ if len(l) > 0:
+ x = 0
+ while x < len(l):
+ obj = reader.to_object(l[x])
+ bulk = reader.to_object(l[x + 1])
+ for y in range(bulk):
+ new_list.append(obj)
+ x = x + 2
+ return new_list
+
+
+class FloatIO(_NumberIO):
+ python_type = FloatType
+ graphson_type = "g:Float"
+ graphson_base_type = "Float"
+
+ @classmethod
+ def dictify(cls, n, writer):
+ if isinstance(n, bool): # because isinstance(False, int) and isinstance(True, int)
+ return n
+ elif math.isnan(n):
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "NaN")
+ elif math.isinf(n) and n > 0:
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "Infinity")
+ elif math.isinf(n) and n < 0:
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "-Infinity")
+ else:
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n)
+
+ @classmethod
+ def objectify(cls, v, _):
+ if isinstance(v, str):
+ if v == 'NaN':
+ return float('nan')
+ elif v == "Infinity":
+ return float('inf')
+ elif v == "-Infinity":
+ return float('-inf')
+
+ return cls.python_type(v)
+
+
+class BigDecimalIO(_NumberIO):
+ python_type = Decimal
+ graphson_type = "gx:BigDecimal"
+ graphson_base_type = "BigDecimal"
+
+ @classmethod
+ def dictify(cls, n, writer):
+ if isinstance(n, bool): # because isinstance(False, int) and isinstance(True, int)
+ return n
+ elif math.isnan(n):
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "NaN", "gx")
+ elif math.isinf(n) and n > 0:
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "Infinity", "gx")
+ elif math.isinf(n) and n < 0:
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "-Infinity", "gx")
+ else:
+ return GraphSONUtil.typed_value(cls.graphson_base_type, str(n), "gx")
+
+ @classmethod
+ def objectify(cls, v, _):
+ if isinstance(v, str):
+ if v == 'NaN':
+ return Decimal('nan')
+ elif v == "Infinity":
+ return Decimal('inf')
+ elif v == "-Infinity":
+ return Decimal('-inf')
+
+ return Decimal(v)
+
+
+class DoubleIO(FloatIO):
+ graphson_type = "g:Double"
+ graphson_base_type = "Double"
+
+
+class Int64IO(_NumberIO):
+ python_type = LongType
+ graphson_type = "g:Int64"
+ graphson_base_type = "Int64"
+
+ @classmethod
+ def dictify(cls, n, writer):
+ # if we exceed Java long range then we need a BigInteger
+ if isinstance(n, bool):
+ return n
+ elif n < -9223372036854775808 or n > 9223372036854775807:
+ return GraphSONUtil.typed_value("BigInteger", str(n), "gx")
+ else:
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n)
+
+
+class BigIntegerIO(Int64IO):
+ graphson_type = "gx:BigInteger"
+
+
+class Int32IO(Int64IO):
+ python_type = IntType
+ graphson_type = "g:Int32"
+ graphson_base_type = "Int32"
+
+ @classmethod
+ def dictify(cls, n, writer):
+ # if we exceed Java int range then we need a long
+ if isinstance(n, bool):
+ return n
+ elif n < -9223372036854775808 or n > 9223372036854775807:
+ return GraphSONUtil.typed_value("BigInteger", str(n), "gx")
+ elif n < -2147483648 or n > 2147483647:
+ return GraphSONUtil.typed_value("Int64", n)
+ else:
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n)
+
+
+class Int16IO(Int64IO):
+ python_type = ShortType
+ graphson_type = "gx:Int16"
+ graphson_base_type = "Int16"
+
+ @classmethod
+ def dictify(cls, n, writer):
+ # if we exceed Java int range then we need a long
+ if isinstance(n, bool):
+ return n
+ elif n < -9223372036854775808 or n > 9223372036854775807:
+ return GraphSONUtil.typed_value("BigInteger", str(n), "gx")
+ elif n < -2147483648 or n > 2147483647:
+ return GraphSONUtil.typed_value("Int64", n)
+ elif n < -32768 or n > 32767:
+ return GraphSONUtil.typed_value("Int32", n)
+ else:
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n, "gx")
+
+ @classmethod
+ def objectify(cls, v, _):
+ return int.__new__(ShortType, v)
+
+
+class ByteIO(_NumberIO):
+ python_type = SingleByte
+ graphson_type = "gx:Byte"
+ graphson_base_type = "Byte"
+
+ @classmethod
+ def dictify(cls, n, writer):
+ if isinstance(n, bool): # because isinstance(False, int) and isinstance(True, int)
+ return n
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n, "gx")
+
+ @classmethod
+ def objectify(cls, v, _):
+ return int.__new__(SingleByte, v)
+
+
+class ByteBufferIO(_GraphSONTypeIO):
+ python_type = ByteBufferType
+ graphson_type = "gx:ByteBuffer"
+ graphson_base_type = "ByteBuffer"
+
+ @classmethod
+ def dictify(cls, n, writer):
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "".join(chr(x) for x in n), "gx")
+
+ @classmethod
+ def objectify(cls, v, _):
+ return cls.python_type(v, "utf8")
+
+
+class CharIO(_GraphSONTypeIO):
+ python_type = SingleChar
+ graphson_type = "gx:Char"
+ graphson_base_type = "Char"
+
+ @classmethod
+ def dictify(cls, n, writer):
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n, "gx")
+
+ @classmethod
+ def objectify(cls, v, _):
+ return str.__new__(SingleChar, v)
+
+
+class DurationIO(_GraphSONTypeIO):
+ python_type = timedelta
+ graphson_type = "gx:Duration"
+ graphson_base_type = "Duration"
+
+ @classmethod
+ def dictify(cls, n, writer):
+ return GraphSONUtil.typed_value(cls.graphson_base_type, duration_isoformat(n), "gx")
+
+ @classmethod
+ def objectify(cls, v, _):
+ return parse_duration(v)
+
+
+class VertexDeserializer(_GraphSONTypeIO):
+ graphson_type = "g:Vertex"
+
+ @classmethod
+ def objectify(cls, d, reader):
+ properties = None
+ if "properties" in d:
+ properties = reader.to_object(d["properties"])
+ if properties is not None:
+ properties = [item for sublist in properties.values() for item in sublist]
+ return Vertex(reader.to_object(d["id"]), d.get("label", "vertex")[0], properties)
+
+
+class EdgeDeserializer(_GraphSONTypeIO):
+ graphson_type = "g:Edge"
+
+ @classmethod
+ def objectify(cls, d, reader):
+ properties = None
+ if "properties" in d:
+ properties = reader.to_object(d["properties"])
+ if properties is not None:
+ properties = list(properties.values())
+ return Edge(reader.to_object(d["id"]),
+ Vertex(reader.to_object(d["outV"]), d.get("outVLabel", "vertex")[0]),
+ d.get("label", "edge")[0],
+ Vertex(reader.to_object(d["inV"]), d.get("inVLabel", "vertex")[0]),
+ properties)
+
+
+class VertexPropertyDeserializer(_GraphSONTypeIO):
+ graphson_type = "g:VertexProperty"
+
+ @classmethod
+ def objectify(cls, d, reader):
+ properties = None
+ if "properties" in d:
+ properties = reader.to_object(d["properties"])
+ if properties is not None:
+ properties = list(map(lambda x: Property(x[0], x[1], None), properties.items()))
+ vertex = Vertex(reader.to_object(d.get("vertex"))) if "vertex" in d else None
+ return VertexProperty(reader.to_object(d["id"]),
+ d["label"][0],
+ reader.to_object(d["value"]),
+ vertex,
+ properties)
+
+
+class PropertyDeserializer(_GraphSONTypeIO):
+ graphson_type = "g:Property"
+
+ @classmethod
+ def objectify(cls, d, reader):
+ element = reader.to_object(d["element"]) if "element" in d else None
+ return Property(d["key"], reader.to_object(d["value"]), element)
+
+
+class PathDeserializer(_GraphSONTypeIO):
+ graphson_type = "g:Path"
+
+ @classmethod
+ def objectify(cls, d, reader):
+ return Path(reader.to_object(d["labels"]), reader.to_object(d["objects"]))
+
+
+class TDeserializer(_GraphSONTypeIO):
+ graphson_type = "g:T"
+
+ @classmethod
+ def objectify(cls, d, reader):
+ return T[d]
+
+
+class DirectionIO(_GraphSONTypeIO):
+ graphson_type = "g:Direction"
+ graphson_base_type = "Direction"
+ python_type = Direction
+
+ @classmethod
+ def dictify(cls, d, writer):
+ return GraphSONUtil.typed_value(cls.graphson_base_type, d.name, "g")
+
+ @classmethod
+ def objectify(cls, d, reader):
+ return Direction[d]
+
+
+class TraversalMetricsDeserializer(_GraphSONTypeIO):
+ graphson_type = "g:TraversalMetrics"
+
+ @classmethod
+ def objectify(cls, d, reader):
+ return reader.to_object(d)
+
+
+class MetricsDeserializer(_GraphSONTypeIO):
+ graphson_type = "g:Metrics"
+
+ @classmethod
+ def objectify(cls, d, reader):
+ return reader.to_object(d)
diff --git a/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py b/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
index 46c977222b..86333fa6ab 100644
--- a/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
+++ b/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
@@ -20,6 +20,7 @@ import os
from datetime import datetime
from gremlin_python import statics
+from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
from gremlin_python.statics import long
from gremlin_python.process.traversal import TraversalStrategy, P, Order, T, DT, Parameter, Cardinality
from gremlin_python.process.graph_traversal import __
@@ -28,13 +29,34 @@ from gremlin_python.structure.graph import Vertex
from gremlin_python.process.strategies import SubgraphStrategy, SeedStrategy
from gremlin_python.structure.io.util import HashableDict
from gremlin_python.driver.protocol import GremlinServerError
+from gremlin_python.driver import serializer
-gremlin_server_url = os.environ.get('GREMLIN_SERVER_URL_HTTP', 'http://localhost:{}/')
+gremlin_server_url = os.environ.get('GREMLIN_SERVER_URL', 'http://localhost:{}/')
test_no_auth_url = gremlin_server_url.format(45940)
class TestDriverRemoteConnection(object):
+ # this is a temporary test for basic graphSONV4 connectivity, once all types are implemented, enable graphSON testing
+ # in conftest.py and remove this
+ def test_graphSONV4_temp(self):
+ remote_conn = DriverRemoteConnection(test_no_auth_url, 'gmodern',
+ message_serializer=serializer.GraphSONSerializerV4())
+ g = traversal().with_(remote_conn)
+ assert long(6) == g.V().count().to_list()[0]
+ # #
+ assert 10 == g.V().repeat(__.both()).times(5)[0:10].count().next()
+ assert 1 == g.V().repeat(__.both()).times(5)[0:1].count().next()
+ assert 0 == g.V().repeat(__.both()).times(5)[0:0].count().next()
+ assert 4 == g.V()[2:].count().next()
+ assert 2 == g.V()[:2].count().next()
+ # #
+ results = g.with_side_effect('a', ['josh', 'peter']).V(1).out('created').in_('created').values('name').where(
+ P.within('a')).to_list()
+ assert 2 == len(results)
+ assert 'josh' in results
+ assert 'peter' in results
+
def test_bulked_request_option(self, remote_connection):
g = traversal().with_(remote_connection)
result = g.inject(1,2,3,2,1).to_list()
(tinkerpop) 06/17: Added request interceptor to gremlin-python. (#2849)
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit f7b4f56768556d02c7eb5ee8a6762ca5e0b742e4
Author: kenhuuu <10...@users.noreply.github.com>
AuthorDate: Thu Oct 24 17:52:58 2024 -0700
Added request interceptor to gremlin-python. (#2849)
The message_serializer is split into the request_serializer and the
response_serializer to allow for setting the request_serializer to None
which prevents payload serialization. This makes it easier for the user
to modify the payload. Auth is not combined with interceptor as it
requires refactoring to provide the URL in protocol and simplifies usage
compared to the Java driver albeit at the cost of flexibility.
---
docs/src/reference/gremlin-variants.asciidoc | 4 +-
docs/src/upgrade/release-4.x.x.asciidoc | 15 ++-
.../main/python/gremlin_python/driver/client.py | 19 +--
.../driver/driver_remote_connection.py | 17 +--
.../main/python/gremlin_python/driver/protocol.py | 33 ++++--
gremlin-python/src/main/python/radish/terrain.py | 4 +-
gremlin-python/src/main/python/tests/conftest.py | 41 ++++++-
.../src/main/python/tests/driver/test_client.py | 11 ++
.../tests/driver/test_driver_remote_connection.py | 8 +-
.../src/main/python/tests/driver/test_protocol.py | 131 +++++++++++++++++++++
.../main/python/tests/process/test_traversal.py | 3 +-
.../tests/structure/io/test_functionalityio.py | 8 +-
12 files changed, 260 insertions(+), 34 deletions(-)
diff --git a/docs/src/reference/gremlin-variants.asciidoc b/docs/src/reference/gremlin-variants.asciidoc
index 9868582b4c..918becc487 100644
--- a/docs/src/reference/gremlin-variants.asciidoc
+++ b/docs/src/reference/gremlin-variants.asciidoc
@@ -2249,7 +2249,9 @@ can be passed to the `Client` or `DriverRemoteConnection` instance as keyword ar
|Key |Description |Default
|headers |Additional headers that will be added to each request message. |`None`
|max_workers |Maximum number of worker threads. |Number of CPUs * 5
-|message_serializer |The message serializer implementation.|`gremlin_python.driver.serializer.GraphBinarySerializersV1`
+|request_serializer |The request serializer implementation.|`gremlin_python.driver.serializer.GraphBinarySerializersV4`
+|response_serializer |The response serializer implementation.|`gremlin_python.driver.serializer.GraphBinarySerializersV4`
+|interceptors |The request interceptors to run after request serialization.|`None`
|password |The password to submit on requests that require authentication. |""
|pool_size |The number of connections used by the pool. |4
|protocol_factory |A callable that returns an instance of `AbstractBaseProtocol`. |`gremlin_python.driver.protocol.GremlinServerWSProtocol`
diff --git a/docs/src/upgrade/release-4.x.x.asciidoc b/docs/src/upgrade/release-4.x.x.asciidoc
index e1f90c74b1..d24a1a6e3b 100644
--- a/docs/src/upgrade/release-4.x.x.asciidoc
+++ b/docs/src/upgrade/release-4.x.x.asciidoc
@@ -85,7 +85,7 @@ The properties file in the above example can either point to a remote configurat
See: link:https://issues.apache.org/jira/browse/TINKERPOP-3017[TINKERPOP-3017]
-==== Changes to RequestInterceptor
+==== Changes to Java RequestInterceptor
Because the underlying transport has been changed from WebSockets to HTTP, the usage of the `RequestInterceptor` has
changed as well. The `RequestInterceptor` will now be run per request and will allow you to completely modify the HTTP
@@ -106,6 +106,19 @@ that is sent to the endpoint. There is a default interceptor added to every `Clu
interceptor is responsible for serializing the request body is which what the server normally expects. This is intended
to be an advanced customization technique that should only be used when needed.
+==== Addition of Python interceptor
+
+HTTP interceptors have been added to `gremlin-python` to enable capability similar to that of Java GLV. These
+interceptors can be passed into either a `DriverRemoteConnection` or a `Client` using the interceptors parameter. An
+interceptor is a `Callable` that accepts one argument which is the HTTP request (dictionary containing header, payload
+and auth) or a list/tuple of these functions. The interceptors will run after the request serializer has run but before
+any auth functions run so the HTTP request may still get modified after your interceptors are run. In situations where
+you don't want the payload to be serialized, the `message_serializer` has been split into a `request_serializer` and a
+`response_serializer`. Simply set the `request_serializer` to `None` and this will prevent the `RequestMessage` from
+being serialized. Again, this is expected to be an advanced feature so some knowledge of implementation details will be
+required to make this work. For example, you'll need to know what payload formats are accepted by `aiohttp` for the
+request to be sent.
+
==== Changes to deserialization for gremlin-javascript
Starting from this version, `gremlin-javascript` will deserialize `Set` data into a ECMAScript 2015 Set. Previously,
diff --git a/gremlin-python/src/main/python/gremlin_python/driver/client.py b/gremlin-python/src/main/python/gremlin_python/driver/client.py
index 73fd96588f..8f948b18a0 100644
--- a/gremlin-python/src/main/python/gremlin_python/driver/client.py
+++ b/gremlin-python/src/main/python/gremlin_python/driver/client.py
@@ -41,8 +41,10 @@ class Client:
def __init__(self, url, traversal_source, protocol_factory=None,
transport_factory=None, pool_size=None, max_workers=None,
- message_serializer=None, auth=None, headers=None,
- enable_user_agent_on_connect=True, enable_bulked_result=False, **transport_kwargs):
+ request_serializer=serializer.GraphBinarySerializersV4(),
+ response_serializer=None, interceptors=None, auth=None,
+ headers=None, enable_user_agent_on_connect=True,
+ enable_bulked_result=False, **transport_kwargs):
log.info("Creating Client with url '%s'", url)
self._closed = False
@@ -53,11 +55,11 @@ class Client:
self._traversal_source = traversal_source
if "max_content_length" not in transport_kwargs:
transport_kwargs["max_content_length"] = 10 * 1024 * 1024
- if message_serializer is None:
- message_serializer = serializer.GraphBinarySerializersV4()
+ if response_serializer is None:
+ response_serializer = serializer.GraphBinarySerializersV4()
- self._message_serializer = message_serializer
self._auth = auth
+ self._response_serializer = response_serializer
if transport_factory is None:
try:
@@ -75,8 +77,8 @@ class Client:
if protocol_factory is None:
def protocol_factory():
return protocol.GremlinServerHTTPProtocol(
- self._message_serializer,
- auth=self._auth)
+ request_serializer, response_serializer, auth=self._auth,
+ interceptors=interceptors)
self._protocol_factory = protocol_factory
if pool_size is None:
@@ -95,6 +97,9 @@ class Client:
@property
def available_pool_size(self):
return self._pool.qsize()
+
+ def response_serializer(self):
+ return self._response_serializer
@property
def executor(self):
diff --git a/gremlin-python/src/main/python/gremlin_python/driver/driver_remote_connection.py b/gremlin-python/src/main/python/gremlin_python/driver/driver_remote_connection.py
index dc8b95eb7a..b32d8f7ad0 100644
--- a/gremlin-python/src/main/python/gremlin_python/driver/driver_remote_connection.py
+++ b/gremlin-python/src/main/python/gremlin_python/driver/driver_remote_connection.py
@@ -33,9 +33,10 @@ class DriverRemoteConnection(RemoteConnection):
def __init__(self, url, traversal_source="g", protocol_factory=None,
transport_factory=None, pool_size=None, max_workers=None,
- auth=None,
- message_serializer=None, headers=None,
- enable_user_agent_on_connect=True, enable_bulked_result=False, **transport_kwargs):
+ request_serializer=serializer.GraphBinarySerializersV4(),
+ response_serializer=None, interceptors=None, auth=None,
+ headers=None, enable_user_agent_on_connect=True,
+ enable_bulked_result=False, **transport_kwargs):
log.info("Creating DriverRemoteConnection with url '%s'", str(url))
self.__url = url
self.__traversal_source = traversal_source
@@ -44,21 +45,21 @@ class DriverRemoteConnection(RemoteConnection):
self.__pool_size = pool_size
self.__max_workers = max_workers
self.__auth = auth
- self.__message_serializer = message_serializer
self.__headers = headers
self.__enable_user_agent_on_connect = enable_user_agent_on_connect
self.__enable_bulked_result = enable_bulked_result
self.__transport_kwargs = transport_kwargs
- if message_serializer is None:
- message_serializer = serializer.GraphBinarySerializersV4()
+ if response_serializer is None:
+ response_serializer = serializer.GraphBinarySerializersV4()
self._client = client.Client(url, traversal_source,
protocol_factory=protocol_factory,
transport_factory=transport_factory,
pool_size=pool_size,
max_workers=max_workers,
- message_serializer=message_serializer,
- auth=auth,
+ request_serializer=request_serializer,
+ response_serializer=response_serializer,
+ interceptors=interceptors, auth=auth,
headers=headers,
enable_user_agent_on_connect=enable_user_agent_on_connect,
enable_bulked_result=enable_bulked_result,
diff --git a/gremlin-python/src/main/python/gremlin_python/driver/protocol.py b/gremlin-python/src/main/python/gremlin_python/driver/protocol.py
index b286c9dec9..930d2de328 100644
--- a/gremlin-python/src/main/python/gremlin_python/driver/protocol.py
+++ b/gremlin-python/src/main/python/gremlin_python/driver/protocol.py
@@ -53,9 +53,19 @@ class AbstractBaseProtocol(metaclass=abc.ABCMeta):
class GremlinServerHTTPProtocol(AbstractBaseProtocol):
- def __init__(self, message_serializer, auth=None):
+ def __init__(self, request_serializer, response_serializer,
+ interceptors=None, auth=None):
+ if callable(interceptors):
+ interceptors = [interceptors]
+ elif not (isinstance(interceptors, tuple)
+ or isinstance(interceptors, list)
+ or interceptors is None):
+ raise TypeError("interceptors must be a callable, tuple, list or None")
+
self._auth = auth
- self._message_serializer = message_serializer
+ self._interceptors = interceptors
+ self._request_serializer = request_serializer
+ self._response_serializer = response_serializer
self._response_msg = {'status': {'code': 0,
'message': '',
'exception': ''},
@@ -67,15 +77,22 @@ class GremlinServerHTTPProtocol(AbstractBaseProtocol):
super(GremlinServerHTTPProtocol, self).connection_made(transport)
def write(self, request_message):
- content_type = str(self._message_serializer.version, encoding='utf-8')
-
+ accept = str(self._response_serializer.version, encoding='utf-8')
message = {
- 'headers': {'content-type': content_type,
- 'accept': content_type},
- 'payload': self._message_serializer.serialize_message(request_message),
+ 'headers': {'accept': accept},
+ 'payload': self._request_serializer.serialize_message(request_message)
+ if self._request_serializer is not None else request_message,
'auth': self._auth
}
+ # The user may not want the payload to be serialized if they are using an interceptor.
+ if self._request_serializer is not None:
+ content_type = str(self._request_serializer.version, encoding='utf-8')
+ message['headers']['content-type'] = content_type
+
+ for interceptor in self._interceptors or []:
+ message = interceptor(message)
+
self._transport.write(message)
# data is received in chunks
@@ -110,7 +127,7 @@ class GremlinServerHTTPProtocol(AbstractBaseProtocol):
self._is_first_chunk = False
def _decode_chunk(self, message, data_buffer, is_first_chunk):
- chunk_msg = self._message_serializer.deserialize_message(data_buffer, is_first_chunk)
+ chunk_msg = self._response_serializer.deserialize_message(data_buffer, is_first_chunk)
if 'result' in chunk_msg:
msg_data = message['result']['data']
diff --git a/gremlin-python/src/main/python/radish/terrain.py b/gremlin-python/src/main/python/radish/terrain.py
index cc000ef881..c54334ab0c 100644
--- a/gremlin-python/src/main/python/radish/terrain.py
+++ b/gremlin-python/src/main/python/radish/terrain.py
@@ -101,4 +101,6 @@ def __create_remote(server_graph_name):
bulked = world.config.user_data["bulked"] == "true" if "bulked" in world.config.user_data else False
- return DriverRemoteConnection(test_no_auth_url, server_graph_name, message_serializer=s, enable_bulked_result=bulked)
+ return DriverRemoteConnection(test_no_auth_url, server_graph_name,
+ request_serializer=s, response_serializer=s,
+ enable_bulked_result=bulked)
diff --git a/gremlin-python/src/main/python/tests/conftest.py b/gremlin-python/src/main/python/tests/conftest.py
index 059600f32c..0aafc43d1b 100644
--- a/gremlin-python/src/main/python/tests/conftest.py
+++ b/gremlin-python/src/main/python/tests/conftest.py
@@ -18,6 +18,7 @@
#
import concurrent.futures
+from json import dumps
import os
import ssl
import pytest
@@ -49,7 +50,7 @@ Tests below are for the HTTP server with GraphBinaryV4
@pytest.fixture
def connection(request):
protocol = GremlinServerHTTPProtocol(
- GraphBinarySerializersV4(),
+ GraphBinarySerializersV4(), GraphBinarySerializersV4(),
auth=basic('stephen', 'password'))
executor = concurrent.futures.ThreadPoolExecutor(5)
pool = queue.Queue()
@@ -181,3 +182,41 @@ def invalid_alias_remote_connection(request):
request.addfinalizer(fin)
return remote_conn
+
+
+@pytest.fixture()
+def remote_connection_with_interceptor(request):
+ try:
+ remote_conn = DriverRemoteConnection(anonymous_url, 'gmodern',
+ request_serializer=None,
+ interceptors=json_interceptor)
+ except OSError:
+ pytest.skip('Gremlin Server is not running')
+ else:
+ def fin():
+ remote_conn.close()
+
+ request.addfinalizer(fin)
+ return remote_conn
+
+
+@pytest.fixture()
+def client_with_interceptor(request):
+ try:
+ client = Client(anonymous_url, 'gmodern', request_serializer=None,
+ response_serializer=GraphBinarySerializersV4(),
+ interceptors=json_interceptor)
+ except OSError:
+ pytest.skip('Gremlin Server is not running')
+ else:
+ def fin():
+ client.close()
+
+ request.addfinalizer(fin)
+ return client
+
+
+def json_interceptor(request):
+ request['headers']['content-type'] = "application/json"
+ request['payload'] = dumps({"gremlin": "g.inject(2)", "g": "g"})
+ return request
diff --git a/gremlin-python/src/main/python/tests/driver/test_client.py b/gremlin-python/src/main/python/tests/driver/test_client.py
index be8dd24259..4b6a9ad11b 100644
--- a/gremlin-python/src/main/python/tests/driver/test_client.py
+++ b/gremlin-python/src/main/python/tests/driver/test_client.py
@@ -26,6 +26,7 @@ from gremlin_python.driver.client import Client
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
from gremlin_python.driver.protocol import GremlinServerError
from gremlin_python.driver.request import RequestMessage
+from gremlin_python.driver.serializer import GraphBinarySerializersV4
from gremlin_python.process.graph_traversal import __, GraphTraversalSource
from gremlin_python.process.traversal import TraversalStrategies, Parameter
from gremlin_python.process.strategies import OptionsStrategy
@@ -564,3 +565,13 @@ def test_client_custom_invalid_request_id_graphbinary_bytecode(client):
def test_client_custom_valid_request_id_bytecode(client):
query = GraphTraversalSource(Graph(), TraversalStrategies()).V().bytecode
assert len(client.submit(query).all().result()) == 6
+
+def test_response_serializer_never_None():
+ client = Client('url', 'g', response_serializer=None)
+ resp_ser = client.response_serializer()
+ assert resp_ser is not None
+
+
+def test_serializer_and_interceptor_forwarded(client_with_interceptor):
+ result = client_with_interceptor.submit("g.inject(1)").next()
+ assert [2] == result # interceptor changes request to g.inject(2)
diff --git a/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py b/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
index 86333fa6ab..1382a0adb6 100644
--- a/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
+++ b/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
@@ -41,7 +41,8 @@ class TestDriverRemoteConnection(object):
# in conftest.py and remove this
def test_graphSONV4_temp(self):
remote_conn = DriverRemoteConnection(test_no_auth_url, 'gmodern',
- message_serializer=serializer.GraphSONSerializerV4())
+ request_serializer=serializer.GraphSONSerializerV4(),
+ response_serializer=serializer.GraphSONSerializerV4())
g = traversal().with_(remote_conn)
assert long(6) == g.V().count().to_list()[0]
# #
@@ -264,3 +265,8 @@ class TestDriverRemoteConnection(object):
g = traversal().with_(remote_connection_authenticated)
assert long(6) == g.V().count().to_list()[0]
+
+ def test_forwards_interceptor_serializers(self, remote_connection_with_interceptor):
+ g = traversal().with_(remote_connection_with_interceptor)
+ result = g.inject(1).next()
+ assert 2 == result # interceptor changes request to g.inject(2)
diff --git a/gremlin-python/src/main/python/tests/driver/test_protocol.py b/gremlin-python/src/main/python/tests/driver/test_protocol.py
new file mode 100644
index 0000000000..3efd12975f
--- /dev/null
+++ b/gremlin-python/src/main/python/tests/driver/test_protocol.py
@@ -0,0 +1,131 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+from gremlin_python.driver.protocol import GremlinServerHTTPProtocol
+from gremlin_python.driver.serializer import GraphBinarySerializersV4
+from gremlin_python.driver.transport import AbstractBaseTransport
+from gremlin_python.driver.request import RequestMessage
+
+class MockHTTPTransport(AbstractBaseTransport):
+ def connect(self, url, headers=None):
+ pass
+
+ def write(self, message):
+ self._message = message
+
+ def get_write(self):
+ return self._message
+
+ def read(self):
+ pass
+
+ def close(self):
+ pass
+
+ def closed(self):
+ pass
+
+def test_none_request_serializer_valid():
+ protocol = GremlinServerHTTPProtocol(None, GraphBinarySerializersV4(), interceptors=None)
+ mock_transport = MockHTTPTransport()
+ protocol.connection_made(mock_transport)
+
+ message = RequestMessage(fields={}, gremlin="g.V()")
+ protocol.write(message)
+ written = mock_transport.get_write()
+
+ assert written["payload"] == message
+ assert 'content-type' not in written["headers"]
+
+def test_graphbinary_request_serializer_serializes_payload():
+ gb_ser = GraphBinarySerializersV4()
+ protocol = GremlinServerHTTPProtocol(gb_ser, gb_ser)
+ mock_transport = MockHTTPTransport()
+ protocol.connection_made(mock_transport)
+
+ message = RequestMessage(fields={}, gremlin="g.V()")
+ protocol.write(message)
+ written = mock_transport.get_write()
+
+ assert written["payload"] == gb_ser.serialize_message(message)
+ assert written["headers"]['content-type'] == str(gb_ser.version, encoding='utf-8')
+
+def test_interceptor_allows_tuple_and_list():
+ try:
+ tuple = GremlinServerHTTPProtocol(None, None, interceptors=(lambda req: req))
+ list = GremlinServerHTTPProtocol(None, None, interceptors=[lambda req: req])
+ assert True
+ except:
+ assert False
+
+def test_interceptor_doesnt_allow_any_type():
+ try:
+ protocol = GremlinServerHTTPProtocol(None, None, interceptors=1)
+ assert False
+ except TypeError:
+ assert True
+
+def test_single_interceptor_runs():
+ changed_req = RequestMessage(fields={}, gremlin="changed")
+ def interceptor(request):
+ request['payload'] = changed_req
+ return request
+
+ protocol = GremlinServerHTTPProtocol(None, GraphBinarySerializersV4(),
+ interceptors=interceptor)
+ mock_transport = MockHTTPTransport()
+ protocol.connection_made(mock_transport)
+
+ message = RequestMessage(fields={}, gremlin="g.V()")
+ protocol.write(message)
+ written = mock_transport.get_write()
+
+ assert written['payload'] == changed_req
+
+def test_interceptor_works_with_request_serializer():
+ gb_ser = GraphBinarySerializersV4()
+ message = RequestMessage(fields={}, gremlin="g.E()")
+
+ def assert_inteceptor(request):
+ assert request['payload'] == gb_ser.serialize_message(message)
+ request['payload'] = "changed"
+ return request
+
+ protocol = GremlinServerHTTPProtocol(gb_ser, gb_ser, interceptors=assert_inteceptor)
+ mock_transport = MockHTTPTransport()
+ protocol.connection_made(mock_transport)
+
+ protocol.write(message)
+ written = mock_transport.get_write()
+
+ assert written["payload"] == "changed"
+
+def test_interceptors_run_sequentially():
+ def three(request): request['payload'].gremlin.append(3); return request
+ def two(request): request['payload'].gremlin.append(2); return request
+ def one(request): request['payload'].gremlin.append(1); return request
+ protocol = GremlinServerHTTPProtocol(None, GraphBinarySerializersV4(),
+ interceptors=[one, two, three])
+ mock_transport = MockHTTPTransport()
+ protocol.connection_made(mock_transport)
+
+ message = RequestMessage(fields={}, gremlin=[])
+ protocol.write(message)
+ written = mock_transport.get_write()
+
+ assert written["payload"].gremlin == [1, 2, 3]
diff --git a/gremlin-python/src/main/python/tests/process/test_traversal.py b/gremlin-python/src/main/python/tests/process/test_traversal.py
index c7f64a5990..a69b8100fe 100644
--- a/gremlin-python/src/main/python/tests/process/test_traversal.py
+++ b/gremlin-python/src/main/python/tests/process/test_traversal.py
@@ -359,8 +359,7 @@ class TestTraversal(object):
def create_connection_to_gtx():
- return DriverRemoteConnection(anonymous_url, 'gtx',
- message_serializer=serializer.GraphBinarySerializersV4())
+ return DriverRemoteConnection(anonymous_url, 'gtx')
def add_node_validate_transaction_state(g, g_add_to, g_start_count, g_add_to_start_count, tx_verify_list):
diff --git a/gremlin-python/src/main/python/tests/structure/io/test_functionalityio.py b/gremlin-python/src/main/python/tests/structure/io/test_functionalityio.py
index 8dbd0a5af5..be74a28710 100644
--- a/gremlin-python/src/main/python/tests/structure/io/test_functionalityio.py
+++ b/gremlin-python/src/main/python/tests/structure/io/test_functionalityio.py
@@ -110,7 +110,7 @@ def test_uuid(remote_connection):
def test_short(remote_connection):
- if not isinstance(remote_connection._client._message_serializer, GraphBinarySerializersV4):
+ if not isinstance(remote_connection._client.response_serializer(), GraphBinarySerializersV4):
return
g = traversal().with_(remote_connection)
@@ -126,7 +126,7 @@ def test_short(remote_connection):
def test_bigint_positive(remote_connection):
- if not isinstance(remote_connection._client._message_serializer, GraphBinarySerializersV4):
+ if not isinstance(remote_connection._client.response_serializer(), GraphBinarySerializersV4):
return
g = traversal().with_(remote_connection)
@@ -142,7 +142,7 @@ def test_bigint_positive(remote_connection):
def test_bigint_negative(remote_connection):
- if not isinstance(remote_connection._client._message_serializer, GraphBinarySerializersV4):
+ if not isinstance(remote_connection._client.response_serializer(), GraphBinarySerializersV4):
return
g = traversal().with_(remote_connection)
@@ -159,7 +159,7 @@ def test_bigint_negative(remote_connection):
@pytest.mark.skip(reason="BigDecimal implementation needs revisiting")
def test_bigdecimal(remote_connection):
- if not isinstance(remote_connection._client._message_serializer, GraphBinarySerializersV4):
+ if not isinstance(remote_connection._client.response_serializer(), GraphBinarySerializersV4):
return
g = traversal().with_(remote_connection)
(tinkerpop) 14/17: change from streaming deserialization to aggregating data and make graphson class naming consistent (#2873)
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 6c7a31fd6dca875a98b5e5da1bf14de64f31778a
Author: Yang Xia <55...@users.noreply.github.com>
AuthorDate: Thu Oct 31 16:28:05 2024 -0700
change from streaming deserialization to aggregating data and make graphson class naming consistent (#2873)
---
gremlin-python/docker-compose.yml | 3 ++-
.../gremlin_python/driver/aiohttp/transport.py | 22 ++++++++++++++-
.../python/gremlin_python/driver/connection.py | 8 +++++-
.../main/python/gremlin_python/driver/protocol.py | 31 ++++++++++++++++++++++
.../python/gremlin_python/driver/serializer.py | 4 +--
gremlin-python/src/main/python/radish/terrain.py | 2 ++
gremlin-python/src/main/python/tests/conftest.py | 11 ++++++--
.../tests/driver/test_driver_remote_connection.py | 4 +--
8 files changed, 76 insertions(+), 9 deletions(-)
diff --git a/gremlin-python/docker-compose.yml b/gremlin-python/docker-compose.yml
index f302c54823..b912c907cc 100644
--- a/gremlin-python/docker-compose.yml
+++ b/gremlin-python/docker-compose.yml
@@ -64,7 +64,8 @@ services:
&& python3 ./setup.py test
&& python3 ./setup.py install
&& radish -f dots -e -t -b ./radish ./gremlin-test --user-data='serializer=application/vnd.graphbinary-v4.0' --user-data='bulked=true'
- && radish -f dots -e -t -b ./radish ./gremlin-test --user-data='serializer=application/vnd.graphbinary-v4.0';
+ && radish -f dots -e -t -b ./radish ./gremlin-test --user-data='serializer=application/vnd.graphbinary-v4.0'
+ && radish -f dots -e -t -b ./radish ./gremlin-test --user-data='serializer=application/vnd.gremlin-v4.0+json';
EXIT_CODE=$$?; chown -R `stat -c "%u:%g" .` .; exit $$EXIT_CODE"
depends_on:
gremlin-server-test-python:
diff --git a/gremlin-python/src/main/python/gremlin_python/driver/aiohttp/transport.py b/gremlin-python/src/main/python/gremlin_python/driver/aiohttp/transport.py
index b2827e9085..399f6806b8 100644
--- a/gremlin-python/src/main/python/gremlin_python/driver/aiohttp/transport.py
+++ b/gremlin-python/src/main/python/gremlin_python/driver/aiohttp/transport.py
@@ -105,7 +105,27 @@ class AiohttpHTTPTransport(AbstractBaseTransport):
def read(self, stream_chunk=None):
if not stream_chunk:
- raise Exception('missing handling of streamed responses to protocol')
+ '''
+ GraphSON does not support streaming deserialization, we are aggregating data and bypassing streamed
+ deserialization while GraphSON is enabled for testing. Remove after GraphSON is removed.
+ '''
+ async def async_read():
+ async with async_timeout.timeout(self._read_timeout):
+ data_buffer = b""
+ async for data, end_of_http_chunk in self._http_req_resp.content.iter_chunks():
+ try:
+ data_buffer += data
+ except ClientPayloadError:
+ # server disconnect during streaming will cause ClientPayLoadError from aiohttp
+ raise GremlinServerError({'code': 500,
+ 'message': 'Server disconnected - please try to reconnect',
+ 'exception': ClientPayloadError})
+ if self._max_content_len and len(
+ data_buffer) > self._max_content_len:
+ raise Exception(f'Response size {len(data_buffer)} exceeds limit {self._max_content_len} bytes')
+ return data_buffer
+ return self._loop.run_until_complete(async_read())
+ # raise Exception('missing handling of streamed responses to protocol')
# Inner function to perform async read.
async def async_read():
diff --git a/gremlin-python/src/main/python/gremlin_python/driver/connection.py b/gremlin-python/src/main/python/gremlin_python/driver/connection.py
index cb34923d0c..434272c5c5 100644
--- a/gremlin-python/src/main/python/gremlin_python/driver/connection.py
+++ b/gremlin-python/src/main/python/gremlin_python/driver/connection.py
@@ -84,7 +84,13 @@ class Connection:
def _receive(self):
try:
- self._transport.read(self.stream_chunk)
+ '''
+ GraphSON does not support streaming deserialization, we are aggregating data and bypassing streamed
+ deserialization while GraphSON is enabled for testing. Remove after GraphSON is removed.
+ '''
+ self._protocol.data_received_aggregate(self._transport.read(), self._result_set)
+ # re-enable streaming after graphSON removal
+ # self._transport.read(self.stream_chunk)
finally:
self._pool.put_nowait(self)
diff --git a/gremlin-python/src/main/python/gremlin_python/driver/protocol.py b/gremlin-python/src/main/python/gremlin_python/driver/protocol.py
index 930d2de328..48c709de17 100644
--- a/gremlin-python/src/main/python/gremlin_python/driver/protocol.py
+++ b/gremlin-python/src/main/python/gremlin_python/driver/protocol.py
@@ -95,6 +95,37 @@ class GremlinServerHTTPProtocol(AbstractBaseProtocol):
self._transport.write(message)
+ '''
+ GraphSON does not support streaming deserialization, we are aggregating data and bypassing streamed
+ deserialization while GraphSON is enabled for testing. Remove after GraphSON is removed.
+ '''
+ def data_received_aggregate(self, response, result_set):
+ response_msg = {'status': {'code': 0,
+ 'message': '',
+ 'exception': ''},
+ 'result': {'meta': {},
+ 'data': []}}
+
+ response_msg = self._decode_chunk(response_msg, response, self._is_first_chunk)
+
+ self._is_first_chunk = False
+ status_code = response_msg['status']['code']
+ aggregate_to = response_msg['result']['meta'].get('aggregateTo', 'list')
+ data = response_msg['result']['data']
+ result_set.aggregate_to = aggregate_to
+ self._is_first_chunk = True
+
+ if status_code == 204 and len(data) == 0:
+ result_set.stream.put_nowait([])
+ elif status_code in [200, 204, 206]:
+ result_set.stream.put_nowait(data)
+ else:
+ log.error("\r\nReceived error message '%s'\r\n\r\nWith result set '%s'",
+ str(self._response_msg), str(result_set))
+ raise GremlinServerError({'code': status_code,
+ 'message': self._response_msg['status']['message'],
+ 'exception': self._response_msg['status']['exception']})
+
# data is received in chunks
def data_received(self, response_chunk, result_set, read_completed=None, http_req_resp=None):
# we shouldn't need to use the http_req_resp code as status is sent in response message, but leaving it for now
diff --git a/gremlin-python/src/main/python/gremlin_python/driver/serializer.py b/gremlin-python/src/main/python/gremlin_python/driver/serializer.py
index 8e207ae986..aa4c70ac7b 100644
--- a/gremlin-python/src/main/python/gremlin_python/driver/serializer.py
+++ b/gremlin-python/src/main/python/gremlin_python/driver/serializer.py
@@ -40,7 +40,7 @@ __author__ = 'David M. Brown (davebshow@gmail.com), Lyndon Bauto (lyndonb@bitqui
GraphSONV4
"""
-class GraphSONSerializerV4(object):
+class GraphSONSerializersV4(object):
"""
Message serializer for GraphSON. Allow users to pass custom reader,
writer, and version kwargs for custom serialization. Otherwise,
@@ -89,7 +89,7 @@ class GraphSONSerializerV4(object):
msg = json.loads(message if isinstance(message, str) else message.decode('utf-8'))
return self._graphson_reader.to_object(msg)
else:
- # graphSON does not stream, the first chunk contains all info
+ # graphSON does not stream, all results are aggregated inside the first chunk
return ""
"""
diff --git a/gremlin-python/src/main/python/radish/terrain.py b/gremlin-python/src/main/python/radish/terrain.py
index c54334ab0c..f341106467 100644
--- a/gremlin-python/src/main/python/radish/terrain.py
+++ b/gremlin-python/src/main/python/radish/terrain.py
@@ -96,6 +96,8 @@ def __create_remote(server_graph_name):
if world.config.user_data["serializer"] == "application/vnd.graphbinary-v4.0":
s = serializer.GraphBinarySerializersV4()
+ elif world.config.user_data["serializer"] == "application/vnd.gremlin-v4.0+json":
+ s = serializer.GraphSONSerializersV4()
else:
raise ValueError('serializer not found - ' + world.config.user_data["serializer"])
diff --git a/gremlin-python/src/main/python/tests/conftest.py b/gremlin-python/src/main/python/tests/conftest.py
index 0aafc43d1b..cc958c7c5d 100644
--- a/gremlin-python/src/main/python/tests/conftest.py
+++ b/gremlin-python/src/main/python/tests/conftest.py
@@ -25,6 +25,7 @@ import pytest
import logging
import queue
+from gremlin_python.driver import serializer
from gremlin_python.driver.client import Client
from gremlin_python.driver.connection import Connection
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
@@ -110,11 +111,17 @@ def graphbinary_serializer_v4(request):
return GraphBinarySerializersV4()
-@pytest.fixture(params=['graphbinaryv4'])
+@pytest.fixture(params=['graphbinaryv4','graphsonv4'])
def remote_connection(request):
try:
if request.param == 'graphbinaryv4':
- remote_conn = DriverRemoteConnection(anonymous_url, 'gmodern')
+ remote_conn = DriverRemoteConnection(anonymous_url, 'gmodern',
+ request_serializer=serializer.GraphBinarySerializersV4(),
+ response_serializer=serializer.GraphBinarySerializersV4())
+ elif request.param == 'graphsonv4':
+ remote_conn = DriverRemoteConnection(anonymous_url, 'gmodern',
+ request_serializer=serializer.GraphSONSerializersV4(),
+ response_serializer=serializer.GraphSONSerializersV4())
else:
raise ValueError("Invalid serializer option - " + request.param)
except OSError:
diff --git a/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py b/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
index 95004076c4..86d6445316 100644
--- a/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
+++ b/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
@@ -40,8 +40,8 @@ class TestDriverRemoteConnection(object):
# in conftest.py and remove this
def test_graphSONV4_temp(self):
remote_conn = DriverRemoteConnection(test_no_auth_url, 'gmodern',
- request_serializer=serializer.GraphSONSerializerV4(),
- response_serializer=serializer.GraphSONSerializerV4())
+ request_serializer=serializer.GraphSONSerializersV4(),
+ response_serializer=serializer.GraphSONSerializersV4())
g = traversal().with_(remote_conn)
assert long(6) == g.V().count().to_list()[0]
# #
(tinkerpop) 07/17: Update BulkSet GraphSONv4 serialization. (#2861)
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit b856745e7be81a7d6bd477a8dd82fb10c0463367
Author: kenhuuu <10...@users.noreply.github.com>
AuthorDate: Tue Oct 29 12:55:04 2024 -0700
Update BulkSet GraphSONv4 serialization. (#2861)
---
docs/src/dev/io/graphson.asciidoc | 24 ++++++++++++++++-
.../structure/io/graphson/GraphSONModule.java | 6 +++--
.../io/graphson/TraversalSerializersV4.java | 31 ++--------------------
.../graphson/GraphSONMapperEmbeddedTypeTest.java | 7 ++---
.../structure/io/graphson/empty-bulklist-v4.json | 4 +++
.../structure/io/graphson/empty-bulkset-v4.json | 4 ---
.../structure/io/graphson/var-bulklist-v4.json | 4 +++
.../structure/io/graphson/var-bulkset-v4.json | 10 -------
.../io/AbstractTypedCompatibilityTest.java | 2 --
9 files changed, 41 insertions(+), 51 deletions(-)
diff --git a/docs/src/dev/io/graphson.asciidoc b/docs/src/dev/io/graphson.asciidoc
index 8d70cd2509..cede81a2da 100644
--- a/docs/src/dev/io/graphson.asciidoc
+++ b/docs/src/dev/io/graphson.asciidoc
@@ -2615,7 +2615,7 @@ The untyped version has one additional required key "type" which is always "vert
==== VertexProperty
-JOSN Object with required keys: "id", "value", "label", "properties" +
+JSON Object with required keys: "id", "value", "label", "properties" +
"id" is any type GraphSON 4.0 type +
"value" is any type GraphSON 4.0 type +
"label" is a `g:List` of `String` +
@@ -2651,6 +2651,28 @@ JOSN Object with required keys: "id", "value", "label", "properties" +
=== Graph Process
+==== BulkSet
+
+JSON Array that contains the expanded entries of the BulkSet.
+Note: BulkSet is serialized to g:List so there is no BulkSet deserializer.
+
+[source,json]
+----
+{
+ "@type": "g:List",
+ "@value": [
+ "marko",
+ "josh",
+ "josh"
+ ]
+}
+----
+
+[source,json]
+----
+[ "marko", "josh", "josh" ]
+----
+
==== Direction
JSON String of the enum value.
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java
index 904809360d..b19cbb4931 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java
@@ -149,6 +149,10 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
put(Double.class, "Double");
put(Float.class, "Float");
+ // BulkSet is expanded to List during serialization but we want the List deserializer to be
+ // registered to g:List so this entry must be added before List.
+ put(BulkSet.class, "List");
+
put(Map.class, "Map");
put(List.class, "List");
put(Set.class, "Set");
@@ -165,7 +169,6 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
put(TraversalExplanation.class, "TraversalExplanation");
put(Traverser.class, "Traverser");
put(Tree.class, "Tree");
- put(BulkSet.class, "BulkSet");
put(AndP.class, "P");
put(OrP.class, "P");
put(P.class, "P");
@@ -299,7 +302,6 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
addDeserializer(Double.class, new GraphSONSerializersV4.DoubleJacksonDeserializer());
// traversal
- addDeserializer(BulkSet.class, new TraversalSerializersV4.BulkSetJacksonDeserializer());
Stream.of(VertexProperty.Cardinality.values(),
Column.values(),
Direction.values(),
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java
index 041824525d..fec60d2aa8 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java
@@ -152,9 +152,8 @@ final class TraversalSerializersV4 {
public void serialize(final BulkSet bulkSet, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider)
throws IOException {
jsonGenerator.writeStartArray();
- for (Map.Entry entry : (Set<Map.Entry>) bulkSet.asBulk().entrySet()) {
- jsonGenerator.writeObject(entry.getKey());
- jsonGenerator.writeObject(entry.getValue());
+ for (Object element : bulkSet) {
+ jsonGenerator.writeObject(element);
}
jsonGenerator.writeEndArray();
}
@@ -401,32 +400,6 @@ final class TraversalSerializersV4 {
}
}
- final static class BulkSetJacksonDeserializer extends StdDeserializer<BulkSet> {
- public BulkSetJacksonDeserializer() {
- super(BulkSet.class);
- }
-
- @Override
- public BulkSet deserialize(final JsonParser jsonParser, final DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
-
- final BulkSet<Object> bulkSet = new BulkSet<>();
-
- while (jsonParser.nextToken() != JsonToken.END_ARRAY) {
- final Object key = deserializationContext.readValue(jsonParser, Object.class);
- jsonParser.nextToken();
- final Long val = deserializationContext.readValue(jsonParser, Long.class);
- bulkSet.add(key, val);
- }
-
- return bulkSet;
- }
-
- @Override
- public boolean isCachable() {
- return true;
- }
- }
-
static class TraverserJacksonDeserializer extends StdDeserializer<Traverser> {
public TraverserJacksonDeserializer() {
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java
index b6ae2dc189..94086371dc 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java
@@ -119,15 +119,16 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
@Test
public void shouldHandleBulkSet() throws Exception {
- // only supported on V3/V4
- assumeThat(version, not(anyOf(startsWith("v1"), startsWith("v2"))));
+ // only supported on V4
+ assumeThat(version, not(anyOf(startsWith("v1"), startsWith("v2"), startsWith("v3"))));
final BulkSet<String> bs = new BulkSet<>();
bs.add("test1", 1);
bs.add("test2", 2);
bs.add("test3", 3);
- assertEquals(bs, serializeDeserialize(mapper, bs, BulkSet.class));
+ final List<String> expandedBs = new ArrayList<>(bs);
+ assertEquals(expandedBs, serializeDeserializeAuto(mapper, bs));
}
@Test
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/empty-bulklist-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/empty-bulklist-v4.json
new file mode 100644
index 0000000000..aacbd803fb
--- /dev/null
+++ b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/empty-bulklist-v4.json
@@ -0,0 +1,4 @@
+{
+ "@type" : "g:List",
+ "@value" : [ ]
+}
\ No newline at end of file
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/empty-bulkset-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/empty-bulkset-v4.json
deleted file mode 100644
index f9fb4a5eed..0000000000
--- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/empty-bulkset-v4.json
+++ /dev/null
@@ -1,4 +0,0 @@
-{
- "@type" : "g:BulkSet",
- "@value" : [ ]
-}
\ No newline at end of file
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-bulklist-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-bulklist-v4.json
new file mode 100644
index 0000000000..ac763480b2
--- /dev/null
+++ b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-bulklist-v4.json
@@ -0,0 +1,4 @@
+{
+ "@type" : "g:List",
+ "@value" : [ "marko", "josh", "josh" ]
+}
\ No newline at end of file
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-bulkset-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-bulkset-v4.json
deleted file mode 100644
index c216a36171..0000000000
--- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/var-bulkset-v4.json
+++ /dev/null
@@ -1,10 +0,0 @@
-{
- "@type" : "g:BulkSet",
- "@value" : [ "marko", {
- "@type" : "g:Int64",
- "@value" : 1
- }, "josh", {
- "@type" : "g:Int64",
- "@value" : 2
- } ]
-}
\ No newline at end of file
diff --git a/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java b/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java
index 3b7bdf9093..bd8dee9cb4 100644
--- a/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java
+++ b/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java
@@ -882,7 +882,6 @@ public abstract class AbstractTypedCompatibilityTest extends AbstractCompatibili
}
@Test
- @Ignore("re-enable after GraphSONV4 bulklist is implemented")
public void shouldReadWriteVarBulkList() throws Exception {
final String resourceName = "var-bulklist";
@@ -898,7 +897,6 @@ public abstract class AbstractTypedCompatibilityTest extends AbstractCompatibili
}
@Test
- @Ignore("re-enable after GraphSONV4 bulklist is implemented")
public void shouldReadWriteEmptyBulkList() throws Exception {
final String resourceName = "empty-bulklist";
(tinkerpop) 09/17: Re-enable feature test of GraphSONv4 CTR
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 5cea695c45cef5542622ce4a43d976ae45da1a2c
Author: Ken Hu <10...@users.noreply.github.com>
AuthorDate: Tue Oct 29 20:45:24 2024 -0700
Re-enable feature test of GraphSONv4 CTR
---
.../remote/GraphSONLangRemoteFeatureTest.java | 51 +++++++++++++++++++++
.../remote/GraphSONRemoteComputerFeatureTest.java | 52 ++++++++++++++++++++++
.../gremlin/driver/remote/RemoteWorld.java | 24 ++++++++++
.../io.cucumber.core.backend.ObjectFactory | 2 +
4 files changed, 129 insertions(+)
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/GraphSONLangRemoteFeatureTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/GraphSONLangRemoteFeatureTest.java
new file mode 100644
index 0000000000..550ed25713
--- /dev/null
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/GraphSONLangRemoteFeatureTest.java
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.driver.remote;
+
+import com.google.inject.AbstractModule;
+import com.google.inject.Guice;
+import com.google.inject.Stage;
+import io.cucumber.guice.CucumberModules;
+import io.cucumber.junit.Cucumber;
+import io.cucumber.junit.CucumberOptions;
+import org.apache.tinkerpop.gremlin.features.AbstractGuiceFactory;
+import org.apache.tinkerpop.gremlin.features.World;
+import org.junit.runner.RunWith;
+
+@RunWith(Cucumber.class)
+@CucumberOptions(
+ tags = "not @RemoteOnly and not @GraphComputerOnly and not @AllowNullPropertyValues",
+ glue = { "org.apache.tinkerpop.gremlin.features" },
+ objectFactory = GraphSONLangRemoteFeatureTest.RemoteGuiceFactory.class,
+ features = { "classpath:/org/apache/tinkerpop/gremlin/test/features" },
+ plugin = {"progress", "junit:target/cucumber.xml"})
+public class GraphSONLangRemoteFeatureTest extends AbstractFeatureTest {
+ public static class RemoteGuiceFactory extends AbstractGuiceFactory {
+ public RemoteGuiceFactory() {
+ super(Guice.createInjector(Stage.PRODUCTION, CucumberModules.createScenarioModule(), new ServiceModule()));
+ }
+ }
+
+ public static final class ServiceModule extends AbstractModule {
+ @Override
+ protected void configure() {
+ bind(World.class).to(RemoteWorld.GraphSONLangRemoteWorld.class);
+ }
+ }
+}
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/GraphSONRemoteComputerFeatureTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/GraphSONRemoteComputerFeatureTest.java
new file mode 100644
index 0000000000..6e92604426
--- /dev/null
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/GraphSONRemoteComputerFeatureTest.java
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.driver.remote;
+
+import com.google.inject.AbstractModule;
+import com.google.inject.Guice;
+import com.google.inject.Stage;
+import io.cucumber.guice.CucumberModules;
+import io.cucumber.junit.Cucumber;
+import io.cucumber.junit.CucumberOptions;
+import org.apache.tinkerpop.gremlin.features.AbstractGuiceFactory;
+import org.apache.tinkerpop.gremlin.features.World;
+import org.junit.runner.RunWith;
+
+@RunWith(Cucumber.class)
+@CucumberOptions(
+ tags = "not @RemoteOnly and not @GraphComputerVerificationElementSupported and " +
+ "not @GraphComputerVerificationReferenceOnly and not @InsertionOrderingRequired",
+ glue = { "org.apache.tinkerpop.gremlin.features" },
+ objectFactory = GraphSONRemoteComputerFeatureTest.RemoteGuiceFactory.class,
+ features = { "classpath:/org/apache/tinkerpop/gremlin/test/features" },
+ plugin = {"progress", "junit:target/cucumber.xml"})
+public class GraphSONRemoteComputerFeatureTest extends AbstractFeatureTest {
+ public static class RemoteGuiceFactory extends AbstractGuiceFactory {
+ public RemoteGuiceFactory() {
+ super(Guice.createInjector(Stage.PRODUCTION, CucumberModules.createScenarioModule(), new ServiceModule()));
+ }
+ }
+
+ public static final class ServiceModule extends AbstractModule {
+ @Override
+ protected void configure() {
+ bind(World.class).to(RemoteWorld.GraphSONRemoteComputerWorld.class);
+ }
+ }
+}
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/RemoteWorld.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/RemoteWorld.java
index 84109b080e..668bc57ca1 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/RemoteWorld.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/remote/RemoteWorld.java
@@ -200,4 +200,28 @@ public abstract class RemoteWorld implements World {
return g.with("language", "groovy-test");
}
}
+
+ ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+ public static class GraphSONLangRemoteWorld extends RemoteWorld {
+ public GraphSONLangRemoteWorld() { super(createTestCluster(Serializers.GRAPHSON_V4)); }
+
+ @Override
+ public GraphTraversalSource getGraphTraversalSource(final LoadGraphWith.GraphData graphData) {
+ final GraphTraversalSource g = super.getGraphTraversalSource(graphData);
+ return g.with("language", "gremlin-lang");
+ }
+ }
+
+ ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+
+ public static class GraphSONRemoteComputerWorld extends RemoteComputerWorld {
+ public GraphSONRemoteComputerWorld() { super(createTestCluster(Serializers.GRAPHSON_V4)); }
+
+ @Override
+ public GraphTraversalSource getGraphTraversalSource(final LoadGraphWith.GraphData graphData) {
+ final GraphTraversalSource g = super.getGraphTraversalSource(graphData);
+ return g.with("language", "groovy-test");
+ }
+ }
}
diff --git a/gremlin-server/src/test/resources/META-INF/services/io.cucumber.core.backend.ObjectFactory b/gremlin-server/src/test/resources/META-INF/services/io.cucumber.core.backend.ObjectFactory
index 7b70b113b0..d7c53bc1bd 100644
--- a/gremlin-server/src/test/resources/META-INF/services/io.cucumber.core.backend.ObjectFactory
+++ b/gremlin-server/src/test/resources/META-INF/services/io.cucumber.core.backend.ObjectFactory
@@ -2,3 +2,5 @@ org.apache.tinkerpop.gremlin.driver.remote.GraphBinaryLangRemoteFeatureTest$Remo
org.apache.tinkerpop.gremlin.driver.remote.GraphBinaryLangBulkedRemoteFeatureTest$RemoteGuiceFactory
org.apache.tinkerpop.gremlin.driver.remote.GraphBinaryGroovyRemoteFeatureTest$RemoteGuiceFactory
org.apache.tinkerpop.gremlin.driver.remote.GraphBinaryRemoteComputerFeatureTest$RemoteGuiceFactory
+org.apache.tinkerpop.gremlin.driver.remote.GraphSONLangRemoteFeatureTest$RemoteGuiceFactory
+org.apache.tinkerpop.gremlin.driver.remote.GraphSONRemoteComputerFeatureTest$RemoteGuiceFactory
(tinkerpop) 10/17: Update GraphSONV4 extended types, element, and datetime in gremlin-python (#2854)
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 7b3caefcf0775708de1a0adf5f7ce81328cfaf0c
Author: Yang Xia <55...@users.noreply.github.com>
AuthorDate: Wed Oct 30 14:54:46 2024 -0700
Update GraphSONV4 extended types, element, and datetime in gremlin-python (#2854)
Fix graphson element, datetime, and naming to comply with v4 specs and added tests.
---
gremlin-python/docker-compose.yml | 2 +
.../python/gremlin_python/driver/serializer.py | 7 +-
.../src/main/python/gremlin_python/statics.py | 1 -
.../gremlin_python/structure/io/graphbinaryV4.py | 12 +-
.../gremlin_python/structure/io/graphsonV4.py | 125 +++--
.../tests/driver/test_driver_remote_connection.py | 41 +-
.../src/main/python/tests/structure/io/model.py | 6 +-
.../tests/structure/io/test_graphbinaryV4.py | 6 +-
.../python/tests/structure/io/test_graphsonV4.py | 531 +++++++++++++++++++++
.../tests/structure/io/test_graphsonv4model.py | 308 ++++++++++++
10 files changed, 955 insertions(+), 84 deletions(-)
diff --git a/gremlin-python/docker-compose.yml b/gremlin-python/docker-compose.yml
index 83d7722b7a..f302c54823 100644
--- a/gremlin-python/docker-compose.yml
+++ b/gremlin-python/docker-compose.yml
@@ -48,12 +48,14 @@ services:
- ${BUILD_DIR:-./src/main/python}:/python_app
- ../gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/test/features:/python_app/gremlin-test/features
- ../gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary:/python_app/gremlin-test/graphbinary
+ - ../gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson:/python_app/gremlin-test/graphson
- ../docker/gremlin-test-server:/python_app/gremlin-test-server
environment:
- DEBIAN_FRONTEND=noninteractive
- GREMLIN_SERVER_URL=http://gremlin-server-test-python:{}/gremlin
- GREMLIN_SERVER_BASIC_AUTH_URL=https://gremlin-server-test-python:{}/gremlin
- IO_TEST_DIRECTORY=/python_app/gremlin-test/graphbinary/
+ - IO_TEST_DIRECTORY_GRAPHSON=/python_app/gremlin-test/graphson/
- VERSION=${VERSION}
working_dir: /python_app
command: >
diff --git a/gremlin-python/src/main/python/gremlin_python/driver/serializer.py b/gremlin-python/src/main/python/gremlin_python/driver/serializer.py
index 9aa4a7c0cc..8e207ae986 100644
--- a/gremlin-python/src/main/python/gremlin_python/driver/serializer.py
+++ b/gremlin-python/src/main/python/gremlin_python/driver/serializer.py
@@ -73,8 +73,11 @@ class GraphSONSerializerV4(object):
return message
def build_message(self, fields, gremlin):
- message = fields
- message['gremlin'] = gremlin
+ message = {
+ 'gremlin': gremlin # gremlin wil always be a gremlin lang string script
+ }
+ for k, v in fields.items():
+ message[k] = self._graphson_writer.to_dict(v)
return self.finalize_message(message)
def finalize_message(self, message):
diff --git a/gremlin-python/src/main/python/gremlin_python/statics.py b/gremlin-python/src/main/python/gremlin_python/statics.py
index c3b9cd8d5b..ac05fe2448 100644
--- a/gremlin-python/src/main/python/gremlin_python/statics.py
+++ b/gremlin-python/src/main/python/gremlin_python/statics.py
@@ -42,7 +42,6 @@ TypeType = type
ListType = list
DictType = dict
SetType = set
-ByteBufferType = bytes
class SingleByte(int):
diff --git a/gremlin-python/src/main/python/gremlin_python/structure/io/graphbinaryV4.py b/gremlin-python/src/main/python/gremlin_python/structure/io/graphbinaryV4.py
index d8edfeb99c..7f646b0cbb 100644
--- a/gremlin-python/src/main/python/gremlin_python/structure/io/graphbinaryV4.py
+++ b/gremlin-python/src/main/python/gremlin_python/structure/io/graphbinaryV4.py
@@ -29,7 +29,7 @@ from struct import pack, unpack
from aenum import Enum
from gremlin_python.process.traversal import Direction, T
from gremlin_python.statics import FloatType, BigDecimal, ShortType, IntType, LongType, BigIntType, \
- DictType, SetType, SingleByte, ByteBufferType, SingleChar
+ DictType, SetType, SingleByte, SingleChar
from gremlin_python.structure.graph import Graph, Edge, Property, Vertex, VertexProperty, Path
from gremlin_python.structure.io.util import HashableDict, SymbolUtil, Marker
@@ -65,7 +65,7 @@ class DataType(Enum):
bigdecimal = 0x22
biginteger = 0x23
byte = 0x24
- bytebuffer = 0x25
+ binary = 0x25
short = 0x26
boolean = 0x27
tree = 0x2b # not supported - no tree object in Python yet
@@ -757,9 +757,9 @@ class ByteIO(_GraphBinaryTypeIO):
nullable)
-class ByteBufferIO(_GraphBinaryTypeIO):
- python_type = ByteBufferType
- graphbinary_type = DataType.bytebuffer
+class BinaryIO(_GraphBinaryTypeIO):
+ python_type = bytes
+ graphbinary_type = DataType.binary
@classmethod
def dictify(cls, obj, writer, to_extend, as_value=False, nullable=True):
@@ -775,7 +775,7 @@ class ByteBufferIO(_GraphBinaryTypeIO):
@classmethod
def _read_bytebuffer(cls, b, r):
size = cls.read_int(b)
- return ByteBufferType(b.read(size))
+ return bytes(b.read(size))
class BooleanIO(_GraphBinaryTypeIO):
diff --git a/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py b/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
index b4aa99bf8e..7a63c8bee7 100644
--- a/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
+++ b/gremlin-python/src/main/python/gremlin_python/structure/io/graphsonV4.py
@@ -14,7 +14,7 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-
+import base64
import calendar
import datetime
import json
@@ -29,7 +29,7 @@ from aenum import Enum
from isodate import parse_duration, duration_isoformat
from gremlin_python import statics
-from gremlin_python.statics import FloatType, FunctionType, ShortType, IntType, LongType, TypeType, DictType, ListType, SetType, SingleByte, ByteBufferType, SingleChar
+from gremlin_python.statics import FloatType, BigDecimal, ShortType, IntType, LongType, TypeType, DictType, ListType, SetType, SingleByte, SingleChar
from gremlin_python.process.traversal import Direction, P, TextP, Traversal, Traverser, TraversalStrategy, T
from gremlin_python.structure.graph import Edge, Property, Vertex, VertexProperty, Path
from gremlin_python.structure.io.util import HashableDict, SymbolUtil
@@ -180,7 +180,7 @@ class VertexSerializer(_GraphSONTypeIO):
@classmethod
def dictify(cls, vertex, writer):
return GraphSONUtil.typed_value("Vertex", {"id": writer.to_dict(vertex.id),
- "label": writer.to_dict([vertex.label])})
+ "label": [writer.to_dict(vertex.label)]})
class EdgeSerializer(_GraphSONTypeIO):
@@ -190,14 +190,15 @@ class EdgeSerializer(_GraphSONTypeIO):
@classmethod
def dictify(cls, edge, writer):
return GraphSONUtil.typed_value("Edge", {"id": writer.to_dict(edge.id),
- "label": writer.to_dict([edge.label]),
- "inV": writer.to_dict(edge.inV),
- "outV": writer.to_dict(edge.outV)})
-
- # "outV": writer.to_dict(edge.outV.id),
- # "outVLabel": writer.to_dict([edge.outV.label]),
- # "inV": writer.to_dict(edge.inV.id),
- # "inVLabel": writer.to_dict([edge.inV.label])})
+ "label": [writer.to_dict(edge.label)],
+ "inV": {
+ "id": writer.to_dict(edge.inV.id),
+ "label": [writer.to_dict(edge.inV.label)]
+ },
+ "outV": {
+ "id": writer.to_dict(edge.outV.id),
+ "label": [writer.to_dict(edge.outV.label)]
+ }})
class VertexPropertySerializer(_GraphSONTypeIO):
@@ -207,9 +208,8 @@ class VertexPropertySerializer(_GraphSONTypeIO):
@classmethod
def dictify(cls, vertex_property, writer):
return GraphSONUtil.typed_value("VertexProperty", {"id": writer.to_dict(vertex_property.id),
- "label": writer.to_dict([vertex_property.label]),
- "value": writer.to_dict(vertex_property.value),
- "vertex": writer.to_dict(vertex_property.vertex.id)})
+ "label": [writer.to_dict(vertex_property.label)],
+ "value": writer.to_dict(vertex_property.value)})
class PropertySerializer(_GraphSONTypeIO):
@@ -218,20 +218,8 @@ class PropertySerializer(_GraphSONTypeIO):
@classmethod
def dictify(cls, property, writer):
- elementDict = writer.to_dict(property.element)
- if elementDict is not None:
- valueDict = elementDict["@value"]
- if "outVLabel" in valueDict:
- del valueDict["outVLabel"]
- if "inVLabel" in valueDict:
- del valueDict["inVLabel"]
- if "properties" in valueDict:
- del valueDict["properties"]
- if "value" in valueDict:
- del valueDict["value"]
return GraphSONUtil.typed_value("Property", {"key": writer.to_dict(property.key),
- "value": writer.to_dict(property.value),
- "element": elementDict})
+ "value": writer.to_dict(property.value)})
class TraversalStrategySerializer(_GraphSONTypeIO):
@@ -313,26 +301,22 @@ class UUIDIO(_GraphSONTypeIO):
return cls.python_type(d)
-class DateIO(_GraphSONTypeIO):
+class DateTimeIO(_GraphSONTypeIO):
python_type = datetime.datetime
- graphson_type = "g:Date"
- graphson_base_type = "Date"
+ graphson_type = "g:DateTime"
+ graphson_base_type = "DateTime"
@classmethod
def dictify(cls, obj, writer):
- try:
- timestamp_seconds = calendar.timegm(obj.utctimetuple())
- pts = timestamp_seconds * 1e3 + getattr(obj, 'microsecond', 0) / 1e3
- except AttributeError:
- pts = calendar.timegm(obj.timetuple()) * 1e3
-
- ts = int(round(pts))
- return GraphSONUtil.typed_value(cls.graphson_base_type, ts)
+ if obj.tzinfo is None:
+ raise AttributeError("Timezone information is required when constructing datetime")
+ return GraphSONUtil.typed_value(cls.graphson_base_type, obj.isoformat())
@classmethod
- def objectify(cls, ts, reader):
- # Python timestamp expects seconds
- return datetime.datetime.utcfromtimestamp(ts / 1000.0)
+ def objectify(cls, dt, reader):
+ # specially handling as python isoformat does not support zulu until 3.11
+ dt_iso = dt[:-1] + '+00:00' if dt.endswith('Z') else dt
+ return datetime.datetime.fromisoformat(dt_iso)
class _NumberIO(_GraphSONTypeIO):
@@ -473,7 +457,7 @@ class FloatIO(_NumberIO):
class BigDecimalIO(_NumberIO):
python_type = Decimal
- graphson_type = "gx:BigDecimal"
+ graphson_type = "g:BigDecimal"
graphson_base_type = "BigDecimal"
@classmethod
@@ -481,13 +465,13 @@ class BigDecimalIO(_NumberIO):
if isinstance(n, bool): # because isinstance(False, int) and isinstance(True, int)
return n
elif math.isnan(n):
- return GraphSONUtil.typed_value(cls.graphson_base_type, "NaN", "gx")
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "NaN", "g")
elif math.isinf(n) and n > 0:
- return GraphSONUtil.typed_value(cls.graphson_base_type, "Infinity", "gx")
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "Infinity", "g")
elif math.isinf(n) and n < 0:
- return GraphSONUtil.typed_value(cls.graphson_base_type, "-Infinity", "gx")
+ return GraphSONUtil.typed_value(cls.graphson_base_type, "-Infinity", "g")
else:
- return GraphSONUtil.typed_value(cls.graphson_base_type, str(n), "gx")
+ return GraphSONUtil.typed_value(cls.graphson_base_type, str(n), "g")
@classmethod
def objectify(cls, v, _):
@@ -518,13 +502,13 @@ class Int64IO(_NumberIO):
if isinstance(n, bool):
return n
elif n < -9223372036854775808 or n > 9223372036854775807:
- return GraphSONUtil.typed_value("BigInteger", str(n), "gx")
+ return GraphSONUtil.typed_value("BigInteger", str(n), "g")
else:
return GraphSONUtil.typed_value(cls.graphson_base_type, n)
class BigIntegerIO(Int64IO):
- graphson_type = "gx:BigInteger"
+ graphson_type = "g:BigInteger"
class Int32IO(Int64IO):
@@ -538,7 +522,7 @@ class Int32IO(Int64IO):
if isinstance(n, bool):
return n
elif n < -9223372036854775808 or n > 9223372036854775807:
- return GraphSONUtil.typed_value("BigInteger", str(n), "gx")
+ return GraphSONUtil.typed_value("BigInteger", str(n), "g")
elif n < -2147483648 or n > 2147483647:
return GraphSONUtil.typed_value("Int64", n)
else:
@@ -547,7 +531,7 @@ class Int32IO(Int64IO):
class Int16IO(Int64IO):
python_type = ShortType
- graphson_type = "gx:Int16"
+ graphson_type = "g:Int16"
graphson_base_type = "Int16"
@classmethod
@@ -556,13 +540,13 @@ class Int16IO(Int64IO):
if isinstance(n, bool):
return n
elif n < -9223372036854775808 or n > 9223372036854775807:
- return GraphSONUtil.typed_value("BigInteger", str(n), "gx")
+ return GraphSONUtil.typed_value("BigInteger", str(n), "g")
elif n < -2147483648 or n > 2147483647:
return GraphSONUtil.typed_value("Int64", n)
elif n < -32768 or n > 32767:
return GraphSONUtil.typed_value("Int32", n)
else:
- return GraphSONUtil.typed_value(cls.graphson_base_type, n, "gx")
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n, "g")
@classmethod
def objectify(cls, v, _):
@@ -571,42 +555,44 @@ class Int16IO(Int64IO):
class ByteIO(_NumberIO):
python_type = SingleByte
- graphson_type = "gx:Byte"
+ graphson_type = "g:Byte"
graphson_base_type = "Byte"
@classmethod
def dictify(cls, n, writer):
if isinstance(n, bool): # because isinstance(False, int) and isinstance(True, int)
return n
- return GraphSONUtil.typed_value(cls.graphson_base_type, n, "gx")
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n, "g")
@classmethod
def objectify(cls, v, _):
return int.__new__(SingleByte, v)
-class ByteBufferIO(_GraphSONTypeIO):
- python_type = ByteBufferType
- graphson_type = "gx:ByteBuffer"
- graphson_base_type = "ByteBuffer"
+class BinaryIO(_GraphSONTypeIO):
+ python_type = bytes
+ graphson_type = "g:Binary"
+ graphson_base_type = "Binary"
@classmethod
def dictify(cls, n, writer):
- return GraphSONUtil.typed_value(cls.graphson_base_type, "".join(chr(x) for x in n), "gx")
+ # writes a JSON String containing base64-encoded bytes
+ n_encoded = "".join(chr(x) for x in base64.b64encode(n))
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n_encoded, "g")
@classmethod
def objectify(cls, v, _):
- return cls.python_type(v, "utf8")
+ return base64.b64decode(v)
class CharIO(_GraphSONTypeIO):
python_type = SingleChar
- graphson_type = "gx:Char"
+ graphson_type = "g:Char"
graphson_base_type = "Char"
@classmethod
def dictify(cls, n, writer):
- return GraphSONUtil.typed_value(cls.graphson_base_type, n, "gx")
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n, "g")
@classmethod
def objectify(cls, v, _):
@@ -615,12 +601,13 @@ class CharIO(_GraphSONTypeIO):
class DurationIO(_GraphSONTypeIO):
python_type = timedelta
- graphson_type = "gx:Duration"
+ graphson_type = "g:Duration"
graphson_base_type = "Duration"
@classmethod
def dictify(cls, n, writer):
- return GraphSONUtil.typed_value(cls.graphson_base_type, duration_isoformat(n), "gx")
+ n_iso = duration_isoformat(n)
+ return GraphSONUtil.typed_value(cls.graphson_base_type, n_iso[:-2] + 'T0S' if n_iso.endswith('0D') else n_iso, "g")
@classmethod
def objectify(cls, v, _):
@@ -637,7 +624,8 @@ class VertexDeserializer(_GraphSONTypeIO):
properties = reader.to_object(d["properties"])
if properties is not None:
properties = [item for sublist in properties.values() for item in sublist]
- return Vertex(reader.to_object(d["id"]), d.get("label", "vertex")[0], properties)
+ label = d.get("label", "vertex")
+ return Vertex(reader.to_object(d["id"]), label if isinstance(label, str) else label[0], properties)
class EdgeDeserializer(_GraphSONTypeIO):
@@ -649,11 +637,12 @@ class EdgeDeserializer(_GraphSONTypeIO):
if "properties" in d:
properties = reader.to_object(d["properties"])
if properties is not None:
- properties = list(properties.values())
+ properties = [item for sublist in properties.values() for item in sublist]
+
return Edge(reader.to_object(d["id"]),
- Vertex(reader.to_object(d["outV"]), d.get("outVLabel", "vertex")[0]),
+ Vertex(reader.to_object(d["outV"]).get("id"), reader.to_object(d["outV"]).get("label")[0]),
d.get("label", "edge")[0],
- Vertex(reader.to_object(d["inV"]), d.get("inVLabel", "vertex")[0]),
+ Vertex(reader.to_object(d["inV"]).get("id"), reader.to_object(d["outV"]).get("label")[0]),
properties)
diff --git a/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py b/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
index 1382a0adb6..95004076c4 100644
--- a/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
+++ b/gremlin-python/src/main/python/tests/driver/test_driver_remote_connection.py
@@ -17,7 +17,6 @@
# under the License.
#
import os
-from datetime import datetime
from gremlin_python import statics
from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection
@@ -46,6 +45,18 @@ class TestDriverRemoteConnection(object):
g = traversal().with_(remote_conn)
assert long(6) == g.V().count().to_list()[0]
# #
+ assert Vertex(1) == g.V(1).next()
+ assert Vertex(1) == g.V(Vertex(1)).next()
+ assert 1 == g.V(1).id_().next()
+ # assert Traverser(Vertex(1)) == g.V(1).nextTraverser() # TODO check back after bulking added back
+ assert 1 == len(g.V(1).to_list())
+ assert isinstance(g.V(1).to_list(), list)
+ results = g.V().repeat(__.out()).times(2).name
+ results = results.to_list()
+ assert 2 == len(results)
+ assert "lop" in results
+ assert "ripple" in results
+ # #
assert 10 == g.V().repeat(__.both()).times(5)[0:10].count().next()
assert 1 == g.V().repeat(__.both()).times(5)[0:1].count().next()
assert 0 == g.V().repeat(__.both()).times(5)[0:0].count().next()
@@ -57,6 +68,34 @@ class TestDriverRemoteConnection(object):
assert 2 == len(results)
assert 'josh' in results
assert 'peter' in results
+ # #
+ results = g.V().has('name', 'peter').as_('a').out('created').as_('b').select('a', 'b').by(
+ __.value_map()).to_list()
+ assert 1 == len(results)
+ assert 'peter' == results[0]['a']['name'][0]
+ assert 35 == results[0]['a']['age'][0]
+ assert 'lop' == results[0]['b']['name'][0]
+ assert 'java' == results[0]['b']['lang'][0]
+ assert 2 == len(results[0]['a'])
+ assert 2 == len(results[0]['b'])
+ # #
+ results = g.V(1).inject(g.V(2).next()).values('name').to_list()
+ assert 2 == len(results)
+ assert 'marko' in results
+ assert 'vadas' in results
+ # #
+ # this test just validates that the underscored versions of steps conflicting with Gremlin work
+ # properly and can be removed when the old steps are removed - TINKERPOP-2272
+ results = g.V().filter_(__.values('age').sum_().and_(
+ __.max_().is_(P.gt(0)), __.min_().is_(P.gt(0)))).range_(0, 1).id_().next()
+ assert 1 == results
+ # #
+ # test dict keys
+ # types for dict
+ results = g.V().has('person', 'name', 'marko').element_map("name").group_count().next()
+ assert {HashableDict.of({T.id: 1, T.label: 'person', 'name': 'marko'}): 1} == results
+ results = g.V().has('person', 'name', 'marko').both('knows').group_count().by(__.values('name').fold()).next()
+ assert {tuple(['vadas']): 1, tuple(['josh']): 1} == results
def test_bulked_request_option(self, remote_connection):
g = traversal().with_(remote_connection)
diff --git a/gremlin-python/src/main/python/tests/structure/io/model.py b/gremlin-python/src/main/python/tests/structure/io/model.py
index e0a33c433c..45f9ec0d39 100644
--- a/gremlin-python/src/main/python/tests/structure/io/model.py
+++ b/gremlin-python/src/main/python/tests/structure/io/model.py
@@ -19,7 +19,7 @@ under the License.
import datetime
import uuid
-from gremlin_python.statics import short, long, bigint, BigDecimal, SingleByte, ByteBufferType
+from gremlin_python.statics import short, long, bigint, BigDecimal, SingleByte
from gremlin_python.structure.graph import Vertex, Edge, Property, VertexProperty, Path
from gremlin_python.process.traversal import Direction, Traverser, T
@@ -48,8 +48,8 @@ model["pos-biginteger"] = bigint(123456789987654321123456789987654321)
model["neg-biginteger"] = bigint(-123456789987654321123456789987654321)
model["min-byte"] = SingleByte(-128)
model["max-byte"] = SingleByte(127)
-model["empty-binary"] = ByteBufferType("", "utf8")
-model["str-binary"] = ByteBufferType("some bytes for you", "utf8")
+model["empty-binary"] = bytes("", "utf8")
+model["str-binary"] = bytes("some bytes for you", "utf8")
model["max-double"] = 1.7976931348623157E308
model["min-double"] = 4.9E-324
model["neg-max-double"] = -1.7976931348623157E308
diff --git a/gremlin-python/src/main/python/tests/structure/io/test_graphbinaryV4.py b/gremlin-python/src/main/python/tests/structure/io/test_graphbinaryV4.py
index ae878f3638..f0c1935514 100644
--- a/gremlin-python/src/main/python/tests/structure/io/test_graphbinaryV4.py
+++ b/gremlin-python/src/main/python/tests/structure/io/test_graphbinaryV4.py
@@ -22,7 +22,7 @@ import math
from collections import OrderedDict
from datetime import datetime, timedelta, timezone
-from gremlin_python.statics import long, bigint, BigDecimal, SingleByte, SingleChar, ByteBufferType
+from gremlin_python.statics import long, bigint, BigDecimal, SingleByte, SingleChar
from gremlin_python.structure.graph import Vertex, Edge, Property, VertexProperty, Path
from gremlin_python.structure.io.graphbinaryV4 import GraphBinaryWriter, GraphBinaryReader
from gremlin_python.process.traversal import Direction
@@ -203,8 +203,8 @@ class TestGraphBinaryV4(object):
output = self.graphbinary_reader.read_object(self.graphbinary_writer.write_object(x))
assert x == output
- def test_bytebuffer(self):
- x = ByteBufferType("c29tZSBieXRlcyBmb3IgeW91", "utf8")
+ def test_binary(self):
+ x = bytes("some bytes for you", "utf8")
output = self.graphbinary_reader.read_object(self.graphbinary_writer.write_object(x))
assert x == output
diff --git a/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py b/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py
new file mode 100644
index 0000000000..942fb0f0ea
--- /dev/null
+++ b/gremlin-python/src/main/python/tests/structure/io/test_graphsonV4.py
@@ -0,0 +1,531 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+__author__ = 'Marko A. Rodriguez (http://markorodriguez.com)'
+
+import datetime
+import calendar
+import json
+import uuid
+import math
+from decimal import *
+
+from unittest.mock import Mock
+
+from gremlin_python.statics import *
+from gremlin_python.structure.graph import Vertex, Edge, Property, VertexProperty, Path
+from gremlin_python.structure.io.graphsonV4 import GraphSONWriter, GraphSONReader, GraphSONUtil
+import gremlin_python.structure.io.graphsonV4
+from gremlin_python.process.traversal import P, Merge, Barrier, Order, Operator, Direction
+from gremlin_python.process.strategies import SubgraphStrategy
+from gremlin_python.process.graph_traversal import __
+
+
+class TestGraphSONReader:
+ graphson_reader = GraphSONReader()
+
+ def test_collections(self):
+ x = self.graphson_reader.read_object(
+ json.dumps({"@type": "g:List", "@value": [{"@type": "g:Int32", "@value": 1},
+ {"@type": "g:Int32", "@value": 2},
+ "3"]}))
+ assert isinstance(x, list)
+ assert x[0] == 1
+ assert x[1] == 2
+ assert x[2] == "3"
+ ##
+
+ x = self.graphson_reader.read_object(
+ json.dumps({"@type": "g:Set", "@value": [{"@type": "g:Int32", "@value": 1},
+ {"@type": "g:Int32", "@value": 2},
+ "3"]}))
+ # return a set as normal
+ assert isinstance(x, set)
+ assert x == {1, 2, "3"}
+
+ x = self.graphson_reader.read_object(
+ json.dumps({"@type": "g:Set", "@value": [{"@type": "g:Int32", "@value": 1},
+ {"@type": "g:Int32", "@value": 2},
+ {"@type": "g:Float", "@value": 2.0},
+ "3"]}))
+ # coerce to list here because Java might return numerics of different types which python won't recognize
+ # see comments of TINKERPOP-1844 for more details
+ assert isinstance(x, list)
+ assert x == list([1, 2, 2.0, "3"])
+ ##
+ x = self.graphson_reader.read_object(
+ json.dumps({"@type": "g:Map",
+ "@value": ['a', {"@type": "g:Int32", "@value": 1}, 'b', "marko"]}))
+ assert isinstance(x, dict)
+ assert x['a'] == 1
+ assert x['b'] == "marko"
+ assert len(x) == 2
+
+ # BulkSet gets coerced to a List - both have the same behavior
+ x = self.graphson_reader.read_object(
+ json.dumps({"@type": "g:BulkSet",
+ "@value": ["marko", {"@type": "g:Int64", "@value": 1}, "josh", {"@type": "g:Int64", "@value": 3}]}))
+ assert isinstance(x, list)
+ assert len(x) == 4
+ assert x.count("marko") == 1
+ assert x.count("josh") == 3
+
+ def test_number_input(self):
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:Byte",
+ "@value": 1
+ }))
+ assert isinstance(x, SingleByte)
+ assert 1 == x
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:Int16",
+ "@value": 16
+ }))
+ assert isinstance(x, short)
+ assert 16 == x
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:Int32",
+ "@value": 31
+ }))
+ assert isinstance(x, int)
+ assert 31 == x
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:Int64",
+ "@value": 31
+ }))
+ assert isinstance(x, long)
+ assert long(31) == x
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:Float",
+ "@value": 31.3
+ }))
+ assert isinstance(x, float)
+ assert 31.3 == x
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:Double",
+ "@value": 31.2
+ }))
+ assert isinstance(x, float)
+ assert 31.2 == x
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:Double",
+ "@value": "NaN"
+ }))
+ assert isinstance(x, float)
+ assert math.isnan(x)
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:Double",
+ "@value": "Infinity"
+ }))
+ assert isinstance(x, float)
+ assert math.isinf(x) and x > 0
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:Double",
+ "@value": "-Infinity"
+ }))
+ assert isinstance(x, float)
+ assert math.isinf(x) and x < 0
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:BigDecimal",
+ "@value": 31.2
+ }))
+ assert isinstance(x, Decimal)
+ assert Decimal(31.2) == x
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:BigDecimal",
+ "@value": 123456789987654321123456789987654321
+ }))
+ assert isinstance(x, Decimal)
+ assert Decimal('123456789987654321123456789987654321') == x
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:BigDecimal",
+ "@value": "NaN"
+ }))
+ assert isinstance(x, Decimal)
+ assert math.isnan(x)
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:BigDecimal",
+ "@value": "Infinity"
+ }))
+ assert isinstance(x, Decimal)
+ assert math.isinf(x) and x > 0
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:BigDecimal",
+ "@value": "-Infinity"
+ }))
+ assert isinstance(x, Decimal)
+ assert math.isinf(x) and x < 0
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:BigInteger",
+ "@value": 31
+ }))
+ assert isinstance(x, long)
+ assert 31 == x
+ ##
+ x = self.graphson_reader.read_object(json.dumps({
+ "@type": "g:BigInteger",
+ "@value": 123456789987654321123456789987654321
+ }))
+ assert isinstance(x, long)
+ assert 123456789987654321123456789987654321 == x
+
+ def test_graph(self):
+ vertex = self.graphson_reader.read_object("""
+ {"@type":"g:Vertex", "@value":{"id":{"@type":"g:Int32","@value":1},"label": ["person"],"outE":{"created":[{"id":{"@type":"g:Int32","@value":9},"inV":{"@type":"g:Int32","@value":3},"properties":{"weight":[{"@type":"g:Property","@value":{"key":"weight","value":{"@type":"g:Double","@value":0.4}}}]}}],"knows":[{"id":{"@type":"g:Int32","@value":7},"inV":{"@type":"g:Int32","@value":2},"properties":{"weight":[{"@type":"g:Property","@value":{"key":"weight","value":{"@type":"g:Double","@v [...]
+ assert isinstance(vertex, Vertex)
+ assert "person" == vertex.label
+ assert 1 == vertex.id
+ assert isinstance(vertex.id, int)
+ assert vertex == Vertex(1)
+ ##
+ vertex = self.graphson_reader.read_object("""
+ {"@type":"g:Vertex", "@value":{"id":{"@type":"g:Float","@value":45.23}}}""")
+ assert isinstance(vertex, Vertex)
+ assert 45.23 == vertex.id
+ assert isinstance(vertex.id, FloatType)
+ assert "vertex" == vertex.label
+ assert vertex == Vertex(45.23)
+ ##
+ vertex_property = self.graphson_reader.read_object("""
+ {"@type":"g:VertexProperty", "@value":{"id":{"@type":"g:Int32","@value":1},"label":["name"],"value":"marko"}}""")
+ assert isinstance(vertex_property, VertexProperty)
+ assert 1 == vertex_property.id
+ assert "name" == vertex_property.label
+ assert "marko" == vertex_property.value
+ assert vertex_property.vertex is None
+ ##
+ edge = self.graphson_reader.read_object("""
+ {"@type":"g:Edge", "@value":{"id":{"@type":"g:Int64","@value":17},"label":["knows"],"inV":{"id":"x","label":["xLabel"]},"outV":{"id":"y","label":["vertex"]},"properties":{"aKey":[{"@type":"g:Property","@value":{"key":"aKey","value":"aValue"}}],"bKey":[{"@type":"g:Property","@value":{"key":"bKey","value":true}}]}}}""")
+ # print edge
+ assert isinstance(edge, Edge)
+ assert 17 == edge.id
+ assert "knows" == edge.label
+ assert edge.inV == Vertex("x", "xLabel")
+ assert edge.outV == Vertex("y", "vertex")
+ ##
+ property = self.graphson_reader.read_object("""
+ {"@type":"g:Property", "@value":{"key":"aKey","value":{"@type":"g:Int64","@value":17}}}""")
+ # print property
+ assert isinstance(property, Property)
+ assert "aKey" == property.key
+ assert 17 == property.value
+
+ def test_path(self):
+ path = self.graphson_reader.read_object(
+ """{"@type":"g:Path","@value":{"labels":{"@type":"g:List","@value":[{"@type":"g:Set","@value":["a"]},{"@type":"g:Set","@value":["b","c"]},{"@type":"g:Set","@value":[]}]},"objects":{"@type":"g:List","@value":[{"@type":"g:Vertex","@value":{"id":{"@type":"g:Int32","@value":1},"label":["person"],"properties":{"name":[{"@type":"g:VertexProperty","@value":{"id":{"@type":"g:Int64","@value":0},"value":"marko","label":["name"]}}],"age":[{"@type":"g:VertexProperty","@value":{"id":{"@ty [...]
+ )
+ assert isinstance(path, Path)
+ assert "path[v[1], v[3], lop]" == str(path)
+ assert Vertex(1) == path[0]
+ assert Vertex(1) == path["a"]
+ assert "lop" == path[2]
+ assert 3 == len(path)
+
+ def test_custom_mapping(self):
+
+ # extended mapping
+ class X:
+ pass
+
+ type_string = "test:Xtype"
+ override_string = "g:Int64"
+ serdes = Mock()
+
+ reader = GraphSONReader(deserializer_map={type_string: serdes})
+ assert type_string in reader.deserializers
+
+ # base dicts are not modified
+ assert type_string not in gremlin_python.structure.io.graphsonV4._deserializers
+
+ x = X()
+ o = reader.to_object({GraphSONUtil.TYPE_KEY: type_string, GraphSONUtil.VALUE_KEY: x})
+ serdes.objectify.assert_called_once_with(x, reader)
+ assert o is serdes.objectify()
+
+ # overridden mapping
+ type_string = "g:Int64"
+ serdes = Mock()
+ reader = GraphSONReader(deserializer_map={type_string: serdes, override_string: serdes})
+ assert gremlin_python.structure.io.graphsonV4._deserializers[type_string] is not reader.deserializers[
+ type_string]
+
+ value = 3
+ o = reader.to_object({GraphSONUtil.TYPE_KEY: type_string, GraphSONUtil.VALUE_KEY: value})
+ serdes.objectify.assert_called_once_with(value, reader)
+ assert o is serdes.objectify()
+
+ def test_datetime(self):
+ tz = datetime.timezone(datetime.timedelta(seconds=36000))
+ ms = 12345678912
+ expected = datetime.datetime(2022, 5, 20, tzinfo=tz) + datetime.timedelta(microseconds=ms)
+ dt = self.graphson_reader.read_object(json.dumps({"@type": "g:DateTime", "@value": expected.isoformat()}))
+ assert isinstance(dt, datetime.datetime)
+ assert dt == expected
+
+ def test_datetime_zulu(self):
+ tz = datetime.timezone.utc
+ ms = 12345678912
+ expected = datetime.datetime(2022, 5, 20, tzinfo=tz) + datetime.timedelta(microseconds=ms)
+ # simulate zulu format
+ expected_zulu = expected.isoformat()[:-6] + 'Z'
+ dt = self.graphson_reader.read_object(json.dumps({"@type": "g:DateTime", "@value": expected_zulu}))
+ assert isinstance(dt, datetime.datetime)
+ assert dt == expected
+
+ def test_duration(self):
+ d = self.graphson_reader.read_object(json.dumps({"@type": "g:Duration", "@value": "PT120H"}))
+ assert isinstance(d, datetime.timedelta)
+ assert d == datetime.timedelta(hours=120)
+
+ def test_uuid(self):
+ prop = self.graphson_reader.read_object(
+ json.dumps({'@type': 'g:UUID', '@value': "41d2e28a-20a4-4ab0-b379-d810dede3786"}))
+ assert isinstance(prop, uuid.UUID)
+ assert str(prop) == '41d2e28a-20a4-4ab0-b379-d810dede3786'
+
+ def test_metrics(self):
+ prop = self.graphson_reader.read_object(
+ json.dumps([{'@type': 'g:TraversalMetrics', '@value': {'dur': 1.468594, 'metrics': [
+ {'@type': 'g:Metrics', '@value': {'dur': 1.380957, 'counts': {}, 'name': 'GraphStep(__.V())', 'annotations': {'percentDur': 94.03259171697556}, 'id': '4.0.0()'}},
+ {'@type': 'g:Metrics', '@value': {'dur': 0.087637, 'counts': {}, 'name': 'ReferenceElementStep', 'annotations': {'percentDur': 5.967408283024444}, 'id': '3.0.0()'}}
+ ]}}]))
+ assert isinstance(prop, list)
+ assert prop == [{'dur': 1.468594, 'metrics': [
+ {'dur': 1.380957, 'counts': {}, 'name': 'GraphStep(__.V())', 'annotations': {'percentDur': 94.03259171697556}, 'id': '4.0.0()'},
+ {'dur': 0.087637, 'counts': {}, 'name': 'ReferenceElementStep', 'annotations': {'percentDur': 5.967408283024444}, 'id': '3.0.0()'}
+ ]}]
+
+ def test_binary(self):
+ bb = self.graphson_reader.read_object(
+ json.dumps({"@type": "g:Binary", "@value": "c29tZSBieXRlcyBmb3IgeW91"}))
+ assert isinstance(bb, bytes)
+ assert bytes("some bytes for you", "utf8") == bb
+
+ def test_char(self):
+ c = self.graphson_reader.read_object(json.dumps({"@type": "g:Char", "@value": "L"}))
+ assert isinstance(c, SingleChar)
+ assert chr(76) == c
+
+ def test_null(self):
+ c = self.graphson_reader.read_object(json.dumps(None))
+ assert c is None
+
+
+class TestGraphSONWriter:
+ graphson_writer = GraphSONWriter()
+ graphson_reader = GraphSONReader()
+
+ def test_collections(self):
+ assert {"@type": "g:List", "@value": [{"@type": "g:Int32", "@value": 1},
+ {"@type": "g:Int32", "@value": 2},
+ {"@type": "g:Int32", "@value": 3}]} == json.loads(
+ self.graphson_writer.write_object([1, 2, 3]))
+ assert {"@type": "g:Set", "@value": [{"@type": "g:Int32", "@value": 1},
+ {"@type": "g:Int32", "@value": 2},
+ {"@type": "g:Int32", "@value": 3}]} == json.loads(
+ self.graphson_writer.write_object({1, 2, 3, 3}))
+ assert {"@type": "g:Map",
+ "@value": ['a', {"@type": "g:Int32", "@value": 1}]} == json.loads(
+ self.graphson_writer.write_object({'a': 1}))
+
+ def test_numbers(self):
+ assert {"@type": "g:Byte", "@value": 1} == json.loads(self.graphson_writer.write_object(int.__new__(SingleByte, 1)))
+ assert {"@type": "g:Int16", "@value": 16} == json.loads(self.graphson_writer.write_object(short(16)))
+ assert {"@type": "g:Int16", "@value": -16} == json.loads(self.graphson_writer.write_object(short(-16)))
+ assert {"@type": "g:Int64", "@value": 2} == json.loads(self.graphson_writer.write_object(long(2)))
+ assert {"@type": "g:Int64", "@value": 851401972585122} == json.loads(self.graphson_writer.write_object(long(851401972585122)))
+ assert {"@type": "g:Int64", "@value": -2} == json.loads(self.graphson_writer.write_object(long(-2)))
+ assert {"@type": "g:Int64", "@value": -851401972585122} == json.loads(self.graphson_writer.write_object(long(-851401972585122)))
+ assert {"@type": "g:Int32", "@value": 1} == json.loads(self.graphson_writer.write_object(1))
+ assert {"@type": "g:Int32", "@value": -1} == json.loads(self.graphson_writer.write_object(-1))
+ assert {"@type": "g:Int64", "@value": 851401972585122} == json.loads(self.graphson_writer.write_object(851401972585122))
+ assert {"@type": "g:Double", "@value": 3.2} == json.loads(self.graphson_writer.write_object(3.2))
+ assert {"@type": "g:Double", "@value": "NaN"} == json.loads(self.graphson_writer.write_object(float('nan')))
+ assert {"@type": "g:Double", "@value": "Infinity"} == json.loads(self.graphson_writer.write_object(float('inf')))
+ assert {"@type": "g:Double", "@value": "-Infinity"} == json.loads(self.graphson_writer.write_object(float('-inf')))
+ assert {"@type": "g:BigDecimal", "@value": "123456789987654321123456789987654321"} == json.loads(self.graphson_writer.write_object(Decimal('123456789987654321123456789987654321')))
+ assert {"@type": "g:BigDecimal", "@value": "NaN"} == json.loads(self.graphson_writer.write_object(Decimal('nan')))
+ assert {"@type": "g:BigDecimal", "@value": "Infinity"} == json.loads(self.graphson_writer.write_object(Decimal('inf')))
+ assert {"@type": "g:BigDecimal", "@value": "-Infinity"} == json.loads(self.graphson_writer.write_object(Decimal('-inf')))
+ assert {"@type": "g:BigInteger", "@value": "123456789987654321123456789987654321"} == json.loads(self.graphson_writer.write_object(long(123456789987654321123456789987654321)))
+ assert {"@type": "g:BigInteger", "@value": "123456789987654321123456789987654321"} == json.loads(self.graphson_writer.write_object(123456789987654321123456789987654321))
+ assert """true""" == self.graphson_writer.write_object(True)
+
+ def test_enum(self):
+ assert {"@type": "g:Merge", "@value": "onMatch"} == json.loads(self.graphson_writer.write_object(Merge.on_match))
+ assert {"@type": "g:Order", "@value": "shuffle"} == json.loads(self.graphson_writer.write_object(Order.shuffle))
+ assert {"@type": "g:Barrier", "@value": "normSack"} == json.loads(self.graphson_writer.write_object(Barrier.norm_sack))
+ assert {"@type": "g:Operator", "@value": "sum"} == json.loads(self.graphson_writer.write_object(Operator.sum_))
+ assert {"@type": "g:Operator", "@value": "sumLong"} == json.loads(self.graphson_writer.write_object(Operator.sum_long))
+ assert {"@type": "g:Direction", "@value": "OUT"} == json.loads(self.graphson_writer.write_object(Direction.OUT))
+ assert {"@type": "g:Direction", "@value": "OUT"} == json.loads(self.graphson_writer.write_object(Direction.from_))
+
+ def test_P(self):
+ result = {'@type': 'g:P',
+ '@value': {
+ 'predicate': 'and',
+ 'value': [{
+ '@type': 'g:P',
+ '@value': {
+ 'predicate': 'or',
+ 'value': [{
+ '@type': 'g:P',
+ '@value': {'predicate': 'lt', 'value': 'b'}
+ },
+ {'@type': 'g:P', '@value': {'predicate': 'gt', 'value': 'c'}}
+ ]
+ }
+ },
+ {'@type': 'g:P', '@value': {'predicate': 'neq', 'value': 'd'}}]}}
+
+ assert result == json.loads(
+ self.graphson_writer.write_object(P.lt("b").or_(P.gt("c")).and_(P.neq("d"))))
+
+ result = {'@type': 'g:P', '@value': {'predicate': 'within', 'value': {'@type': 'g:List', '@value': [
+ {"@type": "g:Int32", "@value": 1}, {"@type": "g:Int32", "@value": 2}]}}}
+ assert result == json.loads(self.graphson_writer.write_object(P.within([1, 2])))
+ assert result == json.loads(self.graphson_writer.write_object(P.within({1, 2})))
+ assert result == json.loads(self.graphson_writer.write_object(P.within(1, 2)))
+
+ result = {'@type': 'g:P', '@value': {'predicate': 'within', 'value': {'@type': 'g:List', '@value': [
+ {"@type": "g:Int32", "@value": 1}]}}}
+ assert result == json.loads(self.graphson_writer.write_object(P.within([1])))
+ assert result == json.loads(self.graphson_writer.write_object(P.within({1})))
+ assert result == json.loads(self.graphson_writer.write_object(P.within(1)))
+
+ def test_graph(self):
+ assert {"@type": "g:Vertex",
+ "@value": {"id": {"@type": "g:Int64", "@value": 12}, "label": ["person"]}} == json.loads(
+ self.graphson_writer.write_object(Vertex(long(12), "person")))
+
+ assert {'@type': 'g:Edge', '@value': {'id': {'@type': 'g:Int32', '@value': 7},
+ 'inV': {'id': {'@type': 'g:Int32', '@value': 1},
+ 'label': ['dog']},
+ 'label': ['knows'],
+ 'outV': {'id': {'@type': 'g:Int32', '@value': 0},
+ 'label': ['person']}}} == json.loads(
+ self.graphson_writer.write_object(Edge(7, Vertex(0, "person"), "knows", Vertex(1, "dog"))))
+ assert {"@type": "g:VertexProperty", "@value": {"id": "blah", "label": ["keyA"], "value": True}} == json.loads(
+ self.graphson_writer.write_object(VertexProperty("blah", "keyA", True, Vertex("stephen"))))
+
+ assert {"@type": "g:Property",
+ "@value": {"key": "name", "value": "marko"}} == json.loads(
+ self.graphson_writer.write_object(
+ Property("name", "marko", VertexProperty(1234, "aKey", 21345, Vertex("vertexId")))))
+
+ vertex = self.graphson_reader.read_object(self.graphson_writer.write_object(Vertex(1, "person")))
+ assert 1 == vertex.id
+ assert "person" == vertex.label
+
+ edge = self.graphson_reader.read_object(
+ self.graphson_writer.write_object(Edge(3, Vertex(1, "person"), "knows", Vertex(2, "dog"))))
+ assert "knows" == edge.label
+ assert 3 == edge.id
+ assert 1 == edge.outV.id
+ assert 2 == edge.inV.id
+
+ vertex_property = self.graphson_reader.read_object(
+ self.graphson_writer.write_object(VertexProperty(1, "age", 32, Vertex(1))))
+ assert 1 == vertex_property.id
+ assert "age" == vertex_property.key
+ assert 32 == vertex_property.value
+
+ property = self.graphson_reader.read_object(self.graphson_writer.write_object(Property("age", 32.2, Edge(1, Vertex(2), "knows", Vertex(3)))))
+ assert "age" == property.key
+ assert 32.2 == property.value
+
+ def test_custom_mapping(self):
+ # extended mapping
+ class X:
+ pass
+
+ serdes = Mock()
+ writer = GraphSONWriter(serializer_map={X: serdes})
+ assert X in writer.serializers
+
+ # base dicts are not modified
+ assert X not in gremlin_python.structure.io.graphsonV4._serializers
+
+ obj = X()
+ d = writer.to_dict(obj)
+ serdes.dictify.assert_called_once_with(obj, writer)
+ assert d is serdes.dictify()
+
+ # overridden mapping
+ serdes = Mock()
+ writer = GraphSONWriter(serializer_map={int: serdes})
+ assert gremlin_python.structure.io.graphsonV4._serializers[int] is not writer.serializers[int]
+
+ value = 3
+ d = writer.to_dict(value)
+ serdes.dictify.assert_called_once_with(value, writer)
+ assert d is serdes.dictify()
+
+ def test_write_long(self):
+ mapping = self.graphson_writer.to_dict(1)
+ assert mapping['@type'] == 'g:Int32'
+ assert mapping['@value'] == 1
+
+ mapping = self.graphson_writer.to_dict(long(1))
+ assert mapping['@type'] == 'g:Int64'
+ assert mapping['@value'] == 1
+
+ def test_datetime(self):
+ expected = json.dumps({"@type": "g:DateTime", "@value": "2022-05-20T03:25:45.678912+10:00"}, separators=(',', ':'))
+ tz = datetime.timezone(datetime.timedelta(seconds=36000))
+ ms = 12345678912
+ dt = datetime.datetime(2022, 5, 20, tzinfo=tz) + datetime.timedelta(microseconds=ms)
+ output = self.graphson_writer.write_object(dt)
+ assert expected == output
+
+ def test_duration(self):
+ expected = json.dumps({"@type": "g:Duration", "@value": "P5D"}, separators=(',', ':'))
+ d = datetime.timedelta(hours=120)
+ output = self.graphson_writer.write_object(d)
+ assert expected == output
+
+ def test_uuid(self):
+ expected = json.dumps({'@type': 'g:UUID', '@value': "41d2e28a-20a4-4ab0-b379-d810dede3786"}, separators=(',', ':'))
+ prop = uuid.UUID("41d2e28a-20a4-4ab0-b379-d810dede3786")
+ output = self.graphson_writer.write_object(prop)
+ assert expected == output
+
+ def test_binary(self):
+ expected = json.dumps({'@type': 'g:Binary', '@value': 'c29tZSBieXRlcyBmb3IgeW91'}, separators=(',', ':'))
+ bb = bytes("some bytes for you", "utf8")
+ output = self.graphson_writer.write_object(bb)
+ assert expected == output
+
+ def test_char(self):
+ expected = json.dumps({'@type': 'g:Char', '@value': 'L'}, separators=(',', ':'))
+ c = str.__new__(SingleChar, chr(76))
+ output = self.graphson_writer.write_object(c)
+ assert expected == output
\ No newline at end of file
diff --git a/gremlin-python/src/main/python/tests/structure/io/test_graphsonv4model.py b/gremlin-python/src/main/python/tests/structure/io/test_graphsonv4model.py
new file mode 100644
index 0000000000..683ff030a5
--- /dev/null
+++ b/gremlin-python/src/main/python/tests/structure/io/test_graphsonv4model.py
@@ -0,0 +1,308 @@
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+"""
+import pytest
+
+"""
+Round trip testing of GraphSONV4 compared to a correct "model".
+
+Set the IO_TEST_DIRECTORY environment variable to the directory where
+the .gbin files that represent the serialized "model" are located.
+"""
+
+from gremlin_python.structure.io.graphsonV4 import GraphSONWriter, GraphSONReader
+import math
+import json
+import os
+
+from .model import model
+
+gremlin_test_dir = "gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/"
+directory_search_pattern = "gremlin-python" + os.sep + "src" + os.sep + "main"
+default_dir = __file__[:__file__.find(directory_search_pattern)]
+test_resource_directory = os.environ.get('IO_TEST_DIRECTORY_GRAPHSON', default_dir + gremlin_test_dir)
+writer = GraphSONWriter()
+reader = GraphSONReader()
+
+def get_entry(title):
+ return model[title]
+
+def read_file_by_name(resource_name):
+ full_name = test_resource_directory + resource_name + "-" + "v4" + ".json"
+ with open(full_name, 'r') as resource_file:
+ return json.dumps(json.load(resource_file), separators=(',', ':'))
+
+@pytest.mark.skip(reason="enable after big decimal implementation/serialization is unified in python (")
+def test_pos_bigdecimal():
+ def decimal_cmp(x, y):
+ if x.scale == y.scale and x.unscaled_value == y.unscaled_value:
+ return True
+ else:
+ return False
+
+ # gremlin-python adds an extra 0 byte to the value.
+ run_writeread("pos-bigdecimal", decimal_cmp)
+
+@pytest.mark.skip(reason="enable after big decimal implementation/serialization is unified in python")
+def test_neg_bigdecimal():
+ def decimal_cmp(x, y):
+ if x.scale == y.scale and x.unscaled_value == y.unscaled_value:
+ return True
+ else:
+ return False
+
+ # gremlin-python adds an extra 0 byte to the value.
+ run_writeread("neg-bigdecimal", decimal_cmp())
+
+def test_pos_biginteger():
+ # gremlin-python adds an extra 0 byte to the value.
+ run_writeread("pos-biginteger")
+
+def test_neg_biginteger():
+ # gremlin-python adds an extra 0 byte to the value.
+ run_writeread("neg-biginteger")
+
+def test_min_byte():
+ run("min-byte")
+
+def test_max_byte():
+ run("max-byte")
+
+def test_empty_binary():
+ run("empty-binary")
+
+def test_str_binary():
+ run("str-binary")
+
+def test_max_double():
+ run("max-double")
+
+def test_min_double():
+ run("min-double")
+
+def test_neg_max_double():
+ run("neg-max-double")
+
+def test_neg_min_double():
+ run("neg-min-double")
+
+def test_nan_double():
+ def nan_cmp(x, y):
+ if math.isnan(x) and math.isnan(y):
+ return True
+ else:
+ return False
+ run("nan-double", nan_cmp)
+
+def test_pos_inf_double():
+ run("pos-inf-double")
+
+def test_neg_inf_double():
+ run("neg-inf-double")
+
+def test_neg_zero_double():
+ run("neg-zero-double")
+
+def test_zero_duration():
+ run("zero-duration")
+
+def test_traversal_edge():
+ # properties aren't serialized in gremlin-python
+ run_writeread("traversal-edge")
+
+def test_no_prop_edge():
+ # gremlin-python serializes/deserializes "null" for props not empty list
+ run_writeread("no-prop-edge")
+
+def test_max_int():
+ run("max-int")
+
+def test_min_int():
+ run("min-int")
+
+def test_max_long():
+ # attempts to serialize a long as an int
+ run_writeread("max-long")
+
+def test_min_long():
+ # attempts to serialize a long as an int
+ run_writeread("min-long")
+
+def test_var_type_list():
+ run("var-type-list")
+
+def test_empty_list():
+ run("empty-list")
+
+def test_var_type_map():
+ # can't write tuple
+ run_read("var-type-map")
+
+def test_empty_map():
+ run("empty-map")
+
+# gremlin-python doesn't serialize path
+def test_traversal_path():
+ run_read("traversal-path")
+
+def test_empty_path():
+ run_read("empty-path")
+
+def test_prop_path():
+ run_read("prop-path")
+
+def test_edge_property():
+ run("edge-property")
+
+def test_null_property():
+ run("null-property")
+
+def test_var_type_set():
+ # ordering within set might be different
+ run_writeread("var-type-set")
+
+def test_empty_set():
+ run("empty-set")
+
+def test_max_short():
+ # short deserialized to int
+ run_writeread("max-short")
+
+def test_min_short():
+ # short deserialized to int
+ run_writeread("min-short")
+
+def test_specified_uuid():
+ run("specified-uuid")
+
+def test_nil_uuid():
+ run("nil-uuid")
+
+def test_no_prop_vertex():
+ # gremlin-python serializes/deserializes "null" for props not empty list
+ run_writeread("no-prop-vertex")
+
+def test_traversal_vertexproperty():
+ # properties aren't serialized in gremlin-python
+ run_writeread("traversal-vertexproperty")
+
+def test_meta_vertexproperty():
+ # properties aren't serialized in gremlin-python
+ run_writeread("meta-vertexproperty")
+
+def test_set_cardinality_vertexproperty():
+ # properties aren't serialized in gremlin-python
+ run_writeread("set-cardinality-vertexproperty")
+
+def test_id_t():
+ run("id-t")
+
+def test_out_direction():
+ run("out-direction")
+
+def test_var_bulklist():
+ run_read("var-bulklist")
+
+def test_empty_bulklist():
+ run_read("empty-bulklist")
+
+def test_single_byte_char():
+ # char is serialized as string
+ run_writeread("single-byte-char")
+
+def test_multi_byte_char():
+ # char is serialized as string
+ run_writeread("multi-byte-char")
+
+def test_unspecified_null():
+ # no serializer for plain null
+ run_writeread("unspecified-null")
+
+def test_true_boolean():
+ run("true-boolean")
+
+def test_false_boolean():
+ run("false-boolean")
+
+def test_single_byte_string():
+ run("single-byte-string")
+
+def test_mixed_string():
+ run("mixed-string")
+
+def run(resource_name, comparator = None):
+ """
+ Runs the regular set of tests for the type which is
+ 1. model to deserialized object
+ 2. written bytes to read bytes
+ 3. round tripped (read then written) bytes to read bytes
+
+ Optionally takes in a comparator that used for comparison 1.
+ """
+ resource_json = read_file_by_name(resource_name)
+
+ model = get_entry(resource_name)
+ read = reader.read_object(resource_json)
+ if comparator is not None:
+ assert comparator(model, read)
+ else:
+ assert model == read
+
+ written = writer.write_object(model)
+ assert resource_json == written
+
+ round_tripped = writer.write_object(reader.read_object(resource_json))
+ assert resource_json == round_tripped
+
+def run_read(resource_name):
+ """
+ Runs the read test which compares the model to deserialized object
+
+ This should only be used in cases where there is only a deserializer
+ but no serializer for the same type.
+ """
+ resource_json = read_file_by_name(resource_name)
+
+ model = get_entry(resource_name)
+ read = reader.read_object(resource_json)
+ assert model == read
+
+def run_writeread(resource_name, comparator = None):
+ """
+ Runs a reduced set of tests for the type which is
+ 1. model to deserialized object
+ 2. model to round tripped (written then read) object
+
+ Optionally takes in a comparator that used for comparison 1.
+
+ Use this in cases where the regular run() function in too stringent.
+ E.g. when ordering doesn't matter like for sets. Ideally, a type
+ should be tested with run() instead when possible.
+ """
+ resource_json = read_file_by_name(resource_name)
+
+ model = get_entry(resource_name)
+ read = reader.read_object(resource_json)
+ round_tripped = reader.read_object(writer.write_object(model))
+
+ if comparator is not None:
+ assert comparator(model, read)
+ assert comparator(model, round_tripped)
+ else:
+ assert model == read
+ assert model == round_tripped
\ No newline at end of file
(tinkerpop) 04/17: Update GraphSONv4 RequestMessage documentation CTR
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 22e94ac224ba7332059d3a2612725e33c74f0ed2
Author: Ken Hu <10...@users.noreply.github.com>
AuthorDate: Mon Oct 21 10:19:12 2024 -0700
Update GraphSONv4 RequestMessage documentation CTR
Documentation showed incorrect format that had a fields object that
should have been flattened out.
---
docs/src/dev/io/graphson.asciidoc | 52 +++++++++++++++------------------------
1 file changed, 20 insertions(+), 32 deletions(-)
diff --git a/docs/src/dev/io/graphson.asciidoc b/docs/src/dev/io/graphson.asciidoc
index 60daf6b9a0..8d70cd2509 100644
--- a/docs/src/dev/io/graphson.asciidoc
+++ b/docs/src/dev/io/graphson.asciidoc
@@ -2695,33 +2695,23 @@ The following `RequestMessage` is an example of a simple sessionless request for
----
{
"gremlin": "g.V(x)",
- "fields": {
+ "materializeProperties": "tokens",
+ "timeoutMs": {
+ "@type": "g:Int64",
+ "@value": 1000
+ },
+ "g": "g",
+ "bindings": {
"@type": "g:Map",
"@value": [
- "materializeProperties",
- "tokens",
- "timeoutMs",
+ "x",
{
- "@type": "g:Int64",
- "@value": 1000
- },
- "g",
- "g",
- "bindings",
- {
- "@type": "g:Map",
- "@value": [
- "x",
- {
- "@type": "g:Int32",
- "@value": 1
- }
- ]
- },
- "language",
- "gremlin-groovy"
+ "@type": "g:Int32",
+ "@value": 1
+ }
]
- }
+ },
+ "language": "gremlin-groovy"
}
----
@@ -2729,15 +2719,13 @@ The following `RequestMessage` is an example of a simple sessionless request for
----
{
"gremlin": "g.V(x)",
- "fields": {
- "materializeProperties": "tokens",
- "timeoutMs": 1000,
- "g": "g",
- "bindings": {
- "x": 1
- },
- "language": "gremlin-groovy"
- }
+ "materializeProperties": "tokens",
+ "timeoutMs": 1000,
+ "g": "g",
+ "bindings": {
+ "x": 1
+ },
+ "language": "gremlin-groovy"
}
----
(tinkerpop) 13/17: Add signal that bytes were read in HttpGremlinResponseDecoder CTR
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 774ef0e0760c3cf5fa8c514f6c0528e2c5b5191e
Author: Ken Hu <10...@users.noreply.github.com>
AuthorDate: Thu Oct 31 16:13:53 2024 -0700
Add signal that bytes were read in HttpGremlinResponseDecoder CTR
---
.../tinkerpop/gremlin/driver/handler/HttpGremlinResponseDecoder.java | 3 +++
1 file changed, 3 insertions(+)
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinResponseDecoder.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinResponseDecoder.java
index 3bd1719342..0a512ae91e 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinResponseDecoder.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinResponseDecoder.java
@@ -56,6 +56,9 @@ public class HttpGremlinResponseDecoder extends MessageToMessageDecoder<FullHttp
@Override
protected void decode(ChannelHandlerContext ctx, FullHttpResponse msg, List<Object> out) throws Exception {
+ // A value must be set in order to signal to the InactiveChannelHandler that data has been received.
+ ctx.channel().attr(InactiveChannelHandler.BYTES_READ).set(0);
+
final ByteBuf content = msg.content();
ResponseMessage response;
(tinkerpop) 11/17: Changed connection pooling to reflect having switched from long lived web socket connections to short lived http connections: (#2860)
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 68c3bd02d676e9957994cbcb8569c3b12913d3df
Author: andreachild <an...@improving.com>
AuthorDate: Thu Oct 31 15:59:42 2024 -0700
Changed connection pooling to reflect having switched from long lived web socket connections to short lived http connections: (#2860)
-sped up obtaining a new connection by removing method-level synchronization of getAvailableConnection and introducing a separate set for available connections
-introduced cleanup of idle connections to release resources
-removed minSize since idle connections are now cleaned up and the pool can become empty if there is no traffic
-increased maxSize default to 128
-increased core pool size of ScheduledThreadPoolExecutor that is used for creating new connections to number of available processors
-added logging to help troubleshoot connection related integration tests failures
---
CHANGELOG.asciidoc | 2 +
docs/src/reference/gremlin-variants.asciidoc | 4 +-
.../tinkerpop/gremlin/driver/Channelizer.java | 26 ++-
.../apache/tinkerpop/gremlin/driver/Client.java | 2 +-
.../apache/tinkerpop/gremlin/driver/Cluster.java | 51 +++---
.../tinkerpop/gremlin/driver/Connection.java | 20 ++-
.../tinkerpop/gremlin/driver/ConnectionPool.java | 180 ++++++++-------------
.../apache/tinkerpop/gremlin/driver/Settings.java | 21 +--
.../driver/handler/HttpGremlinRequestEncoder.java | 2 +-
.../driver/handler/IdleConnectionHandler.java | 49 ++++++
...eckHandler.java => InactiveChannelHandler.java} | 13 +-
.../driver/util/ConfigurationEvaluator.java | 36 ++---
.../gremlin/driver/util/ProfilingApplication.java | 6 +-
.../gremlin/driver/ClusterBuilderTest.java | 6 +-
.../gremlin/driver/ConnectionPoolTest.java | 8 +-
.../tinkerpop/gremlin/driver/SettingsTest.java | 4 +-
.../driver/handler/IdleConnectionHandlerTest.java | 57 +++++++
.../driver/ClientConnectionIntegrateTest.java | 88 +++++++++-
.../gremlin/server/GremlinDriverIntegrateTest.java | 9 +-
19 files changed, 372 insertions(+), 212 deletions(-)
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index e049e260b6..5b791eedcf 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -80,6 +80,8 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima
* Updated `ElementIdStrategy.getConfiguration()` to help with serialization.
* Updated `TraversalStrategyProxy` to utilize strategy names instead of strategy classes
* Use `TraversalStrategyProxy` in Java, or `TraversalStrategy` in Python to pass custom strategies in traversals
+* Removed `minSize` setting for Gremlin Driver connection pool since connections are now short-lived HTTP connections
+* Added `idleConnectionTimeout` setting for Gremlin Driver and automatic closing of idle connections
== TinkerPop 3.7.0 (Gremfir Master of the Pan Flute)
diff --git a/docs/src/reference/gremlin-variants.asciidoc b/docs/src/reference/gremlin-variants.asciidoc
index 918becc487..881ce3a19e 100644
--- a/docs/src/reference/gremlin-variants.asciidoc
+++ b/docs/src/reference/gremlin-variants.asciidoc
@@ -800,12 +800,11 @@ The following table describes the various configuration options for the Gremlin
|connectionPool.maxResponseContentLength |The maximum length in bytes that a message can be received from the server. |2147483647
|connectionPool.maxInProcessPerConnection |The maximum number of in-flight requests that can occur on a connection. |4
|connectionPool.maxSimultaneousUsagePerConnection |The maximum number of times that a connection can be borrowed from the pool simultaneously. |16
-|connectionPool.maxSize |The maximum size of a connection pool for a host. |8
+|connectionPool.maxSize |The maximum size of a connection pool for a host. |128
|connectionPool.maxWaitForConnection |The amount of time in milliseconds to wait for a new connection before timing out. |3000
|connectionPool.maxWaitForClose |The amount of time in milliseconds to wait for pending messages to be returned from the server before closing the connection. |3000
|connectionPool.minInProcessPerConnection |The minimum number of in-flight requests that can occur on a connection. |1
|connectionPool.minSimultaneousUsagePerConnection |The maximum number of times that a connection can be borrowed from the pool simultaneously. |8
-|connectionPool.minSize |The minimum size of a connection pool for a host. |2
|connectionPool.reconnectInterval |The amount of time in milliseconds to wait before trying to reconnect to a dead host. |1000
|connectionPool.resultIterationBatchSize |The override value for the size of the result batches to be returned from the server. |64
|connectionPool.sslCipherSuites |The list of JSSE ciphers to support for SSL connections. If specified, only the ciphers that are listed and supported will be enabled. If not specified, the JVM default is used. |_none_
@@ -815,6 +814,7 @@ The following table describes the various configuration options for the Gremlin
|connectionPool.trustStorePassword |The password of the `trustStore` if it is password-protected |_none_
|connectionPool.validationRequest |A script that is used to test server connectivity. A good script to use is one that evaluates quickly and returns no data. The default simply returns an empty string, but if a graph is required by a particular provider, a good traversal might be `g.inject()`. |_''_
|connectionPool.connectionSetupTimeoutMillis | Duration of time in milliseconds provided for connection setup to complete which includes WebSocket protocol handshake and SSL handshake. |15000
+|connectionPool.idleConnectionTimeout | Duration of time in milliseconds that the driver will allow a channel to not receive read or writes before it automatically closes. |180000
|hosts |The list of hosts that the driver will connect to. |localhost
|jaasEntry |Sets the `AuthProperties.Property.JAAS_ENTRY` properties for authentication to Gremlin Server. |_none_
|nioPoolSize |Size of the pool for handling request/response operations. |available processors
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Channelizer.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Channelizer.java
index b64f969838..1ee56a3aa7 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Channelizer.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Channelizer.java
@@ -28,12 +28,14 @@ import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http.HttpResponseStatus;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslHandler;
+import io.netty.handler.timeout.IdleStateHandler;
import org.apache.tinkerpop.gremlin.driver.exception.ConnectionException;
import org.apache.tinkerpop.gremlin.driver.handler.GremlinResponseHandler;
import org.apache.tinkerpop.gremlin.driver.handler.HttpContentDecompressionHandler;
import org.apache.tinkerpop.gremlin.driver.handler.HttpGremlinRequestEncoder;
import org.apache.tinkerpop.gremlin.driver.handler.HttpGremlinResponseDecoder;
-import org.apache.tinkerpop.gremlin.driver.handler.SslCheckHandler;
+import org.apache.tinkerpop.gremlin.driver.handler.IdleConnectionHandler;
+import org.apache.tinkerpop.gremlin.driver.handler.InactiveChannelHandler;
import org.apache.tinkerpop.gremlin.util.message.ResponseMessage;
import java.util.Collections;
@@ -93,8 +95,9 @@ public interface Channelizer extends ChannelHandler {
private AtomicReference<ResultQueue> pending;
protected static final String PIPELINE_GREMLIN_HANDLER = "gremlin-handler";
- public static final String PIPELINE_SSL_HANDLER = "gremlin-ssl-handler";
-
+ protected static final String PIPELINE_SSL_HANDLER = "gremlin-ssl-handler";
+ protected static final String PIPELINE_IDLE_STATE_HANDLER = "idle-state-handler";
+ protected static final String PIPELINE_IDLE_CONNECTION_HANDLER = "idle-connection-handler";
protected static final String PIPELINE_HTTP_CODEC = "http-codec";
protected static final String PIPELINE_HTTP_AGGREGATOR = "http-aggregator";
protected static final String PIPELINE_HTTP_ENCODER = "gremlin-encoder";
@@ -105,7 +108,7 @@ public interface Channelizer extends ChannelHandler {
"configured at both the client and the server. Ensure that client http handshake " +
"protocol matches the server. Ensure that the server is still reachable.";
- private static final SslCheckHandler sslCheckHandler = new SslCheckHandler();
+ private static final InactiveChannelHandler inactiveChannelHandler = new InactiveChannelHandler();
public boolean supportsSsl() {
return cluster.connectionPoolSettings().enableSsl;
@@ -151,7 +154,7 @@ public interface Channelizer extends ChannelHandler {
sslHandler.setHandshakeTimeoutMillis(0);
pipeline.addLast(PIPELINE_SSL_HANDLER, sslHandler);
} else {
- pipeline.addLast(PIPELINE_SSL_HANDLER, sslCheckHandler);
+ pipeline.addLast(PIPELINE_SSL_HANDLER, inactiveChannelHandler);
}
configure(pipeline);
@@ -187,6 +190,8 @@ public interface Channelizer extends ChannelHandler {
private HttpGremlinResponseDecoder gremlinResponseDecoder;
private HttpContentDecompressionHandler httpCompressionDecoder;
+ private IdleStateHandler idleStateHandler;
+ private IdleConnectionHandler idleConnectionHandler;
@Override
public void init(final Connection connection) {
@@ -196,6 +201,11 @@ public interface Channelizer extends ChannelHandler {
gremlinRequestEncoder = new HttpGremlinRequestEncoder(cluster.getSerializer(), cluster.getRequestInterceptors(),
cluster.isUserAgentOnConnectEnabled(), cluster.isBulkingEnabled(), connection.getUri());
gremlinResponseDecoder = new HttpGremlinResponseDecoder(cluster.getSerializer());
+ if (cluster.getIdleConnectionTimeout() > 0) {
+ final int idleConnectionTimeout = (int) (cluster.getIdleConnectionTimeout() / 1000);
+ idleStateHandler = new IdleStateHandler(idleConnectionTimeout, idleConnectionTimeout, 0);
+ idleConnectionHandler = new IdleConnectionHandler();
+ }
}
@Override
@@ -218,6 +228,12 @@ public interface Channelizer extends ChannelHandler {
if (!supportsSsl() && "https".equalsIgnoreCase(scheme))
throw new IllegalStateException("To use https scheme ensure that enableSsl is set to true in configuration");
+ if (cluster.getIdleConnectionTimeout() > 0) {
+ // idle connection handling is enabled
+ pipeline.addLast(PIPELINE_IDLE_STATE_HANDLER, idleStateHandler);
+ pipeline.addLast(PIPELINE_IDLE_CONNECTION_HANDLER, idleConnectionHandler);
+ }
+
final HttpClientCodec handler = new HttpClientCodec(DEFAULT_MAX_INITIAL_LINE_LENGTH, DEFAULT_MAX_HEADER_SIZE,
1024 * 1024, DEFAULT_FAIL_ON_MISSING_RESPONSE,
DEFAULT_VALIDATE_HEADERS, DEFAULT_INITIAL_BUFFER_SIZE, DEFAULT_PARSE_HTTP_AFTER_CONNECT_REQUEST,
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Client.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Client.java
index ebd03ed359..c843fd49c5 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Client.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Client.java
@@ -56,7 +56,7 @@ import java.util.stream.Collectors;
public abstract class Client {
private static final Logger logger = LoggerFactory.getLogger(Client.class);
- public static final String TOO_MANY_IN_FLIGHT_REQUESTS = "Number of active requests exceeds pool size. " +
+ public static final String TOO_MANY_IN_FLIGHT_REQUESTS = "Number of active requests (%s) exceeds pool size (%s). " +
"Consider increasing the value for maxConnectionPoolSize.";
protected final Cluster cluster;
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Cluster.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Cluster.java
index 8d14d9cf53..7e5d71cb1c 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Cluster.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Cluster.java
@@ -186,8 +186,8 @@ public final class Cluster {
.maxResponseContentLength(settings.connectionPool.maxResponseContentLength)
.maxWaitForConnection(settings.connectionPool.maxWaitForConnection)
.maxConnectionPoolSize(settings.connectionPool.maxSize)
- .minConnectionPoolSize(settings.connectionPool.minSize)
.connectionSetupTimeoutMillis(settings.connectionPool.connectionSetupTimeoutMillis)
+ .idleConnectionTimeoutMillis(settings.connectionPool.idleConnectionTimeout)
.enableUserAgentOnConnect(settings.enableUserAgentOnConnect)
.enableBulkedResult(settings.enableBulkedResult)
.validationRequest(settings.connectionPool.validationRequest);
@@ -309,13 +309,6 @@ public final class Cluster {
return manager.connectionPoolSettings.maxSize;
}
- /**
- * Gets the minimum size of the {@link ConnectionPool}.
- */
- public int minConnectionPoolSize() {
- return manager.connectionPoolSettings.minSize;
- }
-
/**
* Gets the override for the server setting that determines how many results are returned per batch.
*/
@@ -344,6 +337,13 @@ public final class Cluster {
return manager.connectionPoolSettings.maxResponseContentLength;
}
+ /**
+ * Get time in milliseconds that the driver will allow a channel to not receive read or writes before it automatically closes.
+ */
+ public long getIdleConnectionTimeout() {
+ return manager.connectionPoolSettings.idleConnectionTimeout;
+ }
+
/**
* Specifies the load balancing strategy to use on the client side.
*/
@@ -490,7 +490,6 @@ public final class Cluster {
private MessageSerializer<?> serializer = null;
private int nioPoolSize = Runtime.getRuntime().availableProcessors();
private int workerPoolSize = Runtime.getRuntime().availableProcessors() * 2;
- private int minConnectionPoolSize = ConnectionPool.MIN_POOL_SIZE;
private int maxConnectionPoolSize = ConnectionPool.MAX_POOL_SIZE;
private int maxWaitForConnection = Connection.MAX_WAIT_FOR_CONNECTION;
private int maxWaitForClose = Connection.MAX_WAIT_FOR_CLOSE;
@@ -512,6 +511,7 @@ public final class Cluster {
private LoadBalancingStrategy loadBalancingStrategy = new LoadBalancingStrategy.RoundRobin();
private LinkedList<Pair<String, ? extends RequestInterceptor>> interceptors = new LinkedList<>();
private long connectionSetupTimeoutMillis = Connection.CONNECTION_SETUP_TIMEOUT_MILLIS;
+ private long idleConnectionTimeoutMillis = Connection.CONNECTION_IDLE_TIMEOUT_MILLIS;
private boolean enableUserAgentOnConnect = true;
private boolean enableBulkedResult = false;
@@ -688,15 +688,6 @@ public final class Cluster {
return this;
}
- /**
- * The minimum size of the {@link ConnectionPool}. When the {@link Client} is started, {@link Connection}
- * objects will be initially constructed to this size.
- */
- public Builder minConnectionPoolSize(final int minSize) {
- this.minConnectionPoolSize = minSize;
- return this;
- }
-
/**
* Override the server setting that determines how many results are returned per batch.
*/
@@ -880,6 +871,14 @@ public final class Cluster {
return this;
}
+ /**
+ * Sets the time in milliseconds that the driver will allow a channel to not receive read or writes before it automatically closes.
+ */
+ public Builder idleConnectionTimeoutMillis(final long idleConnectionTimeoutMillis) {
+ this.idleConnectionTimeoutMillis = idleConnectionTimeoutMillis;
+ return this;
+ }
+
/**
* Configures whether cluster will send a user agent during
* web socket handshakes
@@ -984,7 +983,6 @@ public final class Cluster {
connectionPoolSettings = new Settings.ConnectionPoolSettings();
connectionPoolSettings.maxSize = builder.maxConnectionPoolSize;
- connectionPoolSettings.minSize = builder.minConnectionPoolSize;
connectionPoolSettings.maxWaitForConnection = builder.maxWaitForConnection;
connectionPoolSettings.maxWaitForClose = builder.maxWaitForClose;
connectionPoolSettings.maxResponseContentLength = builder.maxResponseContentLength;
@@ -1002,6 +1000,7 @@ public final class Cluster {
connectionPoolSettings.sslSkipCertValidation = builder.sslSkipCertValidation;
connectionPoolSettings.validationRequest = builder.validationRequest;
connectionPoolSettings.connectionSetupTimeoutMillis = builder.connectionSetupTimeoutMillis;
+ connectionPoolSettings.idleConnectionTimeout = builder.idleConnectionTimeoutMillis;
sslContextOptional = Optional.ofNullable(builder.sslContext);
@@ -1026,22 +1025,16 @@ public final class Cluster {
// we distinguish between the hostScheduler and the connectionScheduler because you can end in deadlock
// if all the possible jobs the driver allows for go to a single thread pool.
- this.connectionScheduler = new ScheduledThreadPoolExecutor(contactPoints.size() + 1,
+ this.connectionScheduler = new ScheduledThreadPoolExecutor(Runtime.getRuntime().availableProcessors(),
new BasicThreadFactory.Builder().namingPattern("gremlin-driver-conn-scheduler-%d").build());
validationRequest = () -> RequestMessage.build(builder.validationRequest);
}
private void validateBuilder(final Builder builder) {
- if (builder.minConnectionPoolSize < 0)
- throw new IllegalArgumentException("minConnectionPoolSize must be greater than or equal to zero");
-
if (builder.maxConnectionPoolSize < 1)
throw new IllegalArgumentException("maxConnectionPoolSize must be greater than zero");
- if (builder.minConnectionPoolSize > builder.maxConnectionPoolSize)
- throw new IllegalArgumentException("maxConnectionPoolSize cannot be less than minConnectionPoolSize");
-
if (builder.maxWaitForConnection < 1)
throw new IllegalArgumentException("maxWaitForConnection must be greater than zero");
@@ -1065,6 +1058,12 @@ public final class Cluster {
if (builder.connectionSetupTimeoutMillis < 1)
throw new IllegalArgumentException("connectionSetupTimeoutMillis must be greater than zero");
+
+ // zero value will disable idle connection detection
+ // non-zero will be converted to seconds so any value between 1 and 999 is invalid as it will be less than 1 second
+ if (builder.idleConnectionTimeoutMillis != 0 && builder.idleConnectionTimeoutMillis < 1000)
+ throw new IllegalArgumentException("idleConnectionTimeoutMillis must be zero or greater than or equal to 1000");
+
}
synchronized void init() {
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Connection.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Connection.java
index 78424179d9..f39ccb851e 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Connection.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Connection.java
@@ -18,6 +18,7 @@
*/
package org.apache.tinkerpop.gremlin.driver;
+import org.apache.tinkerpop.gremlin.driver.handler.IdleConnectionHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.tinkerpop.gremlin.driver.exception.ConnectionException;
@@ -50,6 +51,7 @@ final class Connection {
public static final int RECONNECT_INTERVAL = 1000;
public static final int RESULT_ITERATION_BATCH_SIZE = 64;
public static final long CONNECTION_SETUP_TIMEOUT_MILLIS = 15000;
+ public static final long CONNECTION_IDLE_TIMEOUT_MILLIS = 180000;
private static final Logger logger = LoggerFactory.getLogger(Connection.class);
private final Channel channel;
@@ -107,19 +109,21 @@ final class Connection {
channel.closeFuture().addListener((ChannelFutureListener) future -> {
logger.debug("OnChannelClose callback called for channel {}", channel);
- // if the closeFuture is not set, it means that closeAsync() wasn't called which means that the
- // close did not come from the client side. it means the server closed the channel for some reason.
- // it's important to distinguish that difference in debugging
+ // if the closeFuture is not set, it means that closeAsync() wasn't called
if (thisConnection.closeFuture.get() == null) {
- logger.error(String.format(
- "Server closed the Connection on channel %s - scheduling removal from %s",
- channel.id().asShortText(), thisConnection.pool.getPoolInfo(thisConnection)));
+ if (!channel.hasAttr(IdleConnectionHandler.IDLE_STATE_EVENT)) {
+ // if idle state event is not present, it means the server closed the channel for some reason.
+ // it's important to distinguish that difference in debugging
+ logger.error(String.format(
+ "Server closed the Connection on channel %s - scheduling removal from %s",
+ channel.id().asShortText(), thisConnection.pool.getPoolInfo(thisConnection)));
+ }
// delegate the task to scheduler thread and free up the event loop
- thisConnection.cluster.connectionScheduler().submit(() -> thisConnection.pool.definitelyDestroyConnection(thisConnection));
+ thisConnection.cluster.connectionScheduler().submit(() -> thisConnection.pool.destroyConnection(thisConnection));
}
});
- logger.info("Created new connection for {}", uri);
+ logger.debug("Created new connection for {}", uri);
} catch (Exception ex) {
throw new ConnectionException(uri, "Could not open " + getConnectionInfo(true), ex);
}
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ConnectionPool.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ConnectionPool.java
index 6187c6e096..45ef031d63 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ConnectionPool.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ConnectionPool.java
@@ -18,6 +18,8 @@
*/
package org.apache.tinkerpop.gremlin.driver;
+import java.util.Queue;
+import java.util.concurrent.ConcurrentLinkedQueue;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.tinkerpop.gremlin.driver.exception.ConnectionException;
@@ -49,8 +51,7 @@ import java.util.concurrent.locks.ReentrantLock;
final class ConnectionPool {
private static final Logger logger = LoggerFactory.getLogger(ConnectionPool.class);
- public static final int MIN_POOL_SIZE = 2;
- public static final int MAX_POOL_SIZE = 16;
+ public static final int MAX_POOL_SIZE = 128;
// A small buffer in millis used for comparing if a connection was created within a certain amount of time.
private static final int CONNECTION_SETUP_TIME_DELTA = 25;
@@ -59,8 +60,8 @@ final class ConnectionPool {
private final Client client;
private final List<Connection> connections;
private final AtomicInteger open;
+ private final Queue<Connection> availableConnections = new ConcurrentLinkedQueue<>();
private final Set<Connection> bin = new CopyOnWriteArraySet<>();
- private final int minPoolSize;
private final int maxPoolSize;
private final String poolLabel;
@@ -69,7 +70,7 @@ final class ConnectionPool {
private final AtomicReference<CompletableFuture<Void>> closeFuture = new AtomicReference<>();
- private volatile int waiter = 0;
+ private final AtomicInteger waiter = new AtomicInteger();
private final Lock waitLock = new ReentrantLock(true);
private final Condition hasAvailableConnection = waitLock.newCondition();
ConnectionFactory connectionFactory;
@@ -90,16 +91,14 @@ final class ConnectionPool {
}
public ConnectionPool(final Host host, final Client client) {
- this(host, client, Optional.empty(), Optional.empty());
+ this(host, client, Optional.empty());
}
- public ConnectionPool(final Host host, final Client client, final Optional<Integer> overrideMinPoolSize,
- final Optional<Integer> overrideMaxPoolSize) {
- this(host, client, overrideMinPoolSize, overrideMaxPoolSize, new ConnectionFactory.DefaultConnectionFactory());
+ public ConnectionPool(final Host host, final Client client, final Optional<Integer> overrideMaxPoolSize) {
+ this(host, client, overrideMaxPoolSize, new ConnectionFactory.DefaultConnectionFactory());
}
- ConnectionPool(final Host host, final Client client, final Optional<Integer> overrideMinPoolSize,
- final Optional<Integer> overrideMaxPoolSize, final ConnectionFactory connectionFactory) {
+ ConnectionPool(final Host host, final Client client, final Optional<Integer> overrideMaxPoolSize, final ConnectionFactory connectionFactory) {
this.host = host;
this.client = client;
this.cluster = client.cluster;
@@ -107,51 +106,43 @@ final class ConnectionPool {
poolLabel = "Connection Pool {host=" + host + "}";
final Settings.ConnectionPoolSettings settings = settings();
- this.minPoolSize = overrideMinPoolSize.orElse(settings.minSize);
this.maxPoolSize = overrideMaxPoolSize.orElse(settings.maxSize);
this.connections = new CopyOnWriteArrayList<>();
this.open = new AtomicInteger();
try {
- final List<CompletableFuture<Void>> connectionCreationFutures = new ArrayList<>();
- for (int i = 0; i < minPoolSize; i++) {
- connectionCreationFutures.add(CompletableFuture.runAsync(() -> {
- final ConnectionResult result = new ConnectionResult();
- try {
- this.connections.add(connectionFactory.create(this));
- this.open.incrementAndGet();
- } catch (ConnectionException e) {
- result.setFailureCause(e);
- throw new CompletionException(e);
- } finally {
- result.setTimeNow();
- if ((latestConnectionResult.get() == null) ||
- (latestConnectionResult.get().getTime() < result.getTime())) {
- latestConnectionResult.set(result);
- }
+ // attempt a single connection to ensure the host is viable
+ // if there is no traffic, it will eventually be closed by IdleConnectionHandler
+ CompletableFuture.runAsync(() -> {
+ final ConnectionResult result = new ConnectionResult();
+ try {
+ final Connection conn = connectionFactory.create(this);
+ this.connections.add(conn);
+ this.availableConnections.add(conn);
+ this.open.incrementAndGet();
+ } catch (ConnectionException e) {
+ result.setFailureCause(e);
+ throw new CompletionException(e);
+ } finally {
+ result.setTimeNow();
+ if ((latestConnectionResult.get() == null) ||
+ (latestConnectionResult.get().getTime() < result.getTime())) {
+ latestConnectionResult.set(result);
}
- }, cluster.connectionScheduler()));
- }
-
- CompletableFuture.allOf(connectionCreationFutures.toArray(new CompletableFuture[0])).join();
+ }
+ }, cluster.connectionScheduler()).join();
} catch (CancellationException ce) {
- logger.warn("Initialization of connections cancelled for {}", this.getPoolInfo(), ce);
+ logger.warn("Initialization of connection pool cancelled for {}", this.getPoolInfo(), ce);
throw ce;
} catch (CompletionException ce) {
- // Some connections might have been initialized. Close the connection pool gracefully to close them.
+ // Close the connection pool gracefully to close any open connections.
this.closeAsync();
-
- final String errMsg = "Could not initialize " + minPoolSize + " (minPoolSize) connections in pool." +
- " Successful connections=" + this.connections.size() +
- ". Closing the connection pool.";
-
final Throwable result = ce.getCause() == null ? ce : ce.getCause();
-
- throw new CompletionException(errMsg, result);
+ throw new CompletionException("Could not initialize connection in pool. Closing the connection pool.", result);
}
- logger.info("Opening connection pool on {} with core size of {}", host, minPoolSize);
+ logger.info("Opening connection pool on {}", host);
}
public Settings.ConnectionPoolSettings settings() {
@@ -159,36 +150,18 @@ final class ConnectionPool {
}
public Connection borrowConnection(final long timeout, final TimeUnit unit) throws TimeoutException, ConnectionException {
- logger.debug("Borrowing connection from pool on {} - timeout in {} {}", host, timeout, unit);
+ logger.debug("Borrowing connection from pool on {} - timeout in {} {} with {}", host, timeout, unit, Thread.currentThread());
if (isClosed()) throw new ConnectionException(host.getHostUri(), host.getAddress(), "Pool is shutdown");
- if (connections.isEmpty()) {
- logger.debug("Tried to borrow connection but the pool was empty for {} - scheduling pool creation and waiting for connection", host);
- for (int i = 0; i < minPoolSize; i++) {
- // If many connections are borrowed at the same time there needs to be a check to make sure no
- // additional ones get scheduled for creation
- if (scheduledForCreation.get() < minPoolSize) {
- scheduledForCreation.incrementAndGet();
- newConnection();
- }
- }
-
- return waitForConnection(timeout, unit);
- }
-
// Get valid connection
final Connection availableConnection = getAvailableConnection();
-
- if (null == availableConnection) {
- if (isClosed())
- throw new ConnectionException(host.getHostUri(), host.getAddress(), "Pool is shutdown");
- logger.debug("Pool was initialized but a connection could not be selected earlier - waiting for connection on {}", host);
+ if (availableConnection == null) {
return waitForConnection(timeout, unit);
}
if (logger.isDebugEnabled())
- logger.debug("Return least used {} on {}", availableConnection.getConnectionInfo(), host);
+ logger.debug("Borrowed {} on {} with {}", availableConnection.getConnectionInfo(), host, Thread.currentThread());
return availableConnection;
}
@@ -217,6 +190,8 @@ final class ConnectionPool {
logger.debug("destroy {}", connection.getConnectionInfo());
destroyConnection(connection);
} else {
+ logger.debug("Pool size is {} - returning connection to pool: {}", poolSize, connection);
+ availableConnections.add(connection);
announceAvailableConnection();
}
}
@@ -240,7 +215,7 @@ final class ConnectionPool {
public synchronized CompletableFuture<Void> closeAsync() {
if (closeFuture.get() != null) return closeFuture.get();
- logger.info("Signalled closing of connection pool on {} with core size of {}", host, minPoolSize);
+ logger.info("Signalled closing of connection pool on {} with current connection size of {}", host, connections.size());
announceAllAvailableConnection();
final CompletableFuture<Void> future = killAvailableConnections();
@@ -291,7 +266,7 @@ final class ConnectionPool {
}
considerNewConnection();
- definitelyDestroyConnection(connection);
+ destroyConnection(connection);
}
private void considerNewConnection() {
@@ -347,7 +322,10 @@ final class ConnectionPool {
final ConnectionResult result = new ConnectionResult();
try {
- connections.add(connectionFactory.create(this));
+ final Connection conn = connectionFactory.create(this);
+ logger.debug("Created connection {}", conn.getConnectionInfo());
+ connections.add(conn);
+ availableConnections.add(conn);
} catch (Exception ex) {
open.decrementAndGet();
logger.error(String.format(
@@ -368,25 +346,12 @@ final class ConnectionPool {
return true;
}
- private boolean destroyConnection(final Connection connection) {
- while (true) {
- final int opened = open.get();
- if (opened <= minPoolSize)
- return false;
-
- if (open.compareAndSet(opened, opened - 1))
- break;
- }
-
- definitelyDestroyConnection(connection);
- return true;
- }
-
- public void definitelyDestroyConnection(final Connection connection) {
+ public void destroyConnection(final Connection connection) {
// only add to the bin for future removal if its not already there.
if (!bin.contains(connection) && !connection.isClosing()) {
bin.add(connection);
connections.remove(connection);
+ availableConnections.remove(connection);
open.decrementAndGet();
}
@@ -408,6 +373,7 @@ final class ConnectionPool {
try {
awaitAvailableConnection(remaining, unit);
} catch (InterruptedException e) {
+ logger.warn("Wait was interrupted", e);
Thread.currentThread().interrupt();
to = 0;
}
@@ -415,16 +381,16 @@ final class ConnectionPool {
if (isClosed())
throw new ConnectionException(host.getHostUri(), host.getAddress(), "Pool is shutdown");
- final Connection leastUsed = getAvailableConnection();
+ final Connection conn = getAvailableConnection();
- if (leastUsed != null) {
+ if (conn != null) {
if (logger.isDebugEnabled())
- logger.debug("Return least used {} on {} after waiting", leastUsed.getConnectionInfo(), host);
- return leastUsed;
+ logger.debug("Return {} on {} after waiting on {}", conn.getConnectionInfo(), host, Thread.currentThread());
+ return conn;
}
remaining = to - TimeUtil.timeSince(start, unit);
- logger.debug("Continue to wait for connection on {} if {} > 0", host, remaining);
+ logger.debug("Continue to wait for connection on {} if {} > 0 on {}", host, remaining, Thread.currentThread());
} while (remaining > 0);
final StringBuilder cause = new StringBuilder("Potential Cause: ");
@@ -436,7 +402,7 @@ final class ConnectionPool {
// Assumes that the root cause will give better information about why the connection failed.
cause.append(ExceptionHelper.getRootCause(res.getFailureCause()).getMessage());
} else if (open.get() >= maxPoolSize) {
- cause.append(Client.TOO_MANY_IN_FLIGHT_REQUESTS);
+ cause.append(String.format(Client.TOO_MANY_IN_FLIGHT_REQUESTS, open.get(), maxPoolSize));
} else {
cause.setLength(0);
}
@@ -478,7 +444,7 @@ final class ConnectionPool {
// let the load-balancer know that the host is acting poorly
if (!host.isAvailable()) {
// if the host is unavailable then we should release the connections
- connections.forEach(this::definitelyDestroyConnection);
+ connections.forEach(this::destroyConnection);
this.cluster.loadBalancingStrategy().onUnavailable(host);
}
}
@@ -522,9 +488,6 @@ final class ConnectionPool {
private void announceAvailableConnection() {
logger.debug("Announce connection available on {}", host);
- if (waiter == 0)
- return;
-
waitLock.lock();
try {
hasAvailableConnection.signal();
@@ -534,27 +497,27 @@ final class ConnectionPool {
}
/**
- * Get the least-used connection from the pool. Also triggers consideration of a new connection if the least-used
+ * Get an available connection from the pool. Also triggers consideration of a new connection if the least-used
* connection has hit the usage maximum or no valid connection could be retrieved from the pool.
*
- * @return The least-used connection from the pool. Returns null if no valid connection could be retrieved from the
+ * @return an available connection from the pool. Returns null if no valid connection could be retrieved from the
* pool.
*/
- private synchronized Connection getAvailableConnection() {
+ private Connection getAvailableConnection() {
Connection available = null;
- int availableConnectionsCount = 0;
- for (Connection connection : connections) {
- if (!connection.isDead() && !connection.isBorrowed().get()) {
- // try to borrow connection
- if (available == null && connection.isBorrowed().compareAndSet(false, true)) {
- available = connection;
- }
- availableConnectionsCount++;
+
+ // remove head of queue until we can borrow a connection or the queue is empty
+ Connection head = availableConnections.poll();
+ while (head != null) {
+ // try to borrow connection
+ if (!head.isDead() && !head.isBorrowed().get() && head.isBorrowed().compareAndSet(false, true)) {
+ available = head;
+ break;
}
+ head = availableConnections.poll();
}
- // todo: may be add new connection when only 10-20% of pool is available?
- if (availableConnectionsCount == 0 && connections.size() < maxPoolSize) {
+ if (available == null && connections.size() < maxPoolSize) {
considerNewConnection();
}
@@ -565,18 +528,17 @@ final class ConnectionPool {
logger.debug("Wait {} {} for an available connection on {} with {}", timeout, unit, host, Thread.currentThread());
waitLock.lock();
- waiter++;
+ waiter.incrementAndGet();
try {
hasAvailableConnection.await(timeout, unit);
} finally {
- waiter--;
+ waiter.decrementAndGet();
waitLock.unlock();
}
}
private void announceAllAvailableConnection() {
- if (waiter == 0)
- return;
+ logger.debug("Announce to all connection available on {}", host);
waitLock.lock();
try {
@@ -608,8 +570,8 @@ final class ConnectionPool {
} else {
final int connectionCount = connections.size();
sb.append(System.lineSeparator());
- sb.append(String.format("Connection Pool Status (size=%s max=%s min=%s toCreate=%s bin=%s)",
- connectionCount, maxPoolSize, minPoolSize, this.scheduledForCreation.get(), bin.size()));
+ sb.append(String.format("Connection Pool Status (size=%s available=%s max=%s toCreate=%s bin=%s waiter=%s)",
+ connectionCount, availableConnections.size(), maxPoolSize, this.scheduledForCreation.get(), bin.size(), waiter.get()));
sb.append(System.lineSeparator());
appendConnections(sb, connectionToCallout, (CopyOnWriteArrayList<Connection>) connections);
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Settings.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Settings.java
index a71983a2be..3bf0e784b4 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Settings.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Settings.java
@@ -190,9 +190,6 @@ public final class Settings {
if (connectionPoolConf.containsKey("sslSkipCertValidation"))
cpSettings.sslSkipCertValidation = connectionPoolConf.getBoolean("sslSkipCertValidation");
- if (connectionPoolConf.containsKey("minSize"))
- cpSettings.minSize = connectionPoolConf.getInt("minSize");
-
if (connectionPoolConf.containsKey("maxSize"))
cpSettings.maxSize = connectionPoolConf.getInt("maxSize");
@@ -217,6 +214,9 @@ public final class Settings {
if (connectionPoolConf.containsKey("connectionSetupTimeoutMillis"))
cpSettings.connectionSetupTimeoutMillis = connectionPoolConf.getLong("connectionSetupTimeoutMillis");
+ if (connectionPoolConf.containsKey("idleConnectionTimeout"))
+ cpSettings.idleConnectionTimeout = connectionPoolConf.getLong("idleConnectionTimeout");
+
settings.connectionPool = cpSettings;
}
@@ -301,12 +301,7 @@ public final class Settings {
public boolean sslSkipCertValidation = false;
/**
- * The minimum size of a connection pool for a {@link Host}. By default this is set to 2.
- */
- public int minSize = ConnectionPool.MIN_POOL_SIZE;
-
- /**
- * The maximum size of a connection pool for a {@link Host}. By default this is set to 8.
+ * The maximum size of a connection pool for a {@link Host}. By default this is set to 500.
*/
public int maxSize = ConnectionPool.MAX_POOL_SIZE;
@@ -352,6 +347,14 @@ public final class Settings {
* complete by then.
*/
public long connectionSetupTimeoutMillis = Connection.CONNECTION_SETUP_TIMEOUT_MILLIS;
+
+ /**
+ * Time in milliseconds that the driver will allow a channel to not receive read or writes before it automatically closes.
+ * Note that while this value is to be provided as milliseconds it will resolve to
+ * second precision. Set this value to 0 to disable this feature.
+ */
+ public long idleConnectionTimeout = Connection.CONNECTION_IDLE_TIMEOUT_MILLIS;
+
}
public static class SerializerSettings {
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinRequestEncoder.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinRequestEncoder.java
index c1ea9ae98c..d5e49bcb46 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinRequestEncoder.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/HttpGremlinRequestEncoder.java
@@ -46,7 +46,7 @@ import java.util.HashMap;
import java.util.List;
import java.util.Map;
-import static org.apache.tinkerpop.gremlin.driver.handler.SslCheckHandler.REQUEST_SENT;
+import static org.apache.tinkerpop.gremlin.driver.handler.InactiveChannelHandler.REQUEST_SENT;
/**
* Converts {@link RequestMessage} to a {@code HttpRequest}.
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/IdleConnectionHandler.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/IdleConnectionHandler.java
new file mode 100644
index 0000000000..fb4253160b
--- /dev/null
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/IdleConnectionHandler.java
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.driver.handler;
+
+import io.netty.channel.ChannelDuplexHandler;
+import io.netty.channel.ChannelHandler;
+import io.netty.channel.ChannelHandlerContext;
+import io.netty.handler.timeout.IdleStateEvent;
+import io.netty.util.AttributeKey;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Detects {@link IdleStateEvent}s and closes the channel with the goal of releasing resources that haven't been used in a while.
+ */
+@ChannelHandler.Sharable
+public class IdleConnectionHandler extends ChannelDuplexHandler {
+ private static final Logger logger = LoggerFactory.getLogger(IdleConnectionHandler.class);
+ public static final AttributeKey<IdleStateEvent> IDLE_STATE_EVENT = AttributeKey.valueOf("idleStateEvent");
+
+ /**
+ * Detects if an {@link IdleStateEvent} has occurred and closes the channel.
+ */
+ @Override
+ public void userEventTriggered(final ChannelHandlerContext ctx, final Object evt) {
+ if (evt instanceof IdleStateEvent) {
+ final IdleStateEvent e = (IdleStateEvent) evt;
+ logger.info("Detected IdleStateEvent {} - closing channel {}", e, ctx.channel().id().asShortText());
+ ctx.channel().attr(IDLE_STATE_EVENT).set(e);
+ ctx.close();
+ }
+ }
+}
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/SslCheckHandler.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/InactiveChannelHandler.java
similarity index 77%
rename from gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/SslCheckHandler.java
rename to gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/InactiveChannelHandler.java
index dc58186d2c..f38dd77305 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/SslCheckHandler.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/handler/InactiveChannelHandler.java
@@ -24,10 +24,10 @@ import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.util.AttributeKey;
/**
- * Handler used to detect if the server requires TLS when the client is sending plain HTTP.
+ * Handler used to detect if the channel has become inactive and throw an error in certain scenarios.
*/
@ChannelHandler.Sharable
-public class SslCheckHandler extends ChannelInboundHandlerAdapter {
+public class InactiveChannelHandler extends ChannelInboundHandlerAdapter {
public static final AttributeKey<Integer> BYTES_READ = AttributeKey.valueOf("bytesRead");
public static final AttributeKey<Boolean> REQUEST_SENT = AttributeKey.valueOf("requestSent");
@@ -40,8 +40,13 @@ public class SslCheckHandler extends ChannelInboundHandlerAdapter {
if ((null == ctx.channel().attr(BYTES_READ).get()) && (null != ctx.channel().attr(REQUEST_SENT).get())) {
// BYTES_READ is set by the HttpGremlinResponseStreamDecoder when any data is received. If it isn't set,
// it can be assumed that no bytes were received when the connection closed.
- final String errMsg = "Connection to server closed unexpectedly. Ensure that the server is still" +
- " reachable. The server may be expecting SSL to be enabled.";
+ String errMsg;
+ if (ctx.channel().hasAttr(IdleConnectionHandler.IDLE_STATE_EVENT)) {
+ errMsg = "Idle timeout occurred before response could be received from server - consider increasing idleConnectionTimeout";
+ } else {
+ errMsg = "Connection to server closed unexpectedly. Ensure that the server is still" +
+ " reachable. The server may be expecting SSL to be enabled.";
+ }
ctx.fireExceptionCaught(new RuntimeException(errMsg));
} else {
super.channelInactive(ctx);
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/util/ConfigurationEvaluator.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/util/ConfigurationEvaluator.java
index 172b8cf04c..b055fb0e90 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/util/ConfigurationEvaluator.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/util/ConfigurationEvaluator.java
@@ -32,7 +32,6 @@ import java.util.stream.Stream;
*/
public class ConfigurationEvaluator {
- private final List<Integer> minConnectionPoolSizeRange = Arrays.asList(4,8,12,16,32,64,96,128,192,256,384,512);
private final List<Integer> maxConnectionPoolSizeRange = Arrays.asList(16,32,64,96,128,192,256,384,512);
private final List<Integer> workerPoolSizeRange = Arrays.asList(1,2,3,4,8,16,32);
private final List<Integer> nioPoolSizeRange = Arrays.asList(1,2,4);
@@ -45,28 +44,23 @@ public class ConfigurationEvaluator {
for (int ir = 0; ir < nioPoolSizeRange.size(); ir++) {
for (int is = 0; is < parallelismSizeRange.size(); is++) {
for (int it = 0; it < workerPoolSizeRange.size(); it++) {
- for (int iw = 0; iw < minConnectionPoolSizeRange.size(); iw++) {
- for (int ix = 0; ix < maxConnectionPoolSizeRange.size(); ix++) {
- if (minConnectionPoolSizeRange.get(iw) <= maxConnectionPoolSizeRange.get(ix)) {
- final String s = String.join(",", String.valueOf(ir), String.valueOf(is), String.valueOf(it), String.valueOf(iw), String.valueOf(ix));
- if (!configsTried.contains(s)) {
- final Object[] argsToProfiler =
- Stream.of("nioPoolSize", nioPoolSizeRange.get(ir).toString(),
- "parallelism", parallelismSizeRange.get(is).toString(),
- "workerPoolSize", workerPoolSizeRange.get(it).toString(),
- "minConnectionPoolSize", minConnectionPoolSizeRange.get(iw).toString(),
- "maxConnectionPoolSize", maxConnectionPoolSizeRange.get(ix).toString(),
- "noExit", Boolean.TRUE.toString()).toArray();
+ for (int ix = 0; ix < maxConnectionPoolSizeRange.size(); ix++) {
+ final String s = String.join(",", String.valueOf(ir), String.valueOf(is), String.valueOf(it), String.valueOf(ix));
+ if (!configsTried.contains(s)) {
+ final Object[] argsToProfiler =
+ Stream.of("nioPoolSize", nioPoolSizeRange.get(ir).toString(),
+ "parallelism", parallelismSizeRange.get(is).toString(),
+ "workerPoolSize", workerPoolSizeRange.get(it).toString(),
+ "maxConnectionPoolSize", maxConnectionPoolSizeRange.get(ix).toString(),
+ "noExit", Boolean.TRUE.toString()).toArray();
- final Object[] withExtraArgs = args.length > 0 ? Stream.concat(Stream.of(args), Stream.of(argsToProfiler)).toArray() : argsToProfiler;
+ final Object[] withExtraArgs = args.length > 0 ? Stream.concat(Stream.of(args), Stream.of(argsToProfiler)).toArray() : argsToProfiler;
- final String[] stringProfilerArgs = Arrays.copyOf(withExtraArgs, withExtraArgs.length, String[].class);
- System.out.println("Testing with: " + Arrays.toString(stringProfilerArgs));
- ProfilingApplication.main(stringProfilerArgs);
- TimeUnit.SECONDS.sleep(5);
- configsTried.add(s);
- }
- }
+ final String[] stringProfilerArgs = Arrays.copyOf(withExtraArgs, withExtraArgs.length, String[].class);
+ System.out.println("Testing with: " + Arrays.toString(stringProfilerArgs));
+ ProfilingApplication.main(stringProfilerArgs);
+ TimeUnit.SECONDS.sleep(5);
+ configsTried.add(s);
}
}
}
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/util/ProfilingApplication.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/util/ProfilingApplication.java
index 1d718f430a..8518d7b64f 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/util/ProfilingApplication.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/util/ProfilingApplication.java
@@ -184,7 +184,6 @@ public class ProfilingApplication {
final int executions = Integer.parseInt(options.getOrDefault("executions", "10").toString());
final int nioPoolSize = Integer.parseInt(options.getOrDefault("nioPoolSize", "1").toString());
final int requests = Integer.parseInt(options.getOrDefault("requests", "10000").toString());
- final int minConnectionPoolSize = Integer.parseInt(options.getOrDefault("minConnectionPoolSize", "256").toString());
final int maxConnectionPoolSize = Integer.parseInt(options.getOrDefault("maxConnectionPoolSize", "256").toString());
final int maxWaitForConnection = Integer.parseInt(options.getOrDefault("maxWaitForConnection", "3000").toString());
final int workerPoolSize = Integer.parseInt(options.getOrDefault("workerPoolSize", Runtime.getRuntime().availableProcessors() * 2).toString());
@@ -195,7 +194,6 @@ public class ProfilingApplication {
final String script = options.getOrDefault("script", "1+1").toString();
final Cluster cluster = Cluster.build(host)
- .minConnectionPoolSize(minConnectionPoolSize)
.maxConnectionPoolSize(maxConnectionPoolSize)
.nioPoolSize(nioPoolSize)
.maxWaitForConnection(maxWaitForConnection)
@@ -226,7 +224,7 @@ public class ProfilingApplication {
final File f = null == fileName ? null : new File(fileName.toString());
if (f != null && f.length() == 0) {
try (final PrintWriter writer = new PrintWriter(new BufferedWriter(new FileWriter(f, true)))) {
- writer.println("parallelism\tnioPoolSize\tminConnectionPoolSize\tmaxConnectionPoolSize\tworkerPoolSize\trequestPerSecond");
+ writer.println("parallelism\tnioPoolSize\tmaxConnectionPoolSize\tworkerPoolSize\trequestPerSecond");
}
}
@@ -257,7 +255,7 @@ public class ProfilingApplication {
System.out.println(String.format("avg req/sec: %s", averageRequestPerSecond));
if (f != null) {
try (final PrintWriter writer = new PrintWriter(new BufferedWriter(new FileWriter(f, true)))) {
- writer.println(String.join("\t", String.valueOf(parallelism), String.valueOf(nioPoolSize), String.valueOf(minConnectionPoolSize), String.valueOf(maxConnectionPoolSize), String.valueOf(workerPoolSize), String.valueOf(averageRequestPerSecond)));
+ writer.println(String.join("\t", String.valueOf(parallelism), String.valueOf(nioPoolSize), String.valueOf(maxConnectionPoolSize), String.valueOf(workerPoolSize), String.valueOf(averageRequestPerSecond)));
}
}
} else if (TestType.LATENCY == testType) {
diff --git a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ClusterBuilderTest.java b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ClusterBuilderTest.java
index 2ee546d0b0..dca023d4b4 100644
--- a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ClusterBuilderTest.java
+++ b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ClusterBuilderTest.java
@@ -40,9 +40,6 @@ public class ClusterBuilderTest {
return Arrays.asList(new Object[][]{
{"maxConnectionPoolSize0", Cluster.build().maxConnectionPoolSize(0), "maxConnectionPoolSize must be greater than zero"},
{"maxConnectionPoolSizeNeg1", Cluster.build().maxConnectionPoolSize(-1), "maxConnectionPoolSize must be greater than zero"},
- {"minConnectionPoolSizeNeg1", Cluster.build().minConnectionPoolSize(-1), "minConnectionPoolSize must be greater than or equal to zero"},
- {"minConnectionPoolSizeLteMax", Cluster.build().minConnectionPoolSize(100).maxConnectionPoolSize(99), "maxConnectionPoolSize cannot be less than minConnectionPoolSize"},
- {"minConnectionPoolSizeLteMax", Cluster.build().minConnectionPoolSize(100).maxConnectionPoolSize(99), "maxConnectionPoolSize cannot be less than minConnectionPoolSize"},
{"maxConnectionPoolSize0", Cluster.build().maxWaitForConnection(0), "maxWaitForConnection must be greater than zero"},
{"maxWaitForClose0", Cluster.build().maxWaitForClose(0), "maxWaitForClose must be greater than zero"},
{"maxWaitForCloseNeg1", Cluster.build().maxWaitForClose(-1), "maxWaitForClose must be greater than zero"},
@@ -54,6 +51,9 @@ public class ClusterBuilderTest {
{"nioPoolSize0", Cluster.build().nioPoolSize(0), "nioPoolSize must be greater than zero"},
{"nioPoolSizeNeg1", Cluster.build().nioPoolSize(-1), "nioPoolSize must be greater than zero"},
{"connectionSetupTimeoutMillis0", Cluster.build().connectionSetupTimeoutMillis(0), "connectionSetupTimeoutMillis must be greater than zero"},
+ {"idleConnectionTimeoutMillisNeg1", Cluster.build().idleConnectionTimeoutMillis(-1), "idleConnectionTimeoutMillis must be zero or greater than or equal to 1000"},
+ {"idleConnectionTimeoutMillisOne", Cluster.build().idleConnectionTimeoutMillis(1), "idleConnectionTimeoutMillis must be zero or greater than or equal to 1000"},
+ {"idleConnectionTimeoutMillis999", Cluster.build().idleConnectionTimeoutMillis(999), "idleConnectionTimeoutMillis must be zero or greater than or equal to 1000"},
{"workerPoolSize0", Cluster.build().workerPoolSize(0), "workerPoolSize must be greater than zero"},
{"workerPoolSizeNeg1", Cluster.build().workerPoolSize(-1), "workerPoolSize must be greater than zero"}});
}
diff --git a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ConnectionPoolTest.java b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ConnectionPoolTest.java
index 01aa1f695e..1c8941b908 100644
--- a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ConnectionPoolTest.java
+++ b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ConnectionPoolTest.java
@@ -63,10 +63,9 @@ public class ConnectionPoolTest {
final Client client = new Client.ClusteredClient(cluster);
- // create pool with 1 connection
- final ConnectionPool connectionPool = new ConnectionPool(host, client,
- Optional.of(1), Optional.of(2), connectionFactory);
- // try to borrow this connection.
+ // create pool - starts with 1 connection
+ final ConnectionPool connectionPool = new ConnectionPool(host, client, Optional.of(2), connectionFactory);
+ // try to borrow a connection.
final Connection conn0 = connectionPool.borrowConnection(100, TimeUnit.MILLISECONDS);
assertNotNull(connectionPool);
@@ -90,6 +89,7 @@ public class ConnectionPoolTest {
}
// return conn0 to pool, can be borrowed again
+ connectionPool.returnConnection(conn0);
when(mockConn0.isBorrowed()).thenReturn(new AtomicBoolean(false));
final Connection conn00 = connectionPool.borrowConnection(100, TimeUnit.MILLISECONDS);
diff --git a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/SettingsTest.java b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/SettingsTest.java
index a2d47ff1db..82b5c2fc99 100644
--- a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/SettingsTest.java
+++ b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/SettingsTest.java
@@ -58,7 +58,6 @@ public class SettingsTest {
conf.setProperty("connectionPool.sslEnabledProtocols", Arrays.asList("TLSv1.1","TLSv1.2"));
conf.setProperty("connectionPool.sslCipherSuites", Arrays.asList("TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"));
conf.setProperty("connectionPool.sslSkipCertValidation", true);
- conf.setProperty("connectionPool.minSize", 100);
conf.setProperty("connectionPool.maxSize", 200);
conf.setProperty("connectionPool.minSimultaneousUsagePerConnection", 300);
conf.setProperty("connectionPool.maxSimultaneousUsagePerConnection", 400);
@@ -71,6 +70,7 @@ public class SettingsTest {
conf.setProperty("connectionPool.channelizer", "channelizer0");
conf.setProperty("connectionPool.validationRequest", "g.inject()");
conf.setProperty("connectionPool.connectionSetupTimeoutMillis", 15000);
+ conf.setProperty("connectionPool.idleConnectionTimeout", 160000);
final Settings settings = Settings.from(conf);
@@ -95,12 +95,12 @@ public class SettingsTest {
assertEquals(Arrays.asList("TLSv1.1","TLSv1.2"), settings.connectionPool.sslEnabledProtocols);
assertEquals(Arrays.asList("TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"), settings.connectionPool.sslCipherSuites);
assertThat(settings.connectionPool.sslSkipCertValidation, is(true));
- assertEquals(100, settings.connectionPool.minSize);
assertEquals(200, settings.connectionPool.maxSize);
assertEquals(700, settings.connectionPool.maxWaitForConnection);
assertEquals(800, settings.connectionPool.maxResponseContentLength);
assertEquals(900, settings.connectionPool.reconnectInterval);
assertEquals(15000, settings.connectionPool.connectionSetupTimeoutMillis);
+ assertEquals(160000, settings.connectionPool.idleConnectionTimeout);
assertEquals(1100, settings.connectionPool.resultIterationBatchSize);
assertEquals("g.inject()", settings.connectionPool.validationRequest);
}
diff --git a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/handler/IdleConnectionHandlerTest.java b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/handler/IdleConnectionHandlerTest.java
new file mode 100644
index 0000000000..f59f4ad7bc
--- /dev/null
+++ b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/handler/IdleConnectionHandlerTest.java
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.tinkerpop.gremlin.driver.handler;
+
+import io.netty.channel.embedded.EmbeddedChannel;
+import io.netty.handler.timeout.IdleStateEvent;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+
+public class IdleConnectionHandlerTest {
+ private EmbeddedChannel testChannel;
+
+ @Before
+ public void setup() {
+ testChannel = new EmbeddedChannel(new IdleConnectionHandler());
+ }
+
+ @After
+ public void teardown() {
+ // if any exceptions happened, throw them otherwise the test will only silently fail
+ testChannel.checkException();
+ }
+
+
+ @Test
+ public void userEventTriggered_setsIdleStateEventAttribute() {
+ testChannel.pipeline().fireUserEventTriggered(IdleStateEvent.WRITER_IDLE_STATE_EVENT);
+ assertTrue(testChannel.hasAttr(IdleConnectionHandler.IDLE_STATE_EVENT));
+ }
+
+ @Test
+ public void userEventTriggered_notIdleStateEvent_doesNotSetAttribute() {
+ testChannel.pipeline().fireUserEventTriggered("some other event");
+ assertFalse(testChannel.hasAttr(IdleConnectionHandler.IDLE_STATE_EVENT));
+ }
+}
\ No newline at end of file
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/ClientConnectionIntegrateTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/ClientConnectionIntegrateTest.java
index f780bba564..34ebd67854 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/ClientConnectionIntegrateTest.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/driver/ClientConnectionIntegrateTest.java
@@ -30,6 +30,7 @@ import org.apache.tinkerpop.gremlin.server.TestClientFactory;
import org.hamcrest.core.Is;
import org.junit.After;
import org.junit.AfterClass;
+import org.junit.Assert;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
@@ -62,7 +63,8 @@ import static org.junit.Assert.*;
public class ClientConnectionIntegrateTest extends AbstractGremlinServerIntegrationTest {
private static LogCaptor logCaptor;
- private Level previousLevel;
+ private Level previousConnectionLevel;
+ private Level previousConnectionPoolLevel;
@BeforeClass
public static void setupLogCaptor() {
@@ -77,8 +79,11 @@ public class ClientConnectionIntegrateTest extends AbstractGremlinServerIntegrat
@Before
public void setupForEachTest() {
final Logger lc = (Logger) LoggerFactory.getLogger(Connection.class);
- previousLevel = lc.getLevel();
+ previousConnectionLevel = lc.getLevel();
lc.setLevel(Level.DEBUG);
+ final Logger lcp = (Logger) LoggerFactory.getLogger(ConnectionPool.class);
+ previousConnectionPoolLevel = lcp.getLevel();
+ lcp.setLevel(Level.DEBUG);
logCaptor.clearLogs();
}
@@ -86,7 +91,9 @@ public class ClientConnectionIntegrateTest extends AbstractGremlinServerIntegrat
@After
public void afterEachTest() {
final Logger lc = (Logger) LoggerFactory.getLogger(Connection.class);
- lc.setLevel(previousLevel);
+ lc.setLevel(previousConnectionLevel);
+ final Logger lcp = (Logger) LoggerFactory.getLogger(ConnectionPool.class);
+ lcp.setLevel(previousConnectionPoolLevel);
}
/**
@@ -97,7 +104,6 @@ public class ClientConnectionIntegrateTest extends AbstractGremlinServerIntegrat
// Set a low value of maxResponseContentLength to intentionally trigger CorruptedFrameException
final Cluster cluster = TestClientFactory.build()
.maxResponseContentLength(64)
- .minConnectionPoolSize(1)
.maxConnectionPoolSize(2)
.create();
final Client.ClusteredClient client = cluster.connect();
@@ -140,7 +146,6 @@ public class ClientConnectionIntegrateTest extends AbstractGremlinServerIntegrat
public void shouldBalanceConcurrentRequestsAcrossConnections() throws InterruptedException {
final int connPoolSize = 16;
final Cluster cluster = TestClientFactory.build()
- .minConnectionPoolSize(connPoolSize)
.maxConnectionPoolSize(connPoolSize)
.create();
final Client.ClusteredClient client = cluster.connect();
@@ -183,10 +188,9 @@ public class ClientConnectionIntegrateTest extends AbstractGremlinServerIntegrat
}
@Test
- public void overLimitOperationsShouldCreateNewHttpConnectionPerRequestAsNeeded() throws InterruptedException {
+ public void shouldCreateNewHttpConnectionPerRequestAsNeeded() throws InterruptedException {
final int operations = 6;
final Cluster cluster = TestClientFactory.build()
- .minConnectionPoolSize(1)
.maxConnectionPoolSize(operations)
.create();
final Client.ClusteredClient client = cluster.connect();
@@ -236,7 +240,7 @@ public class ClientConnectionIntegrateTest extends AbstractGremlinServerIntegrat
*/
@Test
public void shouldSucceedWithJitteryConnection() throws Exception {
- final Cluster cluster = TestClientFactory.build().minConnectionPoolSize(1).maxConnectionPoolSize(128).
+ final Cluster cluster = TestClientFactory.build().maxConnectionPoolSize(128).
reconnectInterval(1000).
maxWaitForConnection(4000).validationRequest("g.inject()").create();
final Client.ClusteredClient client = cluster.connect();
@@ -292,6 +296,74 @@ public class ClientConnectionIntegrateTest extends AbstractGremlinServerIntegrat
cluster.close();
}
+ @Test
+ public void shouldCloseIdleConnectionsAndRecreateNewConnections() throws InterruptedException {
+ int idleMillis = 3000;
+ final Cluster cluster = TestClientFactory.build()
+ .idleConnectionTimeoutMillis(idleMillis)
+ .create();
+ final Client.ClusteredClient client = cluster.connect();
+ client.init();
+ final ExecutorService executorServiceForTesting = cluster.executor();
+
+ try {
+ // create or reuse some connections
+ chooseConnections(3, client, executorServiceForTesting);
+
+ // simulate downtime where there is no traffic so that all connections become idle
+ TimeUnit.MILLISECONDS.sleep(idleMillis * 3);
+ assertEquals(1, client.hostConnectionPools.size());
+ // all connections should have been closed due to idle timeout
+ assertTrue(client.hostConnectionPools.values().iterator().next().getPoolInfo().contains("no connections in pool"));
+
+ // create or reuse some more connections
+ chooseConnections(4, client, executorServiceForTesting);
+
+ } finally {
+ executorServiceForTesting.shutdown();
+ client.close();
+ cluster.close();
+ }
+ }
+
+ @Test
+ public void shouldThrowErrorWithHelpfulMessageWhenIdleTimeoutReachedBeforeResponseReceived() throws InterruptedException {
+ int idleMillis = 1000;
+ final Cluster cluster = TestClientFactory.build()
+ .idleConnectionTimeoutMillis(idleMillis)
+ .create();
+ final Client.ClusteredClient client = cluster.connect();
+
+ try {
+ client.submit("Thread.sleep(" + idleMillis * 3 + ")").all().get();
+ fail("Expected exception due to idle timeout");
+ } catch (Exception e) {
+ assertTrue(e.getMessage().contains("Idle timeout occurred before response could be received from server - consider increasing idleConnectionTimeout"));
+ } finally {
+ client.close();
+ cluster.close();
+ }
+ }
+
+ private static void chooseConnections(int operations, Client.ClusteredClient client, ExecutorService executorServiceForTesting) throws InterruptedException {
+ final RequestMessage.Builder request = client.buildMessage(RequestMessage.build("g.inject(1)"));
+ final Callable<Connection> sendQueryCallable = () -> client.chooseConnection(request.create());
+ final List<Callable<Connection>> listOfTasks = new ArrayList<>();
+ for (int i = 0; i < operations; i++) {
+ listOfTasks.add(sendQueryCallable);
+ }
+
+ final List<Future<Connection>> executorSubmitFutures = executorServiceForTesting.invokeAll(listOfTasks);
+ executorSubmitFutures.parallelStream().map(fut -> {
+ try {
+ return fut.get();
+ } catch (InterruptedException | ExecutionException e) {
+ fail(e.getMessage());
+ return null;
+ }
+ }).forEach(Assert::assertNotNull);
+ }
+
/**
* Introduces random failures when creating a {@link Connection} for the {@link ConnectionPool}.
*/
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
index 19d56491af..acb411a9d0 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
@@ -260,7 +260,7 @@ public class GremlinDriverIntegrateTest extends AbstractGremlinServerIntegration
final AtomicInteger handshakeRequests = new AtomicInteger(0);
final Cluster cluster = TestClientFactory.build().
- minConnectionPoolSize(1).maxConnectionPoolSize(1).
+ maxConnectionPoolSize(1).
addInterceptor("counter", r -> {
handshakeRequests.incrementAndGet();
return r;
@@ -964,8 +964,7 @@ public class GremlinDriverIntegrateTest extends AbstractGremlinServerIntegration
@Test
public void shouldBeThreadSafeToUseOneClient() throws Exception {
final Cluster cluster = TestClientFactory.build().workerPoolSize(2)
- .maxConnectionPoolSize(16)
- .minConnectionPoolSize(8).create();
+ .maxConnectionPoolSize(16).create();
final Client client = cluster.connect();
final Map<Integer, Integer> results = new ConcurrentHashMap<>();
@@ -1168,8 +1167,8 @@ public class GremlinDriverIntegrateTest extends AbstractGremlinServerIntegration
*/
@Test
public void shouldReturnClearExceptionCauseWhenClientIsTooBusyAndConnectionPoolIsFull() throws InterruptedException {
+ int maxSize = 1;
final Cluster cluster = TestClientFactory.build()
- .minConnectionPoolSize(1)
.maxConnectionPoolSize(1)
.connectionSetupTimeoutMillis(100)
.maxWaitForConnection(150)
@@ -1183,7 +1182,7 @@ public class GremlinDriverIntegrateTest extends AbstractGremlinServerIntegration
} catch (Exception e) {
final Throwable root = ExceptionHelper.getRootCause(e);
assertTrue(root instanceof TimeoutException);
- assertTrue(root.getMessage().contains(Client.TOO_MANY_IN_FLIGHT_REQUESTS));
+ assertTrue(root.getMessage().contains(String.format(Client.TOO_MANY_IN_FLIGHT_REQUESTS, maxSize, maxSize)));
}
}
(tinkerpop) 08/17: Remove Traverser serializers for GraphSON/GraphBinary V4 (#2862)
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenhuuu pushed a commit to branch master-http-final
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 191849d7e2cee4befd054a0b47035729e4e38389
Author: kenhuuu <10...@users.noreply.github.com>
AuthorDate: Tue Oct 29 13:31:13 2024 -0700
Remove Traverser serializers for GraphSON/GraphBinary V4 (#2862)
---
.../io/binary/TypeSerializerRegistry.java | 2 -
.../io/binary/types/TraverserSerializer.java | 51 ----------
.../structure/io/graphson/GraphSONModule.java | 3 -
.../io/graphson/GraphSONTypeSerializerV4.java | 2 -
.../io/graphson/TraversalSerializersV4.java | 48 ---------
.../graphson/GraphSONMapperEmbeddedTypeTest.java | 9 --
.../GraphSONMapperPartialEmbeddedTypeTest.java | 7 --
.../server/handler/HttpGremlinEndpointHandler.java | 4 +-
.../gremlin/server/GremlinDriverIntegrateTest.java | 7 +-
.../io/graphbinary/bulked-traverser-v4.gbin | Bin 93 -> 0 bytes
.../io/graphbinary/empty-traverser-v4.gbin | Bin 12 -> 0 bytes
.../io/graphbinary/vertex-traverser-v4.gbin | Bin 451 -> 0 bytes
.../structure/io/graphson/bulked-traverser-v4.json | 32 ------
.../structure/io/graphson/empty-traverser-v4.json | 10 --
.../structure/io/graphson/vertex-traverser-v4.json | 109 ---------------------
.../ser/AbstractGraphSONMessageSerializerV4.java | 3 +
.../io/AbstractTypedCompatibilityTest.java | 48 ---------
17 files changed, 12 insertions(+), 323 deletions(-)
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/binary/TypeSerializerRegistry.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/binary/TypeSerializerRegistry.java
index 6ee2a5fc60..a5b9f79047 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/binary/TypeSerializerRegistry.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/binary/TypeSerializerRegistry.java
@@ -52,7 +52,6 @@ import org.apache.tinkerpop.gremlin.structure.io.binary.types.SingleTypeSerializ
import org.apache.tinkerpop.gremlin.structure.io.binary.types.StringSerializer;
import org.apache.tinkerpop.gremlin.structure.io.binary.types.TransformSerializer;
import org.apache.tinkerpop.gremlin.structure.io.binary.types.TraversalExplanationSerializer;
-import org.apache.tinkerpop.gremlin.structure.io.binary.types.TraverserSerializer;
import org.apache.tinkerpop.gremlin.structure.io.binary.types.TreeSerializer;
import org.apache.tinkerpop.gremlin.structure.io.binary.types.UUIDSerializer;
import org.apache.tinkerpop.gremlin.structure.io.binary.types.VertexPropertySerializer;
@@ -104,7 +103,6 @@ public class TypeSerializerRegistry {
new RegistryEntry<>(Vertex.class, new VertexSerializer()),
new RegistryEntry<>(Direction.class, EnumSerializer.DirectionSerializer),
new RegistryEntry<>(T.class, EnumSerializer.TSerializer),
- new RegistryEntry<>(Traverser.class, new TraverserSerializer()),
new RegistryEntry<>(BigDecimal.class, new BigDecimalSerializer()),
new RegistryEntry<>(BigInteger.class, new BigIntegerSerializer()),
new RegistryEntry<>(Byte.class, SingleTypeSerializer.ByteSerializer),
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/binary/types/TraverserSerializer.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/binary/types/TraverserSerializer.java
deleted file mode 100644
index 6361fb01e8..0000000000
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/binary/types/TraverserSerializer.java
+++ /dev/null
@@ -1,51 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package org.apache.tinkerpop.gremlin.structure.io.binary.types;
-
-import org.apache.tinkerpop.gremlin.structure.io.binary.DataType;
-import org.apache.tinkerpop.gremlin.structure.io.binary.GraphBinaryReader;
-import org.apache.tinkerpop.gremlin.structure.io.binary.GraphBinaryWriter;
-import org.apache.tinkerpop.gremlin.process.remote.traversal.DefaultRemoteTraverser;
-import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
-import org.apache.tinkerpop.gremlin.structure.io.Buffer;
-
-import java.io.IOException;
-
-/**
- * @author Stephen Mallette (http://stephen.genoprime.com)
- */
-public class TraverserSerializer extends SimpleTypeSerializer<Traverser> {
-
- public TraverserSerializer() {
- super(DataType.TRAVERSER);
- }
-
- @Override
- protected Traverser readValue(final Buffer buffer, final GraphBinaryReader context) throws IOException {
- final long bulk = context.readValue(buffer, Long.class, false);
- final Object v = context.read(buffer);
- return new DefaultRemoteTraverser<>(v, bulk);
- }
-
- @Override
- protected void writeValue(final Traverser value, final Buffer buffer, final GraphBinaryWriter context) throws IOException {
- context.writeValue(value.bulk(), buffer, false);
- context.write(value.get(), buffer);
- }
-}
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java
index b19cbb4931..175114d149 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONModule.java
@@ -167,7 +167,6 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
put(Metrics.class, "Metrics");
put(TraversalMetrics.class, "TraversalMetrics");
put(TraversalExplanation.class, "TraversalExplanation");
- put(Traverser.class, "Traverser");
put(Tree.class, "Tree");
put(AndP.class, "P");
put(OrP.class, "P");
@@ -274,7 +273,6 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
T.class).forEach(e -> addSerializer(e, new TraversalSerializersV4.EnumJacksonSerializer()));
addSerializer(P.class, new TraversalSerializersV4.PJacksonSerializer());
addSerializer(Lambda.class, new TraversalSerializersV4.LambdaJacksonSerializer());
- addSerializer(Traverser.class, new TraversalSerializersV4.TraverserJacksonSerializer());
addSerializer(TraversalStrategy.class, new TraversalSerializersV4.TraversalStrategyJacksonSerializer());
/////////////////////// DESERIALIZERS ////////////////////////////
@@ -317,7 +315,6 @@ abstract class GraphSONModule extends TinkerPopJacksonModule {
addDeserializer(P.class, new TraversalSerializersV4.PJacksonDeserializer());
addDeserializer(TextP.class, new TraversalSerializersV4.TextPJacksonDeserializer());
addDeserializer(Lambda.class, new TraversalSerializersV4.LambdaJacksonDeserializer());
- addDeserializer(Traverser.class, new TraversalSerializersV4.TraverserJacksonDeserializer());
Arrays.asList(
ConnectiveStrategy.class,
ElementIdStrategy.class,
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONTypeSerializerV4.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONTypeSerializerV4.java
index 17a406bb8d..a6ba23b576 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONTypeSerializerV4.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONTypeSerializerV4.java
@@ -147,8 +147,6 @@ public class GraphSONTypeSerializerV4 extends AbstractGraphSONTypeSerializer {
mapped = ByteBuffer.class;
else if (InetAddress.class.isAssignableFrom(c))
mapped = InetAddress.class;
- else if (Traverser.class.isAssignableFrom(c))
- mapped = Traverser.class;
else if (Lambda.class.isAssignableFrom(c))
mapped = Lambda.class;
else if (VertexProperty.Cardinality.class.isAssignableFrom(c))
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java
index fec60d2aa8..77a905cc35 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/TraversalSerializersV4.java
@@ -21,12 +21,10 @@ package org.apache.tinkerpop.gremlin.structure.io.graphson;
import org.apache.commons.configuration2.ConfigurationConverter;
import org.apache.commons.configuration2.MapConfiguration;
-import org.apache.tinkerpop.gremlin.process.remote.traversal.DefaultRemoteTraverser;
import org.apache.tinkerpop.gremlin.process.traversal.P;
import org.apache.tinkerpop.gremlin.process.traversal.TextP;
import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
import org.apache.tinkerpop.gremlin.process.traversal.TraversalStrategy;
-import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
import org.apache.tinkerpop.gremlin.process.traversal.step.util.BulkSet;
import org.apache.tinkerpop.gremlin.process.traversal.strategy.TraversalStrategyProxy;
import org.apache.tinkerpop.gremlin.process.traversal.util.AndP;
@@ -159,22 +157,6 @@ final class TraversalSerializersV4 {
}
}
- final static class TraverserJacksonSerializer extends StdScalarSerializer<Traverser> {
-
- public TraverserJacksonSerializer() {
- super(Traverser.class);
- }
-
- @Override
- public void serialize(final Traverser traverserInstance, final JsonGenerator jsonGenerator, final SerializerProvider serializerProvider)
- throws IOException {
- jsonGenerator.writeStartObject();
- jsonGenerator.writeObjectField(GraphSONTokens.BULK, traverserInstance.bulk());
- jsonGenerator.writeObjectField(GraphSONTokens.VALUE, traverserInstance.get());
- jsonGenerator.writeEndObject();
- }
- }
-
final static class TraversalStrategyJacksonSerializer extends StdScalarSerializer<TraversalStrategy> {
public TraversalStrategyJacksonSerializer() {
@@ -400,36 +382,6 @@ final class TraversalSerializersV4 {
}
}
- static class TraverserJacksonDeserializer extends StdDeserializer<Traverser> {
-
- public TraverserJacksonDeserializer() {
- super(Traverser.class);
- }
-
- @Override
- public Traverser deserialize(final JsonParser jsonParser, final DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
- long bulk = 1;
- Object v = null;
-
- while (jsonParser.nextToken() != JsonToken.END_OBJECT) {
- if (jsonParser.getCurrentName().equals(GraphSONTokens.BULK)) {
- jsonParser.nextToken();
- bulk = deserializationContext.readValue(jsonParser, Long.class);
- } else if (jsonParser.getCurrentName().equals(GraphSONTokens.VALUE)) {
- jsonParser.nextToken();
- v = deserializationContext.readValue(jsonParser, Object.class);
- }
- }
-
- return new DefaultRemoteTraverser<>(v, bulk);
- }
-
- @Override
- public boolean isCachable() {
- return true;
- }
- }
-
final static class TraversalStrategyProxyJacksonDeserializer<T extends TraversalStrategy> extends AbstractObjectDeserializer<TraversalStrategyProxy> {
private final Class<T> clazz;
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java
index 94086371dc..872df275e2 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperEmbeddedTypeTest.java
@@ -18,7 +18,6 @@
*/
package org.apache.tinkerpop.gremlin.structure.io.graphson;
-import org.apache.tinkerpop.gremlin.process.remote.traversal.DefaultRemoteTraverser;
import org.apache.tinkerpop.gremlin.process.traversal.Compare;
import org.apache.tinkerpop.gremlin.process.traversal.P;
import org.apache.tinkerpop.gremlin.process.traversal.PBiPredicate;
@@ -242,14 +241,6 @@ public class GraphSONMapperEmbeddedTypeTest extends AbstractGraphSONTest {
assertEquals(o, serializeDeserialize(mapper, o, Lambda.class));
}
- @Test
- public void shouldHandleTraverser() throws Exception {
- assumeThat(version, not(startsWith("v1")));
-
- final Traverser<String> o = new DefaultRemoteTraverser<>("test", 100);
- assertEquals(o, serializeDeserialize(mapper, o, Traverser.class));
- }
-
@Test
public void shouldHandleDuration() throws Exception {
final Duration o = Duration.ZERO;
diff --git a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperPartialEmbeddedTypeTest.java b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperPartialEmbeddedTypeTest.java
index 75331741d2..5315c1a9ec 100644
--- a/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperPartialEmbeddedTypeTest.java
+++ b/gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapperPartialEmbeddedTypeTest.java
@@ -18,7 +18,6 @@
*/
package org.apache.tinkerpop.gremlin.structure.io.graphson;
-import org.apache.tinkerpop.gremlin.process.remote.traversal.DefaultRemoteTraverser;
import org.apache.tinkerpop.gremlin.process.traversal.P;
import org.apache.tinkerpop.gremlin.process.traversal.TextP;
import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
@@ -275,12 +274,6 @@ public class GraphSONMapperPartialEmbeddedTypeTest extends AbstractGraphSONTest
assertNotEquals(myList, read);
}
- @Test
- public void shouldHandleDefaultRemoteTraverser() throws Exception {
- final DefaultRemoteTraverser<String> o = new DefaultRemoteTraverser<>("test", 100);
- assertEquals(o, serializeDeserialize(mapper, o, Traverser.class));
- }
-
@Test
public void shouldHandleVariantsOfP() throws Exception {
final List<P> variantsOfP = Arrays.asList(
diff --git a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/handler/HttpGremlinEndpointHandler.java b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/handler/HttpGremlinEndpointHandler.java
index 040b1e9edf..9aa0fa4f87 100644
--- a/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/handler/HttpGremlinEndpointHandler.java
+++ b/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/handler/HttpGremlinEndpointHandler.java
@@ -61,6 +61,7 @@ import org.apache.tinkerpop.gremlin.util.Tokens;
import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
import org.apache.tinkerpop.gremlin.util.message.RequestMessage;
import org.apache.tinkerpop.gremlin.util.message.ResponseMessage;
+import org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV4;
import org.codehaus.groovy.control.MultipleCompilationErrorsException;
import org.javatuples.Pair;
import org.slf4j.Logger;
@@ -315,7 +316,8 @@ public class HttpGremlinEndpointHandler extends SimpleChannelInboundHandler<Requ
final String bulkingSetting = context.getChannelHandlerContext().channel().attr(StateKey.REQUEST_HEADERS).get().get(Tokens.BULKED);
// bulking only applies if it's gremlin-lang, and per request token setting takes precedence over header setting.
- final boolean bulking = Objects.equals(language, "gremlin-lang") ?
+ // The serializer check is temporarily needed because GraphSON hasn't been removed yet and doesn't support bulking.
+ final boolean bulking = language.equals("gremlin-lang") && serializer instanceof GraphBinaryMessageSerializerV4 ?
(args.containsKey(Tokens.BULKED) ?
Objects.equals(args.get(Tokens.BULKED), "true") :
Objects.equals(bulkingSetting, "true")) :
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
index 6dc11483f1..19d56491af 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
@@ -217,7 +217,12 @@ public class GremlinDriverIntegrateTest extends AbstractGremlinServerIntegration
try {
final Client client = cluster.connect();
- assertEquals(2, client.submit("g.inject(2)").all().get().get(0).getInt());
+ final RequestOptions ro = RequestOptions.build()
+ .addG("gmodern")
+ .language("gremlin-lang")
+ .withBulkedResult(true)
+ .create();
+ assertEquals(6, client.submit("g.V()", ro).all().get().size());
} finally {
cluster.close();
}
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/bulked-traverser-v4.gbin b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/bulked-traverser-v4.gbin
deleted file mode 100644
index 8878bbec8e..0000000000
Binary files a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/bulked-traverser-v4.gbin and /dev/null differ
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/empty-traverser-v4.gbin b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/empty-traverser-v4.gbin
deleted file mode 100644
index c5f3d9af99..0000000000
Binary files a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/empty-traverser-v4.gbin and /dev/null differ
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/vertex-traverser-v4.gbin b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/vertex-traverser-v4.gbin
deleted file mode 100644
index 46e02df795..0000000000
Binary files a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/vertex-traverser-v4.gbin and /dev/null differ
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/bulked-traverser-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/bulked-traverser-v4.json
deleted file mode 100644
index c954e4e9af..0000000000
--- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/bulked-traverser-v4.json
+++ /dev/null
@@ -1,32 +0,0 @@
-{
- "@type" : "g:Traverser",
- "@value" : {
- "bulk" : {
- "@type" : "g:Int64",
- "@value" : 7
- },
- "value" : {
- "@type" : "g:Vertex",
- "@value" : {
- "id" : {
- "@type" : "g:Int32",
- "@value" : 11
- },
- "label" : [ "software" ],
- "properties" : {
- "name" : [ {
- "@type" : "g:VertexProperty",
- "@value" : {
- "id" : {
- "@type" : "g:Int64",
- "@value" : 5
- },
- "value" : "tinkergraph",
- "label" : [ "name" ]
- }
- } ]
- }
- }
- }
- }
-}
\ No newline at end of file
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/empty-traverser-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/empty-traverser-v4.json
deleted file mode 100644
index 08298d82f3..0000000000
--- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/empty-traverser-v4.json
+++ /dev/null
@@ -1,10 +0,0 @@
-{
- "@type" : "g:Traverser",
- "@value" : {
- "bulk" : {
- "@type" : "g:Int64",
- "@value" : 0
- },
- "value" : null
- }
-}
\ No newline at end of file
diff --git a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/vertex-traverser-v4.json b/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/vertex-traverser-v4.json
deleted file mode 100644
index 91dcb6d8ea..0000000000
--- a/gremlin-test/src/main/resources/org/apache/tinkerpop/gremlin/structure/io/graphson/vertex-traverser-v4.json
+++ /dev/null
@@ -1,109 +0,0 @@
-{
- "@type" : "g:Traverser",
- "@value" : {
- "bulk" : {
- "@type" : "g:Int64",
- "@value" : 1
- },
- "value" : {
- "@type" : "g:Vertex",
- "@value" : {
- "id" : {
- "@type" : "g:Int32",
- "@value" : 1
- },
- "label" : [ "person" ],
- "properties" : {
- "name" : [ {
- "@type" : "g:VertexProperty",
- "@value" : {
- "id" : {
- "@type" : "g:Int64",
- "@value" : 0
- },
- "value" : "marko",
- "label" : [ "name" ]
- }
- } ],
- "location" : [ {
- "@type" : "g:VertexProperty",
- "@value" : {
- "id" : {
- "@type" : "g:Int64",
- "@value" : 6
- },
- "value" : "san diego",
- "label" : [ "location" ],
- "properties" : {
- "startTime" : {
- "@type" : "g:Int32",
- "@value" : 1997
- },
- "endTime" : {
- "@type" : "g:Int32",
- "@value" : 2001
- }
- }
- }
- }, {
- "@type" : "g:VertexProperty",
- "@value" : {
- "id" : {
- "@type" : "g:Int64",
- "@value" : 7
- },
- "value" : "santa cruz",
- "label" : [ "location" ],
- "properties" : {
- "startTime" : {
- "@type" : "g:Int32",
- "@value" : 2001
- },
- "endTime" : {
- "@type" : "g:Int32",
- "@value" : 2004
- }
- }
- }
- }, {
- "@type" : "g:VertexProperty",
- "@value" : {
- "id" : {
- "@type" : "g:Int64",
- "@value" : 8
- },
- "value" : "brussels",
- "label" : [ "location" ],
- "properties" : {
- "startTime" : {
- "@type" : "g:Int32",
- "@value" : 2004
- },
- "endTime" : {
- "@type" : "g:Int32",
- "@value" : 2005
- }
- }
- }
- }, {
- "@type" : "g:VertexProperty",
- "@value" : {
- "id" : {
- "@type" : "g:Int64",
- "@value" : 9
- },
- "value" : "santa fe",
- "label" : [ "location" ],
- "properties" : {
- "startTime" : {
- "@type" : "g:Int32",
- "@value" : 2005
- }
- }
- }
- } ]
- }
- }
- }
- }
-}
\ No newline at end of file
diff --git a/gremlin-util/src/main/java/org/apache/tinkerpop/gremlin/util/ser/AbstractGraphSONMessageSerializerV4.java b/gremlin-util/src/main/java/org/apache/tinkerpop/gremlin/util/ser/AbstractGraphSONMessageSerializerV4.java
index 48b0b28bd2..5bca08ebcf 100644
--- a/gremlin-util/src/main/java/org/apache/tinkerpop/gremlin/util/ser/AbstractGraphSONMessageSerializerV4.java
+++ b/gremlin-util/src/main/java/org/apache/tinkerpop/gremlin/util/ser/AbstractGraphSONMessageSerializerV4.java
@@ -335,6 +335,9 @@ public abstract class AbstractGraphSONMessageSerializerV4 extends AbstractMessag
if (data.containsKey(Tokens.ARGS_BATCH_SIZE)) {
builder.addChunkSize((int) data.get(Tokens.ARGS_BATCH_SIZE));
}
+ if (data.containsKey(Tokens.BULKED)) {
+ builder.addBulkedResult(Boolean.parseBoolean(data.get(Tokens.BULKED).toString()));
+ }
return builder.create();
}
diff --git a/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java b/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java
index bd8dee9cb4..5aff82e004 100644
--- a/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java
+++ b/gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractTypedCompatibilityTest.java
@@ -708,54 +708,6 @@ public abstract class AbstractTypedCompatibilityTest extends AbstractCompatibili
IoTest.assertCrewGraph(recycled, false);
}
- @Test
- public void shouldReadWriteVertexTraverser() throws Exception {
- final String resourceName = "vertex-traverser";
-
- final Traverser resource = findModelEntryObject(resourceName);
- final Traverser fromStatic = read(readFromResource(resourceName), Traverser.class);
- final Traverser recycled = read(write(fromStatic, Traverser.class, resourceName), Traverser.class);
- assertNotSame(fromStatic, recycled);
- assertEquals(resource.bulk(), recycled.bulk());
- assertEquals(resource.bulk(), fromStatic.bulk());
- assertEquals(resource.get(), recycled.get());
- assertEquals(resource.get(), fromStatic.get());
- assertVertex((Vertex) resource.get(), (Vertex) recycled.get());
- assertVertex((Vertex) resource.get(), (Vertex) fromStatic.get());
- }
-
- @Test
- public void shouldReadWriteBulkedTraverser() throws Exception {
- final String resourceName = "bulked-traverser";
-
- final Traverser resource = findModelEntryObject(resourceName);
- final Traverser fromStatic = read(readFromResource(resourceName), Traverser.class);
- final Traverser recycled = read(write(fromStatic, Traverser.class, resourceName), Traverser.class);
- assertNotSame(fromStatic, recycled);
- assertEquals(resource.bulk(), recycled.bulk());
- assertEquals(resource.bulk(), fromStatic.bulk());
- assertEquals(resource.get(), recycled.get());
- assertEquals(resource.get(), fromStatic.get());
- assertVertex((Vertex) resource.get(), (Vertex) recycled.get());
- assertVertex((Vertex) resource.get(), (Vertex) fromStatic.get());
- }
-
- @Test
- public void shouldReadWriteEmptyTraverser() throws Exception {
- final String resourceName = "empty-traverser";
-
- final Traverser resource = findModelEntryObject(resourceName);
- final Traverser fromStatic = read(readFromResource(resourceName), Traverser.class);
- final Traverser recycled = read(write(fromStatic, Traverser.class, resourceName), Traverser.class);
- assertNotSame(fromStatic, recycled);
- assertEquals(resource.bulk(), recycled.bulk());
- assertEquals(resource.bulk(), fromStatic.bulk());
- assertEquals(resource.get(), recycled.get());
- assertEquals(resource.get(), fromStatic.get());
- assertEquals(recycled.bulk(), fromStatic.bulk());
- assertEquals(recycled.get(), fromStatic.get());
- }
-
@Test
public void shouldReadWriteUUID() throws Exception {
final String resourceName = "specified-uuid";