You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by sp...@apache.org on 2019/08/19 13:37:04 UTC
[tinkerpop] branch TINKERPOP-2279 updated (807e6af -> 34d26b0)
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a change to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git.
discard 807e6af Refactored python graphbinary to stop using special means of string conversion
discard c3dc0c4 Generalized refactoring to python graphbinary.
discard 4d5deff Renamed python graphbinary "IO" to der or ser depending on function
discard c326d6c All GLV tests passing for GraphBinary
discard 9e37ecc Fixed up BulkSet python graphbinary serialization
discard 317ad54 Cleaned up enum derser for python graphbinary a bit further
discard f27dbfd Fixed lambda serialization in python graphbinary
discard 7006a96 Fixed handling of the null type in python graphbinary
discard 36d8d51 Fixed a basic integration test runs for python 2.
discard d71196d Added fqcn to TraversalStrategy in python to allow GraphBinary serialization to work properly
discard 88060f8 Major refactoring for nullable writes.
discard 036e566 Improved null handling in graphbinary for python
discard 9adcb63 Major refactoring of GraphBinary
discard 7484b56 Added graphbinary test files in gremlin-io-test for 3.4.3
discard bca51a9 Added BulkSet support to python graphbinary
discard de50187 Added boolean and textp to python graphbinary
discard b6ba972 Added traverser, byte, bytearray to python graphbinary
discard 6f2aef1 Added P/T/Scope graphbinary serialization for python
discard 988ca0f Added bytecode graphbinary serialization to python
discard b53b753 Aded Binding and fixed up enum derser in python graphbinary
discard cf24ec2 Add enum graphbinary support in python
discard f2d56e9 Added Graph/Vertex/VertexProperty to python graphbinary
discard 9019ca2 Added property/path to python graphbinary
discard 2368396 Added uuid/edge to python graphbinary derser
discard a9ed4dc Added Dict/Set serialization for graphbinary in python
discard 98492b2 Added double and float graphbinary serialization in python
discard 6cd3ef5 Fixed the graphbinary g;Date serialization - wasn't account for timezone stuff properly (still)
discard 24882df Added timestamp for graphbinary in python
discard 69d2388 Added date graphbinary type to python
discard 45f86a1 Added Long graphbinary support to python
discard 7127e12 Added int graphbinary serialization for python
discard 3602821 Add graphbinary support for list in python
discard b9f2116 Initial setup for GraphBinary in Python
add ebba19f TINKERPOP-2275 bump jackson-databind 2.9.9.3 - CTR
add b35eb2b Merge branch 'tp33' into tp34
add b9f010f Minor changelog fixup
add fd4a44b Merge branch 'tp33' into tp34
add ef6e951 Postpone the timing of transport creation to `connection.write`
add d998dbf Merge branch 'pr-1177' into tp33
add 68b7eea Merge branch 'tp33' into tp34
add 7625874 Added js/.net docs to release announcement email template CTR
add bff78f1 Merge branch 'tp33' into tp34
add 74e0644 Use traversal() AnonymousTraversalSource in archetype CTR
add cf97903 Merge branch 'tp33' into tp34
add 229da54 Mentioned ReferenceElementStrategy in the docs to better clarify the types of elements returned CTR
new 391efe0 Initial setup for GraphBinary in Python
new 75ed2d0 Add graphbinary support for list in python
new 255c7c2 Added int graphbinary serialization for python
new c4c38d3 Added Long graphbinary support to python
new e2b732c Added date graphbinary type to python
new 6693c9c Added timestamp for graphbinary in python
new 04b091b Fixed the graphbinary g;Date serialization - wasn't account for timezone stuff properly (still)
new 7c01e7a Added double and float graphbinary serialization in python
new 3bcaf23 Added Dict/Set serialization for graphbinary in python
new 9cd7f18 Added uuid/edge to python graphbinary derser
new 2623402 Added property/path to python graphbinary
new f498886 Added Graph/Vertex/VertexProperty to python graphbinary
new bbc9bfa Add enum graphbinary support in python
new b0fa0fb Aded Binding and fixed up enum derser in python graphbinary
new 3ab7929 Added bytecode graphbinary serialization to python
new c4369e9 Added P/T/Scope graphbinary serialization for python
new 89e9dd7 Added traverser, byte, bytearray to python graphbinary
new e2cf80f Added boolean and textp to python graphbinary
new 8cda34c Added BulkSet support to python graphbinary
new bca007d Added graphbinary test files in gremlin-io-test for 3.4.3
new 65da362 Major refactoring of GraphBinary
new 8543b87 Improved null handling in graphbinary for python
new a728648 Major refactoring for nullable writes.
new 602edf8 Added fqcn to TraversalStrategy in python to allow GraphBinary serialization to work properly
new f14606b Fixed a basic integration test runs for python 2.
new af5a299 Fixed handling of the null type in python graphbinary
new db2acaa Fixed lambda serialization in python graphbinary
new 221c2a4 Cleaned up enum derser for python graphbinary a bit further
new 9fd5b56 Fixed up BulkSet python graphbinary serialization
new ff92f9b All GLV tests passing for GraphBinary
new eb5e239 Renamed python graphbinary "IO" to der or ser depending on function
new 8de895b Generalized refactoring to python graphbinary.
new 34d26b0 Refactored python graphbinary to stop using special means of string conversion
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (807e6af)
\
N -- N -- N refs/heads/TINKERPOP-2279 (34d26b0)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 33 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG.asciidoc | 2 ++
docs/src/dev/developer/release.asciidoc | 2 ++
docs/src/reference/gremlin-applications.asciidoc | 14 ++++++++++++++
.../archetype-resources/src/test/java/SocialDslTest.java | 6 ++----
.../src/main/jython/gremlin_python/driver/connection.py | 8 ++++++--
gremlin-shaded/pom.xml | 2 +-
6 files changed, 27 insertions(+), 7 deletions(-)
[tinkerpop] 15/33: Added bytecode graphbinary serialization to
python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 3ab7929a4540fdf1e8167ae0734ea7c7b45a9519
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Thu Aug 1 11:30:25 2019 -0400
Added bytecode graphbinary serialization to python
---
.../gremlin_python/structure/io/graphbinaryV1.py | 60 +++++++++++++++++++++-
.../tests/structure/io/test_graphbinaryV1.py | 12 ++++-
.../jython/tests/structure/io/test_graphsonV3d0.py | 3 +-
3 files changed, 71 insertions(+), 4 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 8eb323b..ec8b02d 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -71,8 +71,8 @@ class DataType(Enum):
vertex = 0x11
vertexproperty = 0x12
barrier = 0x13
- binding = 0x14
- bytecode = 0x15 # todo
+ binding = 0x14
+ bytecode = 0x15
cardinality = 0x16
column = 0x17
direction = 0x18
@@ -588,3 +588,59 @@ class BindingIO(_GraphBinaryTypeIO):
def objectify(cls, b, reader):
return Binding(cls.read_string(b), reader.readObject(b))
+
+class BytecodeIO(_GraphBinaryTypeIO):
+ python_type = Bytecode
+ graphbinary_type = DataType.bytecode
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(struct.pack(">i", len(obj.step_instructions)))
+ for inst in obj.step_instructions:
+ inst_name, inst_args = inst[0], inst[1:] if len(inst) > 1 else []
+ ba.extend(cls.string_as_bytes(inst_name))
+ ba.extend(struct.pack(">i", len(inst_args)))
+ for arg in inst_args:
+ ba.extend(writer.writeObject(arg))
+
+ ba.extend(struct.pack(">i", len(obj.source_instructions)))
+ for inst in obj.source_instructions:
+ inst_name, inst_args = inst[0], inst[1:] if len(inst) > 1 else []
+ ba.extend(cls.string_as_bytes(inst_name))
+ ba.extend(struct.pack(">i", len(inst_args)))
+ for arg in inst_args:
+ ba.extend(writer.writeObject(arg))
+
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ bytecode = Bytecode()
+
+ step_count = cls.read_int(b)
+ ix = 0
+ while ix < step_count:
+ inst = [cls.read_string(b)]
+ inst_ct = cls.read_int(b)
+ iy = 0
+ while iy < inst_ct:
+ inst.append(reader.readObject(b))
+ iy += 1
+ bytecode.step_instructions.append(inst)
+ ix += 1
+
+ source_count = cls.read_int(b)
+ ix = 0
+ while ix < source_count:
+ inst = [cls.read_string(b)]
+ inst_ct = cls.read_int(b)
+ iy = 0
+ while iy < inst_ct:
+ inst.append(reader.readObject(b))
+ iy += 1
+ bytecode.source_instructions.append(inst)
+ ix += 1
+
+ return bytecode
+
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index f3ee2c8..c9ce87e 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -31,7 +31,7 @@ import six
from gremlin_python.statics import timestamp, long
from gremlin_python.structure.graph import Vertex, Edge, Property, VertexProperty, Graph, Path
from gremlin_python.structure.io.graphbinaryV1 import GraphBinaryWriter, GraphBinaryReader, DataType
-from gremlin_python.process.traversal import P, Barrier, Binding
+from gremlin_python.process.traversal import P, Barrier, Binding, Bytecode
from gremlin_python.process.strategies import SubgraphStrategy
from gremlin_python.process.graph_traversal import __
@@ -159,3 +159,13 @@ class TestGraphSONWriter(object):
x = Binding("name", "marko")
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+
+ def test_bytecode(self):
+ x = Bytecode()
+ x.source_instructions.append(["withStrategies", "SubgraphStrategy"])
+ x.step_instructions.append(["V", 1, 2, 3])
+ x.step_instructions.append(["out"])
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphsonV3d0.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphsonV3d0.py
index 2b9331f..7cc8d68 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphsonV3d0.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphsonV3d0.py
@@ -409,7 +409,8 @@ class TestGraphSONWriter(object):
assert result == json.loads(self.graphson_writer.writeObject(P.within(1)))
def test_strategies(self):
- # we have a proxy model for now given that we don't want to have to have g:XXX all registered on the Gremlin traversal machine (yet)
+ # we have a proxy model for now given that we don't want to have to have g:XXX all registered on the
+ # Gremlin traversal machine (yet)
assert {"@type": "g:SubgraphStrategy", "@value": {}} == json.loads(
self.graphson_writer.writeObject(SubgraphStrategy))
assert {"@type": "g:SubgraphStrategy", "@value": {
[tinkerpop] 16/33: Added P/T/Scope graphbinary serialization for
python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit c4369e9addac2bfa8fe82d45df3d3c5528aba513
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Thu Aug 1 13:54:46 2019 -0400
Added P/T/Scope graphbinary serialization for python
---
docs/src/dev/io/graphbinary.asciidoc | 2 +-
gremlin-python/glv/TraversalSource.template | 2 +-
.../gremlin_python/structure/io/graphbinaryV1.py | 61 +++++++++++++++++++++-
3 files changed, 62 insertions(+), 3 deletions(-)
diff --git a/docs/src/dev/io/graphbinary.asciidoc b/docs/src/dev/io/graphbinary.asciidoc
index 5c8f818..5fa4afe 100644
--- a/docs/src/dev/io/graphbinary.asciidoc
+++ b/docs/src/dev/io/graphbinary.asciidoc
@@ -423,7 +423,7 @@ Where:
==== P
-Format: `{predicate}{values_length}{value_0}...{value_n}`
+Format: `{name}{values_length}{value_0}...{value_n}`
Where:
diff --git a/gremlin-python/glv/TraversalSource.template b/gremlin-python/glv/TraversalSource.template
index 8a8b04d..8f776d2 100644
--- a/gremlin-python/glv/TraversalSource.template
+++ b/gremlin-python/glv/TraversalSource.template
@@ -394,7 +394,7 @@ class Bindings(object):
class Binding(object):
- def __init__(self,key,value):
+ def __init__(self, key, value):
self.key = key
self.value = value
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index ec8b02d..edb1136 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -38,7 +38,8 @@ from gremlin_python import statics
from gremlin_python.statics import FloatType, FunctionType, IntType, LongType, TypeType, DictType, ListType, SetType, \
SingleByte, ByteBufferType, SingleChar
from gremlin_python.process.traversal import Barrier, Binding, Bytecode, Cardinality, Column, Direction, Operator, \
- Order, Pick, Pop, P, TextP, Traversal, Traverser, TraversalStrategy, T
+ Order, Pick, Pop, P, Scope, TextP, Traversal, Traverser, \
+ TraversalStrategy, T
from gremlin_python.structure.graph import Graph, Edge, Property, Vertex, VertexProperty, Path
log = logging.getLogger(__name__)
@@ -80,6 +81,10 @@ class DataType(Enum):
order = 0x1a
pick = 0x1b
pop = 0x1c
+ lambda_ = 0x1d
+ p = 0x1e
+ scope = 0x1f
+ t = 0x20
class GraphBinaryTypeType(type):
@@ -644,3 +649,57 @@ class BytecodeIO(_GraphBinaryTypeIO):
return bytecode
+
+class LambdaIO(_GraphBinaryTypeIO):
+
+ python_type = FunctionType
+ graphbinary_type = DataType.lambda_
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ lambda_result = obj()
+ script = lambda_result if isinstance(lambda_result, str) else lambda_result[0]
+ language = statics.default_lambda_language if isinstance(lambda_result, str) else lambda_result[1]
+
+ ba.extend(cls.string_as_bytes(language))
+
+ script_cleaned = script
+ script_args = -1
+
+ if language == "gremlin-jython" or language == "gremlin-python":
+ if not script.strip().startswith("lambda"):
+ script_cleaned = "lambda " + script
+ script_args = six.get_function_code(eval(script_cleaned)).co_argcount
+
+ ba.extend(cls.string_as_bytes(script_cleaned))
+ ba.extend(struct.pack(">i", script_args))
+
+ return ba
+
+
+class PIO(_GraphBinaryTypeIO):
+ graphbinary_type = DataType.p
+ python_type = P
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(cls.string_as_bytes(obj.operator))
+ additional = [writer.writeObject(obj.value), writer.writeObject(obj.other)] \
+ if obj.other is not None else [writer.writeObject(obj.value)]
+ ba.extend(struct.pack(">i", len(additional)))
+ for a in additional:
+ ba.extend(a)
+
+ return ba
+
+
+class ScopeIO(_EnumIO):
+ graphbinary_type = DataType.scope
+ python_type = Scope
+
+
+class TIO(_EnumIO):
+ graphbinary_type = DataType.t
+ python_type = T
\ No newline at end of file
[tinkerpop] 20/33: Added graphbinary test files in gremlin-io-test
for 3.4.3
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit bca007d9f79f9e31965c14d85d1a1f4cf26e0fd3
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Aug 2 10:50:00 2019 -0400
Added graphbinary test files in gremlin-io-test for 3.4.3
---
.../jython/gremlin_python/process/traversal.py | 2 +-
gremlin-tools/gremlin-io-test/pom.xml | 26 ++++++
.../scripts/generate-graphbinary-resources.groovy | 81 +++++++++++++++++++
.../tinkerpop/gremlin/structure/io/Model.java | 16 +++-
.../io/graphbinary/GraphBinaryCompatibility.java | 86 ++++++++++++++++++++
.../structure/io/AbstractCompatibilityTest.java | 51 +++++++-----
.../graphbinary/GraphBinaryCompatibilityTest.java | 90 +++++++++++++++++++++
.../io/graphbinary/_3_4_3/barrier-v1.gbin | Bin 0 -> 16 bytes
.../io/graphbinary/_3_4_3/bigdecimal-v1.gbin | Bin 0 -> 25 bytes
.../io/graphbinary/_3_4_3/biginteger-v1.gbin | Bin 0 -> 21 bytes
.../io/graphbinary/_3_4_3/binding-v1.gbin | Bin 0 -> 13 bytes
.../io/graphbinary/_3_4_3/bulkset-v1.gbin | Bin 0 -> 43 bytes
.../structure/io/graphbinary/_3_4_3/byte-v1.gbin | Bin 0 -> 3 bytes
.../io/graphbinary/_3_4_3/bytebuffer-v1.gbin | Bin 0 -> 24 bytes
.../io/graphbinary/_3_4_3/bytecode-v1.gbin | Bin 0 -> 80 bytes
.../io/graphbinary/_3_4_3/cardinality-v1.gbin | Bin 0 -> 12 bytes
.../structure/io/graphbinary/_3_4_3/char-v1.gbin | Bin 0 -> 3 bytes
.../structure/io/graphbinary/_3_4_3/class-v1.gbin | Bin 0 -> 18 bytes
.../structure/io/graphbinary/_3_4_3/column-v1.gbin | Bin 0 -> 12 bytes
.../structure/io/graphbinary/_3_4_3/date-v1.gbin | Bin 0 -> 10 bytes
.../io/graphbinary/_3_4_3/direction-v1.gbin | Bin 0 -> 11 bytes
.../structure/io/graphbinary/_3_4_3/double-v1.gbin | Bin 0 -> 10 bytes
.../io/graphbinary/_3_4_3/duration-v1.gbin | Bin 0 -> 14 bytes
.../structure/io/graphbinary/_3_4_3/edge-v1.gbin | Bin 0 -> 58 bytes
.../structure/io/graphbinary/_3_4_3/float-v1.gbin | Bin 0 -> 6 bytes
.../io/graphbinary/_3_4_3/inetaddress-v1.gbin | Bin 0 -> 10 bytes
.../io/graphbinary/_3_4_3/instant-v1.gbin | Bin 0 -> 14 bytes
.../io/graphbinary/_3_4_3/integer-v1.gbin | Bin 0 -> 6 bytes
.../structure/io/graphbinary/_3_4_3/lambda-v1.gbin | Bin 0 -> 40 bytes
.../structure/io/graphbinary/_3_4_3/list-v1.gbin | Bin 0 -> 27 bytes
.../io/graphbinary/_3_4_3/localdate-v1.gbin | Bin 0 -> 8 bytes
.../io/graphbinary/_3_4_3/localdatetime-v1.gbin | Bin 0 -> 16 bytes
.../io/graphbinary/_3_4_3/localtime-v1.gbin | Bin 0 -> 10 bytes
.../structure/io/graphbinary/_3_4_3/long-v1.gbin | Bin 0 -> 10 bytes
.../structure/io/graphbinary/_3_4_3/map-v1.gbin | Bin 0 -> 75 bytes
.../io/graphbinary/_3_4_3/metrics-v1.gbin | Bin 0 -> 307 bytes
.../io/graphbinary/_3_4_3/monthday-v1.gbin | Bin 0 -> 4 bytes
.../io/graphbinary/_3_4_3/offsetdatetime-v1.gbin | Bin 0 -> 20 bytes
.../io/graphbinary/_3_4_3/offsettime-v1.gbin | Bin 0 -> 14 bytes
.../io/graphbinary/_3_4_3/operator-v1.gbin | Bin 0 -> 11 bytes
.../structure/io/graphbinary/_3_4_3/order-v1.gbin | Bin 0 -> 15 bytes
.../structure/io/graphbinary/_3_4_3/p-v1.gbin | Bin 0 -> 18 bytes
.../structure/io/graphbinary/_3_4_3/pand-v1.gbin | Bin 0 -> 49 bytes
.../structure/io/graphbinary/_3_4_3/path-v1.gbin | Bin 0 -> 96 bytes
.../structure/io/graphbinary/_3_4_3/period-v1.gbin | Bin 0 -> 14 bytes
.../structure/io/graphbinary/_3_4_3/pick-v1.gbin | Bin 0 -> 11 bytes
.../structure/io/graphbinary/_3_4_3/pop-v1.gbin | Bin 0 -> 11 bytes
.../structure/io/graphbinary/_3_4_3/por-v1.gbin | Bin 0 -> 64 bytes
.../io/graphbinary/_3_4_3/property-v1.gbin | Bin 0 -> 19 bytes
.../io/graphbinary/_3_4_3/pwithin-v1.gbin | Bin 0 -> 22 bytes
.../io/graphbinary/_3_4_3/pwithout-v1.gbin | Bin 0 -> 29 bytes
.../structure/io/graphbinary/_3_4_3/scope-v1.gbin | Bin 0 -> 13 bytes
.../structure/io/graphbinary/_3_4_3/set-v1.gbin | Bin 0 -> 27 bytes
.../structure/io/graphbinary/_3_4_3/short-v1.gbin | Bin 0 -> 4 bytes
.../structure/io/graphbinary/_3_4_3/t-v1.gbin | Bin 0 -> 13 bytes
.../structure/io/graphbinary/_3_4_3/textp-v1.gbin | Bin 0 -> 29 bytes
.../io/graphbinary/_3_4_3/timestamp-v1.gbin | Bin 0 -> 10 bytes
.../io/graphbinary/_3_4_3/tinkergraph-v1.gbin | Bin 0 -> 2264 bytes
.../io/graphbinary/_3_4_3/traversalmetrics-v1.gbin | Bin 0 -> 593 bytes
.../io/graphbinary/_3_4_3/traverser-v1.gbin | Bin 0 -> 30 bytes
.../structure/io/graphbinary/_3_4_3/uuid-v1.gbin | Bin 0 -> 18 bytes
.../structure/io/graphbinary/_3_4_3/vertex-v1.gbin | Bin 0 -> 20 bytes
.../io/graphbinary/_3_4_3/vertexproperty-v1.gbin | Bin 0 -> 35 bytes
.../structure/io/graphbinary/_3_4_3/year-v1.gbin | Bin 0 -> 6 bytes
.../io/graphbinary/_3_4_3/yearmonth-v1.gbin | Bin 0 -> 7 bytes
.../io/graphbinary/_3_4_3/zoneddatetime-v1.gbin | Bin 0 -> 20 bytes
.../io/graphbinary/_3_4_3/zoneoffset-v1.gbin | Bin 0 -> 6 bytes
pom.xml | 1 +
68 files changed, 330 insertions(+), 23 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/process/traversal.py b/gremlin-python/src/main/jython/gremlin_python/process/traversal.py
index dbc470f..4fb9f8c 100644
--- a/gremlin-python/src/main/jython/gremlin_python/process/traversal.py
+++ b/gremlin-python/src/main/jython/gremlin_python/process/traversal.py
@@ -678,7 +678,7 @@ class Bindings(object):
class Binding(object):
- def __init__(self,key,value):
+ def __init__(self, key, value):
self.key = key
self.value = value
diff --git a/gremlin-tools/gremlin-io-test/pom.xml b/gremlin-tools/gremlin-io-test/pom.xml
index fc0e2a5..dcef57c 100644
--- a/gremlin-tools/gremlin-io-test/pom.xml
+++ b/gremlin-tools/gremlin-io-test/pom.xml
@@ -147,6 +147,32 @@
</configuration>
</execution>
<execution>
+ <id>generate-io-files-graphbinary</id>
+ <phase>generate-test-resources</phase>
+ <goals>
+ <goal>execute</goal>
+ </goals>
+ <configuration>
+ <properties>
+ <property>
+ <name>projectBuildDir</name>
+ <value>${project.build.directory}</value>
+ </property>
+ <property>
+ <name>projectBaseDir</name>
+ <value>${project.build.directory}</value>
+ </property>
+ <property>
+ <name>projectVersion</name>
+ <value>${project.version}</value>
+ </property>
+ </properties>
+ <scripts>
+ <script>${project.basedir}/scripts/generate-graphbinary-resources.groovy</script>
+ </scripts>
+ </configuration>
+ </execution>
+ <execution>
<id>generate-io-model-csv</id>
<phase>generate-test-resources</phase>
<goals>
diff --git a/gremlin-tools/gremlin-io-test/scripts/generate-graphbinary-resources.groovy b/gremlin-tools/gremlin-io-test/scripts/generate-graphbinary-resources.groovy
new file mode 100644
index 0000000..43eeb0d
--- /dev/null
+++ b/gremlin-tools/gremlin-io-test/scripts/generate-graphbinary-resources.groovy
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import io.netty.buffer.ByteBufAllocator
+import org.apache.tinkerpop.gremlin.driver.ser.binary.GraphBinaryWriter
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.*
+import org.apache.tinkerpop.gremlin.structure.*
+import org.apache.tinkerpop.gremlin.structure.io.*
+import org.apache.commons.configuration.BaseConfiguration
+
+new File("${projectBuildDir}/test-case-data/io/graphbinary").mkdirs()
+
+conf = new BaseConfiguration()
+conf.setProperty(TinkerGraph.GREMLIN_TINKERGRAPH_DEFAULT_VERTEX_PROPERTY_CARDINALITY, VertexProperty.Cardinality.list.name())
+graph = TinkerGraph.open(conf)
+TinkerFactory.generateTheCrew(graph)
+g = graph.traversal()
+
+model = Model.instance()
+
+allocator = ByteBufAllocator.DEFAULT
+
+toGraphBinary = { o, type, mapper, suffix = "" ->
+ def fileToWriteTo = new File("${projectBuildDir}/test-case-data/io/graphbinary/" + type.title.toLowerCase().replace(" ","") + "-" + suffix + ".gbin")
+ if (fileToWriteTo.exists()) fileToWriteTo.delete()
+ filestream = new FileOutputStream(fileToWriteTo)
+ try {
+ buffer = allocator.buffer()
+ writer.write(o, buffer)
+ buffer.readerIndex(0)
+ buffer.readBytes(filestream, buffer.readableBytes())
+ } catch (Exception ex) {
+ if (ex.message == "Serializer for type org.apache.tinkerpop.gremlin.driver.message.RequestMessage not found" ||
+ ex.message == "Serializer for type org.apache.tinkerpop.gremlin.driver.message.ResponseMessage not found" )
+ fileToWriteTo.delete()
+ else
+ throw ex
+ } finally {
+ filestream.close()
+ }
+}
+
+toGraphBinaryV1d0 = { o, type, mapper ->
+ toGraphBinary(o, type, mapper, "v1")
+}
+
+writeSupportedObjects = { mapper, toGraphBinaryFunction ->
+ model.entries().findAll{it.hasGraphBinaryCompatibility()}.each {
+ toGraphBinaryFunction(it.getObject(), it, mapper)
+ }
+}
+
+writer = new GraphBinaryWriter()
+
+writeSupportedObjects(writer, toGraphBinaryV1d0)
+
+def ver = "_" + "${projectVersion}".replace(".","_").replace("-SNAPSHOT","")
+def target = "${projectBaseDir}/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/" + ver
+def targetDir = new File(target)
+if (!targetDir.exists()) targetDir.mkdirs()
+new File("${projectBuildDir}/test-case-data/io/graphbinary/").listFiles().each {
+ def copyTo = new File(target + "/" + it.name)
+ if (copyTo.exists()) copyTo.delete()
+ java.nio.file.Files.copy(it.toPath(), new File(target + "/" + it.name).toPath())
+}
diff --git a/gremlin-tools/gremlin-io-test/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Model.java b/gremlin-tools/gremlin-io-test/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Model.java
index 4cea315..ef939c4 100644
--- a/gremlin-tools/gremlin-io-test/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Model.java
+++ b/gremlin-tools/gremlin-io-test/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Model.java
@@ -40,6 +40,7 @@ import org.apache.tinkerpop.gremlin.structure.Column;
import org.apache.tinkerpop.gremlin.structure.Direction;
import org.apache.tinkerpop.gremlin.structure.T;
import org.apache.tinkerpop.gremlin.structure.VertexProperty;
+import org.apache.tinkerpop.gremlin.structure.io.graphbinary.GraphBinaryCompatibility;
import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONCompatibility;
import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoCompatibility;
import org.apache.tinkerpop.gremlin.structure.util.star.StarGraph;
@@ -92,6 +93,7 @@ public class Model {
private static final List<Compatibility> ALL = Collections.unmodifiableList(new ArrayList<Compatibility>() {{
addAll(Arrays.asList(GraphSONCompatibility.values()));
addAll(Arrays.asList(GryoCompatibility.values()));
+ addAll(Arrays.asList(GraphBinaryCompatibility.values()));
}});
private static final Model model = new Model();
@@ -105,6 +107,10 @@ public class Model {
TinkerFactory.generateTheCrew(graph);
final GraphTraversalSource g = graph.traversal();
+ // TODO: gotta fix graphbinary
+ final Compatibility[] noTypeGraphSONPlusBrokenGraphBinary = Compatibilities.with(GraphSONCompatibility.class)
+ .configuredAs(".*no-types|v1d0").join(Compatibilities.with(GraphBinaryCompatibility.class)).matchToArray();
+
final Compatibility[] noTypeGraphSONPlusGryo3_2_3 = Compatibilities.with(GryoCompatibility.class).beforeRelease("3.2.4").join(Compatibilities.UNTYPED_GRAPHSON).matchToArray();
final Compatibility[] noTypeGraphSONPlusGryo3_3_0 = Compatibilities.with(GryoCompatibility.class).beforeRelease("3.3.0").join(Compatibilities.UNTYPED_GRAPHSON).matchToArray();
@@ -253,7 +259,7 @@ public class Model {
addExtendedEntry(new Short("100"), "Short", "", Compatibilities.UNTYPED_GRAPHSON.matchToArray());
addExtendedEntry(Year.of(2016), "Year", "The following example is of the `Year` \"2016\".", Compatibilities.UNTYPED_GRAPHSON.matchToArray());
addExtendedEntry(YearMonth.of(2016, 6), "YearMonth", "The following example is a `YearMonth` of \"June 2016\"", Compatibilities.UNTYPED_GRAPHSON.matchToArray());
- addExtendedEntry(ZonedDateTime.of(2016, 12, 23, 12, 12, 24, 36, ZoneId.of("GMT+2")), "ZonedDateTime", "", Compatibilities.UNTYPED_GRAPHSON.matchToArray());
+ addExtendedEntry(ZonedDateTime.of(2016, 12, 23, 12, 12, 24, 36, ZoneId.of("GMT+2")), "ZonedDateTime", "", noTypeGraphSONPlusBrokenGraphBinary);
addExtendedEntry(ZoneOffset.ofHoursMinutesSeconds(3, 6, 9), "ZoneOffset", "The following example is a `ZoneOffset` of three hours, six minutes, and nine seconds.", Compatibilities.UNTYPED_GRAPHSON.matchToArray());
}
@@ -338,6 +344,7 @@ public class Model {
private void addRequestMessageEntry(final Object obj, final String title, final String description) {
final List<Compatibility> incompatibilityList = Compatibilities.with(GryoCompatibility.class)
.before("3.0")
+ .join(Compatibilities.with(GraphBinaryCompatibility.class))
.match();
final Compatibility[] incompatibilities = new Compatibility[incompatibilityList.size()];
@@ -348,6 +355,7 @@ public class Model {
private void addResponseMessageEntry(final Object obj, final String title, final String description) {
final List<Compatibility> incompatibilityList = Compatibilities.with(GryoCompatibility.class)
.before("3.0")
+ .join(Compatibilities.with(GraphBinaryCompatibility.class))
.match();
// TODO: temporary problem? seems to be something breaking in vertex serialization
@@ -421,6 +429,8 @@ public class Model {
return "gryo-" + ((GryoCompatibility) c).name();
else if (c instanceof GraphSONCompatibility)
return "graphson-" + ((GraphSONCompatibility) c).name();
+ else if (c instanceof GraphBinaryCompatibility)
+ return "graphbinary-" + ((GraphBinaryCompatibility) c).name();
else
throw new IllegalStateException("No support for the provided Compatibility type");
}).collect(Collectors.toList()));
@@ -485,5 +495,9 @@ public class Model {
public boolean hasGraphSONCompatibility() {
return compatibleWith.stream().anyMatch(c -> c instanceof GryoCompatibility);
}
+
+ public boolean hasGraphBinaryCompatibility() {
+ return compatibleWith.stream().anyMatch(c -> c instanceof GraphBinaryCompatibility);
+ }
}
}
diff --git a/gremlin-tools/gremlin-io-test/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphbinary/GraphBinaryCompatibility.java b/gremlin-tools/gremlin-io-test/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphbinary/GraphBinaryCompatibility.java
new file mode 100644
index 0000000..8c7f428
--- /dev/null
+++ b/gremlin-tools/gremlin-io-test/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphbinary/GraphBinaryCompatibility.java
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.structure.io.graphbinary;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.tinkerpop.gremlin.process.traversal.Path;
+import org.apache.tinkerpop.gremlin.process.traversal.Traverser;
+import org.apache.tinkerpop.gremlin.process.traversal.traverser.B_O_Traverser;
+import org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversalMetrics;
+import org.apache.tinkerpop.gremlin.process.traversal.util.Metrics;
+import org.apache.tinkerpop.gremlin.process.traversal.util.MutableMetrics;
+import org.apache.tinkerpop.gremlin.process.traversal.util.TraversalMetrics;
+import org.apache.tinkerpop.gremlin.structure.Edge;
+import org.apache.tinkerpop.gremlin.structure.Property;
+import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.apache.tinkerpop.gremlin.structure.VertexProperty;
+import org.apache.tinkerpop.gremlin.structure.io.Compatibility;
+import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedEdge;
+import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedPath;
+import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedProperty;
+import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedVertex;
+import org.apache.tinkerpop.gremlin.structure.util.detached.DetachedVertexProperty;
+
+import java.io.File;
+import java.io.IOException;
+
+/**
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public enum GraphBinaryCompatibility implements Compatibility {
+ V1_3_4_3("3.4.3", "1.0", "v1");
+
+ private static final String SEP = File.separator;
+
+ private final String graphBinaryVersion;
+ private final String tinkerpopVersion;
+ private final String configuration;
+
+ GraphBinaryCompatibility(final String tinkerpopVersion, final String graphBinaryVersion, final String configuration) {
+ this.tinkerpopVersion = tinkerpopVersion;
+ this.graphBinaryVersion = graphBinaryVersion;
+ this.configuration = configuration;
+ }
+
+ @Override
+ public byte[] readFromResource(final String resource) throws IOException {
+ final String testResource = "_" + tinkerpopVersion.replace(".", "_") + SEP + resource + "-" + configuration + ".gbin";
+ return IOUtils.toByteArray(getClass().getResourceAsStream(testResource));
+ }
+
+ @Override
+ public String getReleaseVersion() {
+ return tinkerpopVersion;
+ }
+
+ @Override
+ public String getVersion() {
+ return graphBinaryVersion;
+ }
+
+ @Override
+ public String getConfiguration() {
+ return configuration;
+ }
+
+ @Override
+ public String toString() {
+ return tinkerpopVersion + "-" + configuration;
+ }
+}
diff --git a/gremlin-tools/gremlin-io-test/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractCompatibilityTest.java b/gremlin-tools/gremlin-io-test/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractCompatibilityTest.java
index 99f2de0..b3c2266 100644
--- a/gremlin-tools/gremlin-io-test/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractCompatibilityTest.java
+++ b/gremlin-tools/gremlin-io-test/src/test/java/org/apache/tinkerpop/gremlin/structure/io/AbstractCompatibilityTest.java
@@ -22,6 +22,7 @@ import org.apache.tinkerpop.gremlin.structure.Edge;
import org.apache.tinkerpop.gremlin.structure.Property;
import org.apache.tinkerpop.gremlin.structure.Vertex;
import org.apache.tinkerpop.gremlin.structure.VertexProperty;
+import org.apache.tinkerpop.gremlin.structure.io.graphbinary.GraphBinaryCompatibility;
import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
import java.util.Iterator;
@@ -51,17 +52,20 @@ public abstract class AbstractCompatibilityTest {
protected void assertVertex(final Vertex expected, final Vertex actual) {
assertEquals(expected.id(), actual.id());
assertEquals(expected.label(), actual.label());
- assertEquals(IteratorUtils.count(expected.properties()), IteratorUtils.count(actual.properties()));
- for (String k : expected.keys()) {
- final Iterator<VertexProperty<Object>> expectedVps = expected.properties(k);
- final List<VertexProperty<Object>> actualVps = IteratorUtils.list(actual.properties(k));
- while (expectedVps.hasNext()) {
- final VertexProperty expectedVp = expectedVps.next();
- final VertexProperty<Object> found = actualVps.stream()
- .filter(vp -> vp.id().equals(expectedVp.id()))
- .findFirst()
- .orElseThrow(() -> new RuntimeException("Could not find VertexProperty for " + expectedVp.id()));
- assertVertexProperty(expectedVp, found);
+
+ if (!(getCompatibility() instanceof GraphBinaryCompatibility)) {
+ assertEquals(IteratorUtils.count(expected.properties()), IteratorUtils.count(actual.properties()));
+ for (String k : expected.keys()) {
+ final Iterator<VertexProperty<Object>> expectedVps = expected.properties(k);
+ final List<VertexProperty<Object>> actualVps = IteratorUtils.list(actual.properties(k));
+ while (expectedVps.hasNext()) {
+ final VertexProperty expectedVp = expectedVps.next();
+ final VertexProperty<Object> found = actualVps.stream()
+ .filter(vp -> vp.id().equals(expectedVp.id()))
+ .findFirst()
+ .orElseThrow(() -> new RuntimeException("Could not find VertexProperty for " + expectedVp.id()));
+ assertVertexProperty(expectedVp, found);
+ }
}
}
}
@@ -69,26 +73,31 @@ public abstract class AbstractCompatibilityTest {
protected void assertEdge(final Edge expected, final Edge actual) {
assertEquals(expected.id(), actual.id());
assertEquals(expected.label(), actual.label());
- assertEquals(IteratorUtils.count(expected.properties()), IteratorUtils.count(actual.properties()));
assertEquals(expected.inVertex().id(), actual.inVertex().id());
assertEquals(expected.outVertex().id(), actual.outVertex().id());
assertEquals(expected.inVertex().label(), actual.inVertex().label());
assertEquals(expected.outVertex().label(), actual.outVertex().label());
- final Iterator<Property<Object>> itty = expected.properties();
- while(itty.hasNext()) {
- final Property p = itty.next();
- assertProperty(p, actual.property(p.key()));
+ if (!(getCompatibility() instanceof GraphBinaryCompatibility)) {
+ assertEquals(IteratorUtils.count(expected.properties()), IteratorUtils.count(actual.properties()));
+ final Iterator<Property<Object>> itty = expected.properties();
+ while(itty.hasNext()) {
+ final Property p = itty.next();
+ assertProperty(p, actual.property(p.key()));
+ }
}
}
protected void assertVertexProperty(final VertexProperty expected, final VertexProperty actual) {
assertEquals(expected.id(), actual.id());
assertEquals(expected.label(), actual.label());
- assertEquals(IteratorUtils.count(expected.properties()), IteratorUtils.count(actual.properties()));
- final Iterator<Property> itty = expected.properties();
- while(itty.hasNext()) {
- final Property p = itty.next();
- assertProperty(p, actual.property(p.key()));
+
+ if (!(getCompatibility() instanceof GraphBinaryCompatibility)) {
+ assertEquals(IteratorUtils.count(expected.properties()), IteratorUtils.count(actual.properties()));
+ final Iterator<Property> itty = expected.properties();
+ while (itty.hasNext()) {
+ final Property p = itty.next();
+ assertProperty(p, actual.property(p.key()));
+ }
}
}
diff --git a/gremlin-tools/gremlin-io-test/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphbinary/GraphBinaryCompatibilityTest.java b/gremlin-tools/gremlin-io-test/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphbinary/GraphBinaryCompatibilityTest.java
new file mode 100644
index 0000000..aa2ad0b
--- /dev/null
+++ b/gremlin-tools/gremlin-io-test/src/test/java/org/apache/tinkerpop/gremlin/structure/io/graphbinary/GraphBinaryCompatibilityTest.java
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.structure.io.graphbinary;
+
+import io.netty.buffer.ByteBuf;
+import io.netty.buffer.ByteBufAllocator;
+import io.netty.buffer.UnpooledDirectByteBuf;
+import org.apache.commons.io.output.ByteArrayOutputStream;
+import org.apache.tinkerpop.gremlin.driver.ser.binary.GraphBinaryReader;
+import org.apache.tinkerpop.gremlin.driver.ser.binary.GraphBinaryWriter;
+import org.apache.tinkerpop.gremlin.structure.io.AbstractTypedCompatibilityTest;
+import org.apache.tinkerpop.gremlin.structure.io.Compatibility;
+import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoCompatibility;
+import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoMapper;
+import org.apache.tinkerpop.gremlin.structure.io.gryo.GryoVersion;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV2d0;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV3d0;
+import org.apache.tinkerpop.shaded.kryo.Kryo;
+import org.apache.tinkerpop.shaded.kryo.io.Input;
+import org.apache.tinkerpop.shaded.kryo.io.Output;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+import java.util.Arrays;
+
+/**
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+@RunWith(Parameterized.class)
+public class GraphBinaryCompatibilityTest extends AbstractTypedCompatibilityTest {
+
+ private static final ByteBufAllocator allocator = ByteBufAllocator.DEFAULT;
+ private static final GraphBinaryWriter writerV1 = new GraphBinaryWriter();
+ private static final GraphBinaryReader readerV1 = new GraphBinaryReader();
+
+ @Parameterized.Parameters(name = "expect({0})")
+ public static Iterable<Object[]> data() {
+ return Arrays.asList(new Object[][]{
+ {GraphBinaryCompatibility.V1_3_4_3, readerV1, writerV1 }
+ });
+ }
+
+ @Parameterized.Parameter(value = 0)
+ public Compatibility compatibility;
+
+ @Parameterized.Parameter(value = 1)
+ public GraphBinaryReader reader;
+
+ @Parameterized.Parameter(value = 2)
+ public GraphBinaryWriter writer;
+
+ @Override
+ public <T> T read(final byte[] bytes, final Class<T> clazz) throws Exception {
+ final ByteBuf buffer = allocator.buffer();
+ buffer.writeBytes(bytes);
+ return reader.read(buffer);
+ }
+
+ @Override
+ public byte[] write(final Object o, final Class<?> clazz) throws Exception {
+ final ByteBuf buffer = allocator.buffer();
+ try (ByteArrayOutputStream stream = new ByteArrayOutputStream()) {
+ writer.write(o, buffer);
+ buffer.readerIndex(0);
+ buffer.readBytes(stream, buffer.readableBytes());
+ return stream.toByteArray();
+ }
+ }
+
+ @Override
+ public Compatibility getCompatibility() {
+ return compatibility;
+ }
+}
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/barrier-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/barrier-v1.gbin
new file mode 100644
index 0000000..f905861
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/barrier-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bigdecimal-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bigdecimal-v1.gbin
new file mode 100644
index 0000000..921d957
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bigdecimal-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/biginteger-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/biginteger-v1.gbin
new file mode 100644
index 0000000..d841fbe
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/biginteger-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/binding-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/binding-v1.gbin
new file mode 100644
index 0000000..79620eb
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/binding-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bulkset-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bulkset-v1.gbin
new file mode 100644
index 0000000..4542906
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bulkset-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/byte-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/byte-v1.gbin
new file mode 100644
index 0000000..af9544f
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/byte-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bytebuffer-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bytebuffer-v1.gbin
new file mode 100644
index 0000000..39793d2
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bytebuffer-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bytecode-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bytecode-v1.gbin
new file mode 100644
index 0000000..bc76c2e
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/bytecode-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/cardinality-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/cardinality-v1.gbin
new file mode 100644
index 0000000..3c875a5
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/cardinality-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/char-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/char-v1.gbin
new file mode 100644
index 0000000..b9d97e6
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/char-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/class-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/class-v1.gbin
new file mode 100644
index 0000000..6be272d
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/class-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/column-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/column-v1.gbin
new file mode 100644
index 0000000..c6805c8
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/column-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/date-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/date-v1.gbin
new file mode 100644
index 0000000..e68e17a
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/date-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/direction-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/direction-v1.gbin
new file mode 100644
index 0000000..3caaba3
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/direction-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/double-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/double-v1.gbin
new file mode 100644
index 0000000..e538a62
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/double-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/duration-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/duration-v1.gbin
new file mode 100644
index 0000000..7ffa3ad
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/duration-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/edge-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/edge-v1.gbin
new file mode 100644
index 0000000..086e85d
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/edge-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/float-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/float-v1.gbin
new file mode 100644
index 0000000..322772c
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/float-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/inetaddress-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/inetaddress-v1.gbin
new file mode 100644
index 0000000..b613ddb
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/inetaddress-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/instant-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/instant-v1.gbin
new file mode 100644
index 0000000..3335532
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/instant-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/integer-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/integer-v1.gbin
new file mode 100644
index 0000000..7dc246a
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/integer-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/lambda-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/lambda-v1.gbin
new file mode 100644
index 0000000..b0e98f9
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/lambda-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/list-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/list-v1.gbin
new file mode 100644
index 0000000..1aa405d
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/list-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/localdate-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/localdate-v1.gbin
new file mode 100644
index 0000000..2c9e211
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/localdate-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/localdatetime-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/localdatetime-v1.gbin
new file mode 100644
index 0000000..2e0c83c
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/localdatetime-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/localtime-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/localtime-v1.gbin
new file mode 100644
index 0000000..05785c6
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/localtime-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/long-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/long-v1.gbin
new file mode 100644
index 0000000..cee0083
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/long-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/map-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/map-v1.gbin
new file mode 100644
index 0000000..46cd9e2
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/map-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/metrics-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/metrics-v1.gbin
new file mode 100644
index 0000000..dea77a5
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/metrics-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/monthday-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/monthday-v1.gbin
new file mode 100644
index 0000000..c128263
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/monthday-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/offsetdatetime-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/offsetdatetime-v1.gbin
new file mode 100644
index 0000000..4b2aabb
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/offsetdatetime-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/offsettime-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/offsettime-v1.gbin
new file mode 100644
index 0000000..a4c8c5b
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/offsettime-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/operator-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/operator-v1.gbin
new file mode 100644
index 0000000..84717cc
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/operator-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/order-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/order-v1.gbin
new file mode 100644
index 0000000..311b6ad
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/order-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/p-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/p-v1.gbin
new file mode 100644
index 0000000..ac60799
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/p-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pand-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pand-v1.gbin
new file mode 100644
index 0000000..3237c94
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pand-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/path-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/path-v1.gbin
new file mode 100644
index 0000000..ba075ac
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/path-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/period-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/period-v1.gbin
new file mode 100644
index 0000000..f7dab35
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/period-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pick-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pick-v1.gbin
new file mode 100644
index 0000000..77aca2f
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pick-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pop-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pop-v1.gbin
new file mode 100644
index 0000000..f698c94
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pop-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/por-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/por-v1.gbin
new file mode 100644
index 0000000..7c0b3ab
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/por-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/property-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/property-v1.gbin
new file mode 100644
index 0000000..ddc34e5
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/property-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pwithin-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pwithin-v1.gbin
new file mode 100644
index 0000000..5ad30a9
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pwithin-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pwithout-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pwithout-v1.gbin
new file mode 100644
index 0000000..5b195db
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/pwithout-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/scope-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/scope-v1.gbin
new file mode 100644
index 0000000..534b956
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/scope-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/set-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/set-v1.gbin
new file mode 100644
index 0000000..02df059
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/set-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/short-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/short-v1.gbin
new file mode 100644
index 0000000..e1d2d7d
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/short-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/t-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/t-v1.gbin
new file mode 100644
index 0000000..a6376db
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/t-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/textp-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/textp-v1.gbin
new file mode 100644
index 0000000..c796468
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/textp-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/timestamp-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/timestamp-v1.gbin
new file mode 100644
index 0000000..4fc4e5e
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/timestamp-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/tinkergraph-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/tinkergraph-v1.gbin
new file mode 100644
index 0000000..a0a76d1
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/tinkergraph-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/traversalmetrics-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/traversalmetrics-v1.gbin
new file mode 100644
index 0000000..c947e62
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/traversalmetrics-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/traverser-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/traverser-v1.gbin
new file mode 100644
index 0000000..8e47cc0
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/traverser-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/uuid-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/uuid-v1.gbin
new file mode 100644
index 0000000..7f1775d
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/uuid-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/vertex-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/vertex-v1.gbin
new file mode 100644
index 0000000..7fb9d65
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/vertex-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/vertexproperty-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/vertexproperty-v1.gbin
new file mode 100644
index 0000000..89f0329
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/vertexproperty-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/year-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/year-v1.gbin
new file mode 100644
index 0000000..1fe0c74
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/year-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/yearmonth-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/yearmonth-v1.gbin
new file mode 100644
index 0000000..481a121
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/yearmonth-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/zoneddatetime-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/zoneddatetime-v1.gbin
new file mode 100644
index 0000000..32e2e8b
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/zoneddatetime-v1.gbin differ
diff --git a/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/zoneoffset-v1.gbin b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/zoneoffset-v1.gbin
new file mode 100644
index 0000000..2867c97
Binary files /dev/null and b/gremlin-tools/gremlin-io-test/src/test/resources/org/apache/tinkerpop/gremlin/structure/io/graphbinary/_3_4_3/zoneoffset-v1.gbin differ
diff --git a/pom.xml b/pom.xml
index d7340a9..00f8c27 100644
--- a/pom.xml
+++ b/pom.xml
@@ -375,6 +375,7 @@ limitations under the License.
<exclude>**/target/**</exclude>
<exclude>data/*.txt</exclude>
<exclude>**/*.kryo</exclude>
+ <exclude>**/*.gbin</exclude>
<exclude>**/*.iml</exclude>
<exclude>**/*.json</exclude>
<exclude>**/*.xml</exclude>
[tinkerpop] 02/33: Add graphbinary support for list in python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 75ed2d09b4b8c71a31e703169c2c52af246db63c
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Mon Jul 22 13:37:21 2019 -0400
Add graphbinary support for list in python
Just getting the basic container model setup and validated.
---
.../gremlin_python/structure/io/graphbinaryV1.py | 52 +++++++++++++++++-----
.../tests/structure/io/test_graphbinaryV1.py | 5 +++
2 files changed, 46 insertions(+), 11 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 4eff156..5e06da4 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -22,6 +22,7 @@ import time
import uuid
import math
import base64
+import io
import numbers
from collections import OrderedDict
from decimal import *
@@ -48,6 +49,7 @@ _deserializers = {}
class DataType(Enum):
string = 0x03
+ list = 0x09
class GraphBinaryTypeType(type):
@@ -95,11 +97,15 @@ class GraphBinaryReader(object):
self.deserializers.update(deserializer_map)
def readObject(self, b):
- return self.toObject(b)
+ if isinstance(b, bytearray):
+ return self.toObject(io.BytesIO(b))
+ elif isinstance(b, io.BufferedIOBase):
+ return self.toObject(b)
- def toObject(self, obj):
- binary_type, rest = DataType(obj[0]), obj[1:]
- return self.deserializers[binary_type].objectify(rest, self)
+ def toObject(self, buff):
+ bt = buff.read(1)
+ bt_value = struct.unpack('>b', bt)[0]
+ return self.deserializers[DataType(bt_value)].objectify(buff, self)
@six.add_metaclass(GraphBinaryTypeType)
@@ -112,13 +118,14 @@ class _GraphBinaryTypeIO(object):
"set_": "set", "list_": "list", "all_": "all", "with_": "with"}
@classmethod
- def as_bytes(cls, graphbin_type, *args):
+ def as_bytes(cls, graphbin_type, size=None, *args):
ba = bytearray([graphbin_type.value])
+ if size is not None:
+ ba.extend(struct.pack(">i", size))
+
for arg in args:
if isinstance(arg, str):
ba.extend(arg.encode("utf-8"))
- elif isinstance(arg, numbers.Number):
- ba.extend(bytes([arg]))
elif isinstance(arg, (bytes, bytearray)):
ba.extend(arg)
return ba
@@ -145,7 +152,30 @@ class StringIO(_GraphBinaryTypeIO):
@classmethod
def objectify(cls, b, reader):
- len_block, rest = b[0:4], b[4:]
- size = struct.unpack("<L", len_block)[0]
- string_block, b = rest[0:size], rest[size:]
- return string_block
+ size = struct.unpack(">i", b.read(4))[0]
+ return b.read(size)
+
+
+class ListIO(_GraphBinaryTypeIO):
+
+ python_type = list
+ graphbinary_type = DataType.list
+
+ @classmethod
+ def dictify(cls, n, writer):
+ list_data = bytearray()
+ for item in n:
+ list_data.extend(writer.writeObject(item))
+
+ return cls.as_bytes(cls.graphbinary_type, len(n), list_data)
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ x = buff.read(4)
+ size = struct.unpack(">i", x)[0]
+ the_list = []
+ while size > 0:
+ the_list.append(reader.readObject(buff))
+ size = size - 1
+
+ return the_list
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 9eb82cf..1fd2a19 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -47,3 +47,8 @@ class TestGraphSONWriter(object):
x = "serialize this!"
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+
+ def test_homogeneous_list(self):
+ x = ["serialize this!", "serialize that!", "stop telling me what to serialize"]
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
[tinkerpop] 05/33: Added date graphbinary type to python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit e2b732c2d0626db7e548f3c328d034e08d192524
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Mon Jul 22 18:35:40 2019 -0400
Added date graphbinary type to python
---
.../gremlin_python/structure/io/graphbinaryV1.py | 48 ++++++++++++++++------
.../tests/structure/io/test_graphbinaryV1.py | 5 +++
2 files changed, 41 insertions(+), 12 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 225b399..a3c7bc9 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -51,6 +51,7 @@ class DataType(Enum):
int = 0x01
long = 0x02
string = 0x03
+ date = 0x04
list = 0x09
@@ -126,10 +127,10 @@ class _GraphBinaryTypeIO(object):
ba.extend(struct.pack(">i", size))
for arg in args:
- if isinstance(arg, str):
- ba.extend(arg.encode("utf-8"))
- elif isinstance(arg, (bytes, bytearray)):
+ if isinstance(arg, (bytes, bytearray)):
ba.extend(arg)
+ else:
+ raise Exception("MISSING")
return ba
@classmethod
@@ -154,15 +155,15 @@ class LongIO(_GraphBinaryTypeIO):
byte_format = ">q"
@classmethod
- def dictify(cls, n, writer):
- if n < -9223372036854775808 or n > 9223372036854775807:
+ def dictify(cls, obj, writer):
+ if obj < -9223372036854775808 or obj > 9223372036854775807:
raise Exception("TODO: don't forget bigint")
else:
- return cls.as_bytes(cls.graphbinary_type, None, struct.pack(cls.byte_format, n))
+ return cls.as_bytes(cls.graphbinary_type, None, struct.pack(cls.byte_format, obj))
@classmethod
def objectify(cls, buff, reader):
- return struct.unpack(">q", buff.read(8))[0]
+ return struct.unpack(cls.byte_format, buff.read(8))[0]
class IntIO(LongIO):
@@ -176,14 +177,37 @@ class IntIO(LongIO):
return cls.read_int(buff)
+class DateIO(_GraphBinaryTypeIO):
+
+ python_type = datetime.datetime
+ graphbinary_type = DataType.date
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ if six.PY3:
+ pts = (obj - datetime.datetime(1970, 1, 1)) / datetime.timedelta(seconds=1)
+ else:
+ # Hack for legacy Python - timestamp() in Python 3.3
+ pts = (time.mktime(obj.timetuple()) + obj.microsecond / 1e6) - \
+ (time.mktime(datetime.datetime(1970, 1, 1).timetuple()))
+
+ # Java timestamp expects milliseconds - have to use int because of legacy Python
+ ts = int(round(pts * 1000))
+ return cls.as_bytes(cls.graphbinary_type, None, struct.pack(">q", ts))
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ return datetime.datetime.utcfromtimestamp(struct.unpack(">q", buff.read(8))[0] / 1000.0)
+
+
class StringIO(_GraphBinaryTypeIO):
python_type = str
graphbinary_type = DataType.string
@classmethod
- def dictify(cls, n, writer):
- return cls.as_bytes(cls.graphbinary_type, len(n), n)
+ def dictify(cls, obj, writer):
+ return cls.as_bytes(cls.graphbinary_type, len(obj), obj.encode("utf-8"))
@classmethod
def objectify(cls, b, reader):
@@ -196,12 +220,12 @@ class ListIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.list
@classmethod
- def dictify(cls, n, writer):
+ def dictify(cls, obj, writer):
list_data = bytearray()
- for item in n:
+ for item in obj:
list_data.extend(writer.writeObject(item))
- return cls.as_bytes(cls.graphbinary_type, len(n), list_data)
+ return cls.as_bytes(cls.graphbinary_type, len(obj), list_data)
@classmethod
def objectify(cls, buff, reader):
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 766544d..a0a5be1 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -53,6 +53,11 @@ class TestGraphSONWriter(object):
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+ def test_date(self):
+ x = datetime.datetime(2016, 12, 14, 16, 14, 36, 295000)
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
def test_string(self):
x = "serialize this!"
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
[tinkerpop] 32/33: Generalized refactoring to python graphbinary.
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 8de895bb780d73d25a9dc0fdfcc694187e64c953
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Aug 16 13:59:05 2019 -0400
Generalized refactoring to python graphbinary.
Tried to streamline conversions to binary a bit within each serializer.
---
.../gremlin_python/structure/io/graphbinaryV1.py | 103 +++++++++------------
.../gremlin_python/structure/io/graphsonV3d0.py | 2 -
2 files changed, 45 insertions(+), 60 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index cbdda22..6fc73ab 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -21,24 +21,16 @@ import six
import datetime
import calendar
import struct
-import time
import uuid
import math
-import base64
import io
-import numbers
from collections import OrderedDict
-from decimal import *
import logging
-from datetime import timedelta
-import six
from aenum import Enum
-from isodate import parse_duration, duration_isoformat
-
from gremlin_python import statics
from gremlin_python.statics import FloatType, FunctionType, IntType, LongType, TypeType, DictType, ListType, SetType, \
- SingleByte, ByteBufferType, SingleChar, GremlinType
+ SingleByte, ByteBufferType, GremlinType
from gremlin_python.process.traversal import Barrier, Binding, Bytecode, Cardinality, Column, Direction, Operator, \
Order, Pick, Pop, P, Scope, TextP, Traversal, Traverser, \
TraversalStrategy, T
@@ -185,15 +177,12 @@ class _GraphBinaryTypeIO(object):
"filter_": "filter", "id_": "id", "max_": "max", "min_": "min", "sum_": "sum"}
@classmethod
- def as_bytes(cls, graphbin_type=None, size=None, nullable=True, *args):
- ba = bytearray() if graphbin_type is None else bytearray([graphbin_type.value])
+ def as_bytes(cls, graphbin_type, as_value=False, nullable=True, *args):
+ ba = bytearray() if as_value else bytearray([graphbin_type.value])
if nullable:
ba.extend(struct.pack(">b", 0))
- if size is not None:
- ba.extend(struct.pack(">i", size))
-
for arg in args:
if isinstance(arg, (bytes, bytearray)):
ba.extend(arg)
@@ -221,8 +210,8 @@ class _GraphBinaryTypeIO(object):
return cls.symbolMap.get(symbol, symbol)
@classmethod
- def write_as_value(cls, graph_binary_type, as_value):
- return None if as_value else graph_binary_type
+ def write_as_value(cls, graphbin_type, as_value):
+ return None if as_value else graphbin_type
@classmethod
def is_null(cls, buff, reader, else_opt, nullable=True):
@@ -246,8 +235,7 @@ class LongIO(_GraphBinaryTypeIO):
if obj < -9223372036854775808 or obj > 9223372036854775807:
raise Exception("TODO: don't forget bigint")
else:
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, nullable, struct.pack(cls.byte_format, obj))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, struct.pack(cls.byte_format, obj))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -279,8 +267,7 @@ class DateIO(_GraphBinaryTypeIO):
pts = calendar.timegm(obj.timetuple()) * 1e3
ts = int(round(pts))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, nullable, struct.pack(">q", ts))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, struct.pack(">q", ts))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -299,8 +286,7 @@ class TimestampIO(_GraphBinaryTypeIO):
def dictify(cls, obj, writer, as_value=False, nullable=True):
# Java timestamp expects milliseconds integer - Have to use int because of legacy Python
ts = int(round(obj * 1000))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, nullable, struct.pack(">q", ts))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, struct.pack(">q", ts))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -328,17 +314,15 @@ class FloatIO(LongIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
if math.isnan(obj):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, nullable, struct.pack(cls.byte_format, NAN))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, struct.pack(cls.byte_format, NAN))
elif math.isinf(obj) and obj > 0:
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, nullable, struct.pack(cls.byte_format, POSITIVE_INFINITY))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable,
+ struct.pack(cls.byte_format, POSITIVE_INFINITY))
elif math.isinf(obj) and obj < 0:
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, nullable, struct.pack(cls.byte_format, NEGATIVE_INFINITY))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable,
+ struct.pack(cls.byte_format, NEGATIVE_INFINITY))
else:
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, nullable, struct.pack(cls.byte_format, obj))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, struct.pack(cls.byte_format, obj))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -366,8 +350,10 @@ class StringIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- len(obj), nullable, obj.encode("utf-8"))
+ ba = bytearray()
+ ba.extend(struct.pack(">i", len(obj)))
+ ba.extend(obj.encode("utf-8"))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -382,11 +368,11 @@ class ListIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
list_data = bytearray()
+ list_data.extend(struct.pack(">i", len(obj)))
for item in obj:
list_data.extend(writer.writeObject(item))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- len(obj), nullable, list_data)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, list_data)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -415,18 +401,18 @@ class SetDeserializer(ListIO):
class MapIO(_GraphBinaryTypeIO):
- python_type = dict
+ python_type = DictType
graphbinary_type = DataType.map
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
map_data = bytearray()
+ map_data.extend(struct.pack(">i", len(obj)))
for k, v in obj.items():
map_data.extend(writer.writeObject(k))
map_data.extend(writer.writeObject(v))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- len(obj), nullable, map_data)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, map_data)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -452,8 +438,7 @@ class UuidIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, nullable, obj.bytes)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, obj.bytes)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -476,7 +461,7 @@ class EdgeIO(_GraphBinaryTypeIO):
ba.extend(cls.string_as_bytes(obj.outV.label))
ba.extend(NULL_BYTES)
ba.extend(NULL_BYTES)
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -502,7 +487,7 @@ class PathIO(_GraphBinaryTypeIO):
ba = bytearray()
ba.extend(writer.writeObject(obj.labels))
ba.extend(writer.writeObject(obj.objects))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -520,7 +505,7 @@ class PropertyIO(_GraphBinaryTypeIO):
ba.extend(cls.string_as_bytes(obj.key))
ba.extend(writer.writeObject(obj.value))
ba.extend(NULL_BYTES)
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -558,7 +543,7 @@ class VertexIO(_GraphBinaryTypeIO):
ba.extend(writer.writeObject(obj.id))
ba.extend(cls.string_as_bytes(obj.label))
ba.extend(NULL_BYTES)
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -584,7 +569,7 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
ba.extend(writer.writeObject(obj.value))
ba.extend(NULL_BYTES)
ba.extend(NULL_BYTES)
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -603,7 +588,7 @@ class _EnumIO(_GraphBinaryTypeIO):
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(StringIO.dictify(cls.unmangleKeyword(str(obj.name)), writer))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -665,7 +650,7 @@ class BindingIO(_GraphBinaryTypeIO):
ba = bytearray()
ba.extend(cls.string_as_bytes(obj.key))
ba.extend(writer.writeObject(obj.value))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -699,7 +684,7 @@ class BytecodeIO(_GraphBinaryTypeIO):
else:
ba.extend(writer.writeObject(arg))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -765,7 +750,7 @@ class LambdaSerializer(_GraphBinaryTypeIO):
ba.extend(StringIO.dictify(script_cleaned, writer, True, False))
ba.extend(struct.pack(">i", script_args))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
class PSerializer(_GraphBinaryTypeIO):
@@ -791,7 +776,7 @@ class PSerializer(_GraphBinaryTypeIO):
for a in args:
ba.extend(writer.writeObject(a))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
class ScopeIO(_EnumIO):
@@ -813,7 +798,7 @@ class TraverserIO(_GraphBinaryTypeIO):
ba = bytearray()
ba.extend(struct.pack(">q", obj.bulk))
ba.extend(writer.writeObject(obj.object))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -832,7 +817,7 @@ class ByteIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, struct.pack(">b", obj))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, struct.pack(">b", obj))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -847,7 +832,10 @@ class ByteBufferIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), len(obj), nullable, obj)
+ ba = bytearray()
+ ba.extend(struct.pack(">i", len(obj)))
+ ba.extend(obj)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -865,8 +853,7 @@ class BooleanIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
- return cls.as_bytes(cls.write_as_value(
- cls.graphbinary_type, as_value), None, nullable, struct.pack(">b", 0x01 if obj else 0x00))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, struct.pack(">b", 0x01 if obj else 0x00))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -898,7 +885,7 @@ class TextPSerializer(_GraphBinaryTypeIO):
for a in args:
ba.extend(writer.writeObject(a))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
class BulkSetDeserializer(_GraphBinaryTypeIO):
@@ -971,8 +958,8 @@ class ClassSerializer(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
- return cls.as_bytes(cls.write_as_value(
- cls.graphbinary_type, as_value), None, nullable, StringIO.dictify(obj.gremlin_type, writer, True, False))
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable,
+ StringIO.dictify(obj.gremlin_type, writer, True, False))
class TraversalStrategySerializer(_GraphBinaryTypeIO):
@@ -985,7 +972,7 @@ class TraversalStrategySerializer(_GraphBinaryTypeIO):
ba.extend(ClassSerializer.dictify(GremlinType(obj.fqcn), writer, True, False))
conf = {k: cls._convert(v) for k, v in obj.configuration.items()}
ba.extend(MapIO.dictify(conf, writer, True, False))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+ return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@classmethod
def _convert(cls, v):
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphsonV3d0.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphsonV3d0.py
index 10a74da..4caea49 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphsonV3d0.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphsonV3d0.py
@@ -19,10 +19,8 @@ under the License.
import calendar
import datetime
import json
-import time
import uuid
import math
-import base64
from collections import OrderedDict
from decimal import *
import logging
[tinkerpop] 19/33: Added BulkSet support to python graphbinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 8cda34ca0ec61444b9ce3767090173c9ee09674d
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Aug 2 07:34:58 2019 -0400
Added BulkSet support to python graphbinary
---
docs/src/dev/io/graphbinary.asciidoc | 4 ++--
.../gremlin_python/structure/io/graphbinaryV1.py | 28 ++++++++++++++++++----
2 files changed, 25 insertions(+), 7 deletions(-)
diff --git a/docs/src/dev/io/graphbinary.asciidoc b/docs/src/dev/io/graphbinary.asciidoc
index 5fa4afe..0262387 100644
--- a/docs/src/dev/io/graphbinary.asciidoc
+++ b/docs/src/dev/io/graphbinary.asciidoc
@@ -532,8 +532,8 @@ Where:
`{type_code}{type_info}{value_flag}{value}` followed by the "bulk" which is a `Long` value.
If the implementing language does not have a `BulkSet` object to deserialize into, this format can be coerced to a
-`List` and still be considered compliant with Gremlin. Simply create "expand the bulk" by adding the item to the `List`
-the number of times specified by the bulk.
+`List` and still be considered compliant with Gremlin. Simply "expand the bulk" by adding the item to the `List` the
+number of times specified by the bulk.
==== Tree
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 2f64ba2..e282dda 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -58,7 +58,7 @@ class DataType(Enum):
string = 0x03
date = 0x04
timestamp = 0x05
- clazz = 0x06
+ clazz = 0x06 #todo
double = 0x07
float = 0x08
list = 0x09
@@ -86,14 +86,14 @@ class DataType(Enum):
scope = 0x1f
t = 0x20
traverser = 0x21
- bigdecimal = 0x22 #todo
- biginteger = 0x23 #todo
+ bigdecimal = 0x22 #todo
+ biginteger = 0x23 #todo
byte = 0x24
bytebuffer = 0x25
- short = 0x26 #todo?
+ short = 0x26 #todo?
boolean = 0x27
textp = 0x28
- traversalstrategy = 0x29
+ traversalstrategy = 0x29 #todo
bulkset = 0x2a
tree = 0x2b
metrics = 0x2c
@@ -791,3 +791,21 @@ class TextPIO(_GraphBinaryTypeIO):
ba.extend(a)
return ba
+
+
+class BulkSetIO(_GraphBinaryTypeIO):
+
+ graphbinary_type = DataType.bulkset
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ size = cls.read_int(buff)
+ the_list = []
+ while size > 0:
+ itm = reader.readObject(buff)
+ bulk = cls.read_int(buff)
+ for y in range(bulk):
+ the_list.append(itm)
+ size = size - 1
+
+ return the_list
[tinkerpop] 09/33: Added Dict/Set serialization for graphbinary in
python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 3bcaf233f5837709c3a01c5fe5311f2b50805475
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Jul 26 14:19:34 2019 -0400
Added Dict/Set serialization for graphbinary in python
---
.../gremlin_python/structure/io/graphbinaryV1.py | 39 ++++++++++++++++++++++
.../tests/structure/io/test_graphbinaryV1.py | 23 +++++++++++--
2 files changed, 60 insertions(+), 2 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 4a5dd0e..3d7d7da 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -58,6 +58,8 @@ class DataType(Enum):
double = 0x07
float = 0x08
list = 0x09
+ map = 0x0a
+ set = 0x0b
class GraphBinaryTypeType(type):
@@ -311,3 +313,40 @@ class ListIO(_GraphBinaryTypeIO):
size = size - 1
return the_list
+
+
+class SetIO(ListIO):
+
+ python_type = SetType
+ graphbinary_type = DataType.set
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ return set(ListIO.objectify(buff, reader))
+
+
+class MapIO(_GraphBinaryTypeIO):
+
+ python_type = dict
+ graphbinary_type = DataType.map
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ map_data = bytearray()
+ for k, v in obj.items():
+ map_data.extend(writer.writeObject(k))
+ map_data.extend(writer.writeObject(v))
+
+ return cls.as_bytes(cls.graphbinary_type, len(obj), map_data)
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ size = cls.read_int(buff)
+ the_dict = {}
+ while size > 0:
+ k = reader.readObject(buff)
+ v = reader.readObject(buff)
+ the_dict[k] = v
+ size = size - 1
+
+ return the_dict
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 75b4392..54acb55 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -92,11 +92,30 @@ class TestGraphSONWriter(object):
assert x == output
def test_homogeneous_list(self):
- x = ["serialize this!", "serialize that!", "stop telling me what to serialize"]
+ x = ["serialize this!", "serialize that!", "serialize that!","stop telling me what to serialize"]
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
def test_heterogeneous_list(self):
- x = ["serialize this!", 0, "serialize that!", 1, "stop telling me what to serialize", 2]
+ x = ["serialize this!", 0, "serialize that!", "serialize that!", 1, "stop telling me what to serialize", 2]
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ def test_homogeneous_set(self):
+ x = {"serialize this!", "serialize that!", "stop telling me what to serialize"}
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ def test_heterogeneous_set(self):
+ x = {"serialize this!", 0, "serialize that!", 1, "stop telling me what to serialize", 2}
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ def test_dict(self):
+ x = {"yo": "what?",
+ "go": "no!",
+ "number": 123,
+ 321: "crazy with the number for a key",
+ 987: ["go", "deep", {"here": "!"}]}
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
[tinkerpop] 27/33: Fixed lambda serialization in python graphbinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit db2acaaf574ffbac251c2a16a1b893fef1c03ce7
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Tue Aug 13 15:56:09 2019 -0400
Fixed lambda serialization in python graphbinary
---
.../src/main/jython/gremlin_python/structure/io/graphbinaryV1.py | 6 +++---
.../src/main/jython/tests/driver/test_driver_remote_connection.py | 4 ++++
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index a8eb853..d9844e9 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -735,12 +735,12 @@ class LambdaIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
- ba = bytearray() if as_value else bytearray([cls.graphbinary_type.value])
+ ba = bytearray()
lambda_result = obj()
script = lambda_result if isinstance(lambda_result, str) else lambda_result[0]
language = statics.default_lambda_language if isinstance(lambda_result, str) else lambda_result[1]
- ba.extend(cls.string_as_bytes(language))
+ ba.extend(StringIO.dictify(language, writer, True, False))
script_cleaned = script
script_args = -1
@@ -750,7 +750,7 @@ class LambdaIO(_GraphBinaryTypeIO):
script_cleaned = "lambda " + script
script_args = six.get_function_code(eval(script_cleaned)).co_argcount
- ba.extend(cls.string_as_bytes(script_cleaned))
+ ba.extend(StringIO.dictify(script_cleaned, writer, True, False))
ba.extend(struct.pack(">i", script_args))
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
diff --git a/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py b/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py
index 1c689d9..9073ad6 100644
--- a/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py
+++ b/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py
@@ -82,6 +82,10 @@ class TestDriverRemoteConnection(object):
assert 'marko' in results
assert 'vadas' in results
# #
+ results = g.V().has('person', 'name', 'marko').map(lambda: ("it.get().value('name')", "gremlin-groovy")).toList()
+ assert 1 == len(results)
+ assert 'marko' in results
+ # #
# this test just validates that the underscored versions of steps conflicting with Gremlin work
# properly and can be removed when the old steps are removed - TINKERPOP-2272
results = g.V().filter_(__.values('age').sum_().and_(
[tinkerpop] 30/33: All GLV tests passing for GraphBinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit ff92f9be95ad4338812c4157fd12828e3d8d95c1
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Thu Aug 15 15:27:09 2019 -0400
All GLV tests passing for GraphBinary
Had to make a nubmer of tweaks to get this working. Most specifically PSerializer which wasn't properly accounting for the normalization that java seems to do.
---
.../gremlin/driver/ser/binary/types/PSerializer.java | 18 ++++++++++++------
.../binary/GraphBinaryReaderWriterRoundTripTest.java | 1 +
gremlin-python/pom.xml | 2 +-
.../gremlin_python/structure/io/graphbinaryV1.py | 9 ++++++++-
.../jython/tests/structure/io/test_graphbinaryV1.py | 11 ++++++++++-
5 files changed, 32 insertions(+), 9 deletions(-)
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ser/binary/types/PSerializer.java b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ser/binary/types/PSerializer.java
index f83816f..be399f0 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ser/binary/types/PSerializer.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ser/binary/types/PSerializer.java
@@ -62,14 +62,15 @@ public class PSerializer<T extends P> extends SimpleTypeSerializer<T> {
args[i] = context.read(buffer);
argumentClasses[i] = args[i].getClass();
}
-
- if ("and".equals(predicateName)) {
+
+ if ("and".equals(predicateName))
return (T) ((P) args[0]).and((P) args[1]);
- } else if ("or".equals(predicateName)) {
+ else if ("or".equals(predicateName))
return (T) ((P) args[0]).or((P) args[1]);
- }
+ else if ("not".equals(predicateName))
+ return (T) P.not((P) args[0]);
- CheckedFunction<Object[], T> f = getMethod(predicateName, argumentClasses);
+ final CheckedFunction<Object[], T> f = getMethod(predicateName, argumentClasses);
try {
return f.apply(args);
@@ -99,7 +100,12 @@ public class PSerializer<T extends P> extends SimpleTypeSerializer<T> {
try {
m = classOfP.getMethod(predicateName, Object.class);
} catch (NoSuchMethodException ex2) {
- throw new SerializationException(String.format("Can't find predicate method: '%s'", predicateName), ex2);
+ // finally go for the generics
+ try {
+ m = classOfP.getMethod(predicateName, Object.class, Object.class);
+ } catch (NoSuchMethodException ex3) {
+ throw new SerializationException(String.format("Can't find predicate method: '%s'", predicateName), ex2);
+ }
}
}
}
diff --git a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/binary/GraphBinaryReaderWriterRoundTripTest.java b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/binary/GraphBinaryReaderWriterRoundTripTest.java
index 48e324c..8f7874e 100644
--- a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/binary/GraphBinaryReaderWriterRoundTripTest.java
+++ b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/binary/GraphBinaryReaderWriterRoundTripTest.java
@@ -228,6 +228,7 @@ public class GraphBinaryReaderWriterRoundTripTest {
new Object[] {"Por", P.gt(1).or(P.lt(2)), null},
new Object[] {"Pnot", P.not(P.lte(1)), null},
new Object[] {"Pwithout", P.without(1,2,3,4), null},
+ new Object[] {"Pinside", P.inside(0.0d, 0.6d), null},
new Object[] {"TextP", TextP.startingWith("mark"), null},
// graph
diff --git a/gremlin-python/pom.xml b/gremlin-python/pom.xml
index 48a45b0..4fd9c47 100644
--- a/gremlin-python/pom.xml
+++ b/gremlin-python/pom.xml
@@ -419,7 +419,7 @@ limitations under the License.
<env key="PYTHONPATH" value=""/>
<arg line="setup.py install"/>
</exec>
- <!-- run for graphson 2.0 -->
+ <!-- run for graphson 3.0 -->
<exec executable="env/bin/radish" dir="${project.build.directory}/python2"
failonerror="true">
<env key="PYTHONPATH" value=""/>
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index ce03f40..738cfe7 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -16,6 +16,8 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
"""
+
+import six
import datetime
import calendar
import struct
@@ -127,7 +129,12 @@ class GraphBinaryWriter(object):
def toDict(self, obj):
try:
- return self.serializers[type(obj)].dictify(obj, self)
+ t = type(obj)
+
+ # coerce unicode to str so the serializer will be found properly in the cache...better way?
+ if not six.PY3:
+ t = str if isinstance(obj, unicode) else t
+ return self.serializers[t].dictify(obj, self)
except KeyError:
for key, serializer in self.serializers.items():
if isinstance(obj, key):
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index bcaad3d..917e7f2 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -119,7 +119,16 @@ class TestGraphSONWriter(object):
987: ["go", "deep", {"here": "!"}]}
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
-
+
+ x = {"marko": [666], "noone": ["blah"]}
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ x = {"ripple": [], "peter": ["created"], "noone": ["blah"], "vadas": [],
+ "josh": ["created", "created"], "lop": [], "marko": [666, "created", "knows", "knows"]}
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
def test_uuid(self):
x = uuid.UUID("41d2e28a-20a4-4ab0-b379-d810dede3786")
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
[tinkerpop] 28/33: Cleaned up enum derser for python graphbinary a
bit further
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 221c2a41a931852f7b59080d14d03395cf2adf25
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Tue Aug 13 17:02:31 2019 -0400
Cleaned up enum derser for python graphbinary a bit further
---
.../src/main/jython/gremlin_python/structure/io/graphbinaryV1.py | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index d9844e9..5261952 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -595,12 +595,17 @@ class _EnumIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
- ba.extend(cls.string_as_bytes(cls.unmangleKeyword(str(obj.name))))
+ ba.extend(StringIO.dictify(cls.unmangleKeyword(str(obj.name)), writer))
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
- return cls.is_null(buff, reader, lambda b, r: cls.python_type[cls.read_string(b)], nullable)
+ return cls.is_null(buff, reader, cls._read_enumval, nullable)
+
+ @classmethod
+ def _read_enumval(cls, b, r):
+ enum_name = r.toObject(b)
+ return cls.python_type[enum_name]
class BarrierIO(_EnumIO):
[tinkerpop] 23/33: Major refactoring for nullable writes.
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit a728648fdf845c758041c4c15d8b83d92fed4f56
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Aug 9 14:11:13 2019 -0400
Major refactoring for nullable writes.
Added more serializer support with metrics, class, and strategies. Still a bit rough but more tests are passing.
---
gremlin-python/pom.xml | 9 +-
.../jython/gremlin_python/driver/serializer.py | 2 +-
.../jython/gremlin_python/process/strategies.py | 5 +-
.../jython/gremlin_python/process/traversal.py | 3 +-
.../src/main/jython/gremlin_python/statics.py | 15 +-
.../gremlin_python/structure/io/graphbinaryV1.py | 272 ++++++++++++++-------
.../gremlin_python/structure/io/graphsonV3d0.py | 1 +
gremlin-python/src/main/jython/radish/terrain.py | 2 +
gremlin-python/src/main/jython/tests/conftest.py | 6 +-
.../tests/driver/test_driver_remote_connection.py | 129 +---------
10 files changed, 211 insertions(+), 233 deletions(-)
diff --git a/gremlin-python/pom.xml b/gremlin-python/pom.xml
index f3c3c85..48a45b0 100644
--- a/gremlin-python/pom.xml
+++ b/gremlin-python/pom.xml
@@ -419,13 +419,20 @@ limitations under the License.
<env key="PYTHONPATH" value=""/>
<arg line="setup.py install"/>
</exec>
- <!-- run for graphson 3.0 -->
+ <!-- run for graphson 2.0 -->
<exec executable="env/bin/radish" dir="${project.build.directory}/python2"
failonerror="true">
<env key="PYTHONPATH" value=""/>
<env key="PYTHONIOENCODING" value="utf-8:surrogateescape"/>
<arg line="-f dots -e -t -b ${project.build.directory}/python2/radish ${project.basedir}/../gremlin-test/features/ --user-data="serializer=application/vnd.gremlin-v3.0+json""/> <!-- -no-line-jump -->
</exec>
+ <!-- run for graphbinary 1.0 -->
+ <exec executable="env/bin/radish" dir="${project.build.directory}/python2"
+ failonerror="true">
+ <env key="PYTHONPATH" value=""/>
+ <env key="PYTHONIOENCODING" value="utf-8:surrogateescape"/>
+ <arg line="-f dots -e -t -b ${project.build.directory}/python2/radish ${project.basedir}/../gremlin-test/features/ --user-data="serializer=application/vnd.graphbinary-v1.0""/> <!-- -no-line-jump -->
+ </exec>
</target>
</configuration>
</execution>
diff --git a/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py b/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
index f805220..a95d292 100644
--- a/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
+++ b/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
@@ -170,7 +170,7 @@ class GraphSONSerializersV3d0(GraphSONMessageSerializer):
super(GraphSONSerializersV3d0, self).__init__(reader, writer, version)
-class GraphBinaryMessageSerializerV1(object):
+class GraphBinarySerializersV1(object):
DEFAULT_READER_CLASS = graphbinaryV1.GraphBinaryReader
DEFAULT_WRITER_CLASS = graphbinaryV1.GraphBinaryWriter
DEFAULT_VERSION = b"application/vnd.graphbinary-v1.0"
diff --git a/gremlin-python/src/main/jython/gremlin_python/process/strategies.py b/gremlin-python/src/main/jython/gremlin_python/process/strategies.py
index cbd5e08..186e303 100644
--- a/gremlin-python/src/main/jython/gremlin_python/process/strategies.py
+++ b/gremlin-python/src/main/jython/gremlin_python/process/strategies.py
@@ -64,8 +64,9 @@ class PartitionStrategy(TraversalStrategy):
class SubgraphStrategy(TraversalStrategy):
+
def __init__(self, vertices=None, edges=None, vertex_properties=None):
- TraversalStrategy.__init__(self)
+ TraversalStrategy.__init__(self, fqcn="org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.SubgraphStrategy")
if vertices is not None:
self.configuration["vertices"] = vertices
if edges is not None:
@@ -77,7 +78,7 @@ class SubgraphStrategy(TraversalStrategy):
class VertexProgramStrategy(TraversalStrategy):
def __init__(self, graph_computer=None, workers=None, persist=None, result=None, vertices=None, edges=None,
configuration=None):
- TraversalStrategy.__init__(self)
+ TraversalStrategy.__init__(self, fqcn="org.apache.tinkerpop.gremlin.process.computer.traversal.strategy.decoration.VertexProgramStrategy")
if graph_computer is not None:
self.configuration["graphComputer"] = graph_computer
if workers is not None:
diff --git a/gremlin-python/src/main/jython/gremlin_python/process/traversal.py b/gremlin-python/src/main/jython/gremlin_python/process/traversal.py
index 4fb9f8c..5a73702 100644
--- a/gremlin-python/src/main/jython/gremlin_python/process/traversal.py
+++ b/gremlin-python/src/main/jython/gremlin_python/process/traversal.py
@@ -581,8 +581,9 @@ class TraversalStrategies(object):
class TraversalStrategy(object):
- def __init__(self, strategy_name=None, configuration=None):
+ def __init__(self, strategy_name=None, configuration=None, fqcn=None):
self.strategy_name = type(self).__name__ if strategy_name is None else strategy_name
+ self.fqcn = fqcn
self.configuration = {} if configuration is None else configuration
def apply(self, traversal):
diff --git a/gremlin-python/src/main/jython/gremlin_python/statics.py b/gremlin-python/src/main/jython/gremlin_python/statics.py
index 2f06b97..012f52f 100644
--- a/gremlin-python/src/main/jython/gremlin_python/statics.py
+++ b/gremlin-python/src/main/jython/gremlin_python/statics.py
@@ -43,19 +43,20 @@ else:
from types import TypeType
from types import ListType
from types import DictType
+ from types import TypeType
class timestamp(float):
"""
In Python a timestamp is simply a float. This dummy class (similar to long), allows users to wrap a float
- in a GLV script to make sure the value is serialized as a GraphSON timestamp.
+ in a GLV script to make sure the value is serialized as a Gremlin timestamp.
"""
pass
class SingleByte(int):
"""
- Provides a way to pass a single byte via GraphSON.
+ Provides a way to pass a single byte via Gremlin.
"""
def __new__(cls, b):
if -128 <= b < 128:
@@ -66,7 +67,7 @@ class SingleByte(int):
class SingleChar(str):
"""
- Provides a way to pass a single character via GraphSON.
+ Provides a way to pass a single character via Gremlin.
"""
def __new__(cls, c):
if len(b) == 1:
@@ -75,6 +76,14 @@ class SingleChar(str):
raise ValueError("string must contain a single character")
+class GremlinType(object):
+ """
+ Provides a way to represent a "Java class" for Gremlin.
+ """
+ def __init__(self, gremlin_type):
+ self.gremlin_type = gremlin_type
+
+
staticMethods = {}
staticEnums = {}
default_lambda_language = "gremlin-python"
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index f34a3d4..dd829cf 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -36,7 +36,7 @@ from isodate import parse_duration, duration_isoformat
from gremlin_python import statics
from gremlin_python.statics import FloatType, FunctionType, IntType, LongType, TypeType, DictType, ListType, SetType, \
- SingleByte, ByteBufferType, SingleChar
+ SingleByte, ByteBufferType, SingleChar, GremlinType
from gremlin_python.process.traversal import Barrier, Binding, Bytecode, Cardinality, Column, Direction, Operator, \
Order, Pick, Pop, P, Scope, TextP, Traversal, Traverser, \
TraversalStrategy, T
@@ -58,7 +58,7 @@ class DataType(Enum):
string = 0x03
date = 0x04
timestamp = 0x05
- clazz = 0x06 #todo
+ clazz = 0x06
double = 0x07
float = 0x08
list = 0x09
@@ -68,7 +68,7 @@ class DataType(Enum):
edge = 0x0d
path = 0x0e
property = 0x0f
- graph = 0x10
+ graph = 0x10 # no graph object in python yet
vertex = 0x11
vertexproperty = 0x12
barrier = 0x13
@@ -93,14 +93,17 @@ class DataType(Enum):
short = 0x26 #todo?
boolean = 0x27
textp = 0x28
- traversalstrategy = 0x29 #todo
+ traversalstrategy = 0x29
bulkset = 0x2a
- tree = 0x2b #todo
- metrics = 0x2c #todo
- traversalmetrics = 0x2d #todo
+ tree = 0x2b # no tree object in Python yet
+ metrics = 0x2c
+ traversalmetrics = 0x2d
custom = 0x00 #todo
+NULL_BYTES = [DataType.null.value, 0x01]
+
+
class GraphBinaryTypeType(type):
def __new__(mcs, name, bases, dct):
cls = super(GraphBinaryTypeType, mcs).__new__(mcs, name, bases, dct)
@@ -172,11 +175,11 @@ class _GraphBinaryTypeIO(object):
"filter_": "filter", "id_": "id", "max_": "max", "min_": "min", "sum_": "sum"}
@classmethod
- def as_bytes(cls, graphbin_type=None, size=None, *args):
+ def as_bytes(cls, graphbin_type=None, size=None, nullable=True, *args):
ba = bytearray() if graphbin_type is None else bytearray([graphbin_type.value])
- # todo: empty value flag just hardcoded in
- ba.extend(struct.pack(">b", 0))
+ if nullable:
+ ba.extend(struct.pack(">b", 0))
if size is not None:
ba.extend(struct.pack(">i", size))
@@ -215,7 +218,7 @@ class _GraphBinaryTypeIO(object):
def is_null(cls, buff, reader, else_opt, nullable=True):
return None if nullable and buff.read(1)[0] == 0x01 else else_opt(buff, reader)
- def dictify(self, obj, writer, as_value=False):
+ def dictify(self, obj, writer, as_value=False, nullable=True):
raise NotImplementedError()
def objectify(self, d, reader, nullable=True):
@@ -229,12 +232,12 @@ class LongIO(_GraphBinaryTypeIO):
byte_format = ">q"
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
if obj < -9223372036854775808 or obj > 9223372036854775807:
raise Exception("TODO: don't forget bigint")
else:
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, struct.pack(cls.byte_format, obj))
+ None, nullable, struct.pack(cls.byte_format, obj))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -258,7 +261,7 @@ class DateIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.date
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
try:
timestamp_seconds = calendar.timegm(obj.utctimetuple())
pts = timestamp_seconds * 1e3 + getattr(obj, 'microsecond', 0) / 1e3
@@ -267,7 +270,7 @@ class DateIO(_GraphBinaryTypeIO):
ts = int(round(pts))
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, struct.pack(">q", ts))
+ None, nullable, struct.pack(">q", ts))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -283,18 +286,18 @@ class TimestampIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.timestamp
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
# Java timestamp expects milliseconds integer - Have to use int because of legacy Python
ts = int(round(obj * 1000))
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, struct.pack(">q", ts))
+ None, nullable, struct.pack(">q", ts))
@classmethod
def objectify(cls, buff, reader, nullable=True):
# Python timestamp expects seconds
return cls.is_null(buff, reader, lambda b, r: statics.timestamp(struct.unpack(">q", b.read(8))[0] / 1000.0),
nullable)
-
+
def _long_bits_to_double(bits):
return struct.unpack('d', struct.pack('Q', bits))[0]
@@ -313,19 +316,19 @@ class FloatIO(LongIO):
byte_format = ">f"
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
if math.isnan(obj):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, struct.pack(cls.byte_format, NAN))
+ None, nullable, struct.pack(cls.byte_format, NAN))
elif math.isinf(obj) and obj > 0:
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, struct.pack(cls.byte_format, POSITIVE_INFINITY))
+ None, nullable, struct.pack(cls.byte_format, POSITIVE_INFINITY))
elif math.isinf(obj) and obj < 0:
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, struct.pack(cls.byte_format, NEGATIVE_INFINITY))
+ None, nullable, struct.pack(cls.byte_format, NEGATIVE_INFINITY))
else:
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, struct.pack(cls.byte_format, obj))
+ None, nullable, struct.pack(cls.byte_format, obj))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -346,23 +349,15 @@ class DoubleIO(FloatIO):
return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(8))[0], nullable)
-class TypeSerializer(_GraphBinaryTypeIO):
- python_type = TypeType
-
- @classmethod
- def dictify(cls, typ, writer, as_value=False):
- return writer.toDict(typ())
-
-
class StringIO(_GraphBinaryTypeIO):
python_type = str
graphbinary_type = DataType.string
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- len(obj), obj.encode("utf-8"))
+ len(obj), nullable, obj.encode("utf-8"))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -375,13 +370,13 @@ class ListIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.list
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
list_data = bytearray()
for item in obj:
list_data.extend(writer.writeObject(item))
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- len(obj), list_data)
+ len(obj), nullable, list_data)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -414,14 +409,14 @@ class MapIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.map
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
map_data = bytearray()
for k, v in obj.items():
map_data.extend(writer.writeObject(k))
map_data.extend(writer.writeObject(v))
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- len(obj), map_data)
+ len(obj), nullable, map_data)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -446,9 +441,9 @@ class UuidIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.uuid
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
- None, obj.bytes)
+ None, nullable, obj.bytes)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -461,7 +456,7 @@ class EdgeIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.edge
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(writer.writeObject(obj.id))
ba.extend(cls.string_as_bytes(obj.label))
@@ -469,9 +464,9 @@ class EdgeIO(_GraphBinaryTypeIO):
ba.extend(cls.string_as_bytes(obj.inV.label))
ba.extend(writer.writeObject(obj.outV.id))
ba.extend(cls.string_as_bytes(obj.outV.label))
- ba.extend([DataType.null.value])
- ba.extend([DataType.null.value])
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+ ba.extend(NULL_BYTES)
+ ba.extend(NULL_BYTES)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -483,7 +478,7 @@ class EdgeIO(_GraphBinaryTypeIO):
edgelbl = cls.read_string(b)
edge = Edge(edgeid, Vertex(r.readObject(b), cls.read_string(b)),
edgelbl, Vertex(r.readObject(b), cls.read_string(b)))
- b.read(2)
+ b.read(4)
return edge
@@ -493,11 +488,11 @@ class PathIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.path
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(writer.writeObject(obj.labels))
ba.extend(writer.writeObject(obj.objects))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -510,12 +505,12 @@ class PropertyIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.property
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(cls.string_as_bytes(obj.key))
ba.extend(writer.writeObject(obj.value))
- ba.extend([DataType.null.value])
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+ ba.extend(NULL_BYTES)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -524,7 +519,7 @@ class PropertyIO(_GraphBinaryTypeIO):
@classmethod
def _read_property(cls, b, r):
p = Property(cls.read_string(b), r.readObject(b), None)
- b.read(1)
+ b.read(2)
return p
@@ -534,7 +529,7 @@ class TinkerGraphIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.graph
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
raise AttributeError("TinkerGraph serialization is not currently supported by gremlin-python")
@classmethod
@@ -548,12 +543,12 @@ class VertexIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.vertex
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(writer.writeObject(obj.id))
ba.extend(cls.string_as_bytes(obj.label))
- ba.extend([DataType.null.value])
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+ ba.extend(NULL_BYTES)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -562,7 +557,7 @@ class VertexIO(_GraphBinaryTypeIO):
@classmethod
def _read_vertex(cls, b, r):
vertex = Vertex(r.readObject(b), cls.read_string(b))
- b.read(1)
+ b.read(2)
return vertex
@@ -572,14 +567,14 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.vertexproperty
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(writer.writeObject(obj.id))
ba.extend(cls.string_as_bytes(obj.label))
ba.extend(writer.writeObject(obj.value))
- ba.extend([DataType.null.value])
- ba.extend([DataType.null.value])
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+ ba.extend(NULL_BYTES)
+ ba.extend(NULL_BYTES)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -588,18 +583,17 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
@classmethod
def _read_vertexproperty(cls, b, r):
vp = VertexProperty(r.readObject(b), cls.read_string(b), r.readObject(b), None)
- b.read(1)
- b.read(1)
+ b.read(4)
return vp
class _EnumIO(_GraphBinaryTypeIO):
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(cls.string_as_bytes(cls.unmangleKeyword(str(obj.name))))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -652,11 +646,11 @@ class BindingIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.binding
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(cls.string_as_bytes(obj.key))
ba.extend(writer.writeObject(obj.value))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -668,7 +662,7 @@ class BytecodeIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.bytecode
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(struct.pack(">i", len(obj.step_instructions)))
for inst in obj.step_instructions:
@@ -684,9 +678,12 @@ class BytecodeIO(_GraphBinaryTypeIO):
ba.extend(cls.string_as_bytes(inst_name))
ba.extend(struct.pack(">i", len(inst_args)))
for arg in inst_args:
- ba.extend(writer.writeObject(arg))
+ if isinstance(arg, TypeType):
+ ba.extend(writer.writeObject(GremlinType(arg().fqcn)))
+ else:
+ ba.extend(writer.writeObject(arg))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -729,7 +726,7 @@ class LambdaIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.lambda_
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray() if as_value else bytearray([cls.graphbinary_type.value])
lambda_result = obj()
script = lambda_result if isinstance(lambda_result, str) else lambda_result[0]
@@ -748,7 +745,7 @@ class LambdaIO(_GraphBinaryTypeIO):
ba.extend(cls.string_as_bytes(script_cleaned))
ba.extend(struct.pack(">i", script_args))
- return ba
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
class PIO(_GraphBinaryTypeIO):
@@ -756,16 +753,25 @@ class PIO(_GraphBinaryTypeIO):
python_type = P
@classmethod
- def dictify(cls, obj, writer, as_value=False):
- ba = bytearray() if as_value else bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
+ ba = bytearray()
ba.extend(cls.string_as_bytes(obj.operator))
- additional = [writer.writeObject(obj.value), writer.writeObject(obj.other)] \
- if obj.other is not None else [writer.writeObject(obj.value)]
- ba.extend(struct.pack(">i", len(additional)))
- for a in additional:
- ba.extend(a)
+
+ args = []
+ if obj.other is None:
+ if isinstance(obj.value, ListType):
+ args = obj.value
+ else:
+ args.append(obj.value)
+ else:
+ args.append(obj.value)
+ args.append(obj.other)
+
+ ba.extend(struct.pack(">i", len(args)))
+ for a in args:
+ ba.extend(writer.writeObject(a))
- return ba
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
class ScopeIO(_EnumIO):
@@ -783,11 +789,11 @@ class TraverserIO(_GraphBinaryTypeIO):
python_type = Traverser
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(struct.pack(">q", obj.bulk))
ba.extend(writer.writeObject(obj.object))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -805,8 +811,8 @@ class ByteIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.byte
@classmethod
- def dictify(cls, obj, writer, as_value=False):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, struct.pack(">b", obj))
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, struct.pack(">b", obj))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -820,8 +826,8 @@ class ByteBufferIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.bytebuffer
@classmethod
- def dictify(cls, obj, writer, as_value=False):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), len(obj), obj)
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), len(obj), nullable, obj)
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -838,9 +844,9 @@ class BooleanIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.boolean
@classmethod
- def dictify(cls, obj, writer, as_value=False):
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
return cls.as_bytes(cls.write_as_value(
- cls.graphbinary_type, as_value), None, struct.pack(">b", 0x01 if obj else 0x00))
+ cls.graphbinary_type, as_value), None, nullable, struct.pack(">b", 0x01 if obj else 0x00))
@classmethod
def objectify(cls, buff, reader, nullable=True):
@@ -854,16 +860,25 @@ class TextPIO(_GraphBinaryTypeIO):
python_type = TextP
@classmethod
- def dictify(cls, obj, writer, as_value=False):
- ba = bytearray() if as_value else bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
+ ba = bytearray()
ba.extend(cls.string_as_bytes(obj.operator))
- additional = [writer.writeObject(obj.value), writer.writeObject(obj.other)] \
- if obj.other is not None else [writer.writeObject(obj.value)]
- ba.extend(struct.pack(">i", len(additional)))
- for a in additional:
- ba.extend(a)
- return ba
+ args = []
+ if obj.other is None:
+ if isinstance(obj.value, ListType):
+ args = obj.value
+ else:
+ args.append(obj.value)
+ else:
+ args.append(obj.value)
+ args.append(obj.other)
+
+ ba.extend(struct.pack(">i", len(args)))
+ for a in args:
+ ba.extend(writer.writeObject(a))
+
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
class BulkSetIO(_GraphBinaryTypeIO):
@@ -886,3 +901,72 @@ class BulkSetIO(_GraphBinaryTypeIO):
size = size - 1
return the_list
+
+
+class MetricsIO(_GraphBinaryTypeIO):
+
+ graphbinary_type = DataType.metrics
+
+ @classmethod
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_metrics, nullable)
+
+ @classmethod
+ def _read_metrics(cls, b, r):
+ metricid = cls.read_string(b)
+ name = cls.read_string(b)
+ duration = r.toObject(b, DataType.long, nullable=False)
+ counts = r.toObject(b, DataType.map, nullable=False)
+ annotations = r.toObject(b, DataType.map, nullable=False)
+ metrics = r.toObject(b, DataType.list, nullable=False)
+
+ return {"id": metricid,
+ "name": name,
+ "dur": duration,
+ "counts": counts,
+ "annotations": annotations,
+ "metrics": metrics}
+
+
+class TraversalMetricsIO(_GraphBinaryTypeIO):
+
+ graphbinary_type = DataType.traversalmetrics
+
+ @classmethod
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_traversalmetrics, nullable)
+
+ @classmethod
+ def _read_traversalmetrics(cls, b, r):
+ duration = r.toObject(b, DataType.long, nullable=False)
+ metrics = r.toObject(b, DataType.list, nullable=False)
+
+ return {"dur": duration,
+ "metrics": metrics}
+
+
+class ClassIO(_GraphBinaryTypeIO):
+ graphbinary_type = DataType.clazz
+ python_type = GremlinType
+
+ @classmethod
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
+ return cls.as_bytes(cls.write_as_value(
+ cls.graphbinary_type, as_value), None, nullable, StringIO.dictify(obj.gremlin_type, writer, True, False))
+
+
+class TraversalStrategyIO(_GraphBinaryTypeIO):
+ graphbinary_type = DataType.traversalstrategy
+ python_type = TraversalStrategy
+
+ @classmethod
+ def dictify(cls, obj, writer, as_value=False, nullable=True):
+ ba = bytearray()
+ ba.extend(ClassIO.dictify(GremlinType(obj.fqcn), writer, True, False))
+ conf = {k: cls._convert(v) for k, v in obj.configuration.items()}
+ ba.extend(MapIO.dictify(conf, writer, True, False))
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
+
+ @classmethod
+ def _convert(cls, v):
+ return v.bytecode if isinstance(v, Traversal) else v
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphsonV3d0.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphsonV3d0.py
index 8c3cd9f..10a74da 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphsonV3d0.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphsonV3d0.py
@@ -711,6 +711,7 @@ class TDeserializer(_GraphSONTypeIO):
def objectify(cls, d, reader):
return T[d]
+
class TraversalMetricsDeserializer(_GraphSONTypeIO):
graphson_type = "g:TraversalMetrics"
diff --git a/gremlin-python/src/main/jython/radish/terrain.py b/gremlin-python/src/main/jython/radish/terrain.py
index 9122eac..245fd9b 100644
--- a/gremlin-python/src/main/jython/radish/terrain.py
+++ b/gremlin-python/src/main/jython/radish/terrain.py
@@ -89,6 +89,8 @@ def __create_remote(server_graph_name):
if world.config.user_data["serializer"] == "application/vnd.gremlin-v3.0+json":
s = serializer.GraphSONSerializersV3d0()
+ elif world.config.user_data["serializer"] == "application/vnd.graphbinary-v1.0":
+ s = serializer.GraphBinarySerializersV1()
else:
raise ValueError('serializer not found - ' + world.config.user_data["serializer"])
diff --git a/gremlin-python/src/main/jython/tests/conftest.py b/gremlin-python/src/main/jython/tests/conftest.py
index fb31c31..9597f4d 100644
--- a/gremlin-python/src/main/jython/tests/conftest.py
+++ b/gremlin-python/src/main/jython/tests/conftest.py
@@ -29,7 +29,7 @@ from gremlin_python.driver.driver_remote_connection import (
from gremlin_python.driver.protocol import GremlinServerWSProtocol
from gremlin_python.driver.serializer import (
GraphSONMessageSerializer, GraphSONSerializersV2d0, GraphSONSerializersV3d0,
- GraphBinaryMessageSerializerV1)
+ GraphBinarySerializersV1)
from gremlin_python.driver.tornado.transport import TornadoTransport
gremlin_server_host = "localhost"
@@ -88,7 +88,7 @@ def remote_connection(request):
try:
if request.param == 'graphbinaryv1':
remote_conn = DriverRemoteConnection(gremlin_server_url, 'gmodern',
- message_serializer=serializer.GraphBinaryMessageSerializerV1())
+ message_serializer=serializer.GraphBinarySerializersV1())
elif request.param == 'graphsonv2':
remote_conn = DriverRemoteConnection(gremlin_server_url, 'gmodern',
message_serializer=serializer.GraphSONSerializersV2d0())
@@ -129,4 +129,4 @@ def graphson_serializer_v3(request):
@pytest.fixture
def graphbinary_serializer_v1(request):
- return GraphBinaryMessageSerializerV1()
+ return GraphBinarySerializersV1()
diff --git a/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py b/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py
index 2ecd113..b2b4a7d 100644
--- a/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py
+++ b/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py
@@ -16,7 +16,6 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
'''
-import pytest
from tornado import ioloop, gen
@@ -25,7 +24,6 @@ from gremlin_python.statics import long
from gremlin_python.driver.driver_remote_connection import (
DriverRemoteConnection)
from gremlin_python.process.traversal import Traverser
-from gremlin_python.process.traversal import TraversalStrategy
from gremlin_python.process.traversal import P
from gremlin_python.process.graph_traversal import __
from gremlin_python.process.anonymous_traversal import traversal
@@ -38,7 +36,6 @@ __author__ = 'Marko A. Rodriguez (http://markorodriguez.com)'
class TestDriverRemoteConnection(object):
def test_traversals(self, remote_connection):
statics.load_statics(globals())
- assert "remoteconnection[ws://localhost:45940/gremlin,gmodern]" == str(remote_connection)
g = traversal().withRemote(remote_connection)
assert long(6) == g.V().count().toList()[0]
@@ -60,8 +57,7 @@ class TestDriverRemoteConnection(object):
assert 4 == g.V()[2:].count().next()
assert 2 == g.V()[:2].count().next()
# #
- results = g.withSideEffect('a', ['josh', 'peter']).V(1).out('created').in_('created').values('name').where(
- within('a')).toList()
+ results = g.withSideEffect('a', ['josh', 'peter']).V(1).out('created').in_('created').values('name').where(P.within('a')).toList()
assert 2 == len(results)
assert 'josh' in results
assert 'peter' in results
@@ -94,7 +90,6 @@ class TestDriverRemoteConnection(object):
def test_iteration(self, remote_connection):
statics.load_statics(globals())
- assert "remoteconnection[ws://localhost:45940/gremlin,gmodern]" == str(remote_connection)
g = traversal().withRemote(remote_connection)
t = g.V().count()
@@ -134,16 +129,6 @@ class TestDriverRemoteConnection(object):
statics.load_statics(globals())
#
g = traversal().withRemote(remote_connection). \
- withStrategies(TraversalStrategy("SubgraphStrategy",
- {"vertices": __.hasLabel("person"),
- "edges": __.hasLabel("created")}))
- assert 4 == g.V().count().next()
- assert 0 == g.E().count().next()
- assert 1 == g.V().label().dedup().count().next()
- assert 4 == g.V().filter(lambda: ("lambda x: True", "gremlin-python")).count().next()
- assert "person" == g.V().label().dedup().next()
- #
- g = traversal().withRemote(remote_connection). \
withStrategies(SubgraphStrategy(vertices=__.hasLabel("person"), edges=__.hasLabel("created")))
assert 4 == g.V().count().next()
assert 0 == g.E().count().next()
@@ -168,118 +153,6 @@ class TestDriverRemoteConnection(object):
assert 6 == g.V().count().next()
assert 6 == g.E().count().next()
- def test_side_effects(self, remote_connection):
- statics.load_statics(globals())
- #
- g = traversal().withRemote(remote_connection)
- ###
- t = g.V().hasLabel("project").name.iterate()
- assert 0 == len(t.side_effects.keys())
- with pytest.raises(Exception):
- m = t.side_effects["m"]
- ###
- t = g.V().out("created").groupCount("m").by("name")
- results = t.toSet()
- assert 2 == len(results)
- assert Vertex(3) in results
- assert Vertex(5) in results
- assert 1 == len(t.side_effects.keys())
- assert "m" in t.side_effects.keys()
- m = t.side_effects["m"]
- assert isinstance(m, dict)
- assert 2 == len(m)
- assert 3 == m["lop"]
- assert 1 == m["ripple"]
- assert isinstance(m["lop"], long)
- assert isinstance(m["ripple"], long)
-
- # check status attributes
- assert "host" in t.side_effects.status_attributes
-
- ##
- t = g.V().out("created").groupCount("m").by("name").name.aggregate("n")
- results = t.toSet()
- assert 2 == len(results)
- assert "lop" in results
- assert "ripple" in results
- assert 2 == len(t.side_effects.keys())
- assert "m" in t.side_effects.keys()
- assert "n" in t.side_effects.keys()
- n = t.side_effects.get("n")
- assert isinstance(n, dict)
- assert 2 == len(n)
- assert "lop" in n.keys()
- assert "ripple" in n.keys()
- assert 3 == n["lop"]
- assert 1 == n["ripple"]
-
- t = g.withSideEffect('m', 32).V().map(lambda: "x: x.sideEffects('m')")
- results = t.toSet()
- assert 1 == len(results)
- assert 32 == list(results)[0]
- assert 32 == t.side_effects['m']
- assert 1 == len(t.side_effects.keys())
- with pytest.raises(Exception):
- x = t.side_effects["x"]
-
- a = g.V().has("name", "marko").next()
- b = g.V().has("name", "peter").next()
- edge = g.withSideEffect("b", b).V(a).addE("knows").to("b").next()
- assert "knows" == edge.label
- assert a == edge.outV
- assert b == edge.inV
- g.V().has("name", "marko").outE("knows").where(__.inV().has("name", "peter")).drop().iterate()
- ##
- edge = g.withSideEffect("a", a).withSideEffect("b", b).V().limit(1).addE("knows").from_("a").to("b").next()
- assert "knows" == edge.label
- assert a == edge.outV
- assert b == edge.inV
- g.V().has("name", "marko").outE("knows").where(__.inV().has("name", "peter")).drop().iterate()
-
- def test_side_effect_close(self, remote_connection):
- g = traversal().withRemote(remote_connection)
- t = g.V().aggregate('a').aggregate('b')
- t.toList()
-
- # The 'a' key should return some side effects
- results = t.side_effects.get('a')
- assert results
-
- # Close result is None
- results = t.side_effects.close()
- assert not results
-
- # Shouldn't get any new info from server
- # 'b' isn't in local cache
- results = t.side_effects.get('b')
- assert not results
-
- # But 'a' should still be cached locally
- results = t.side_effects.get('a')
- assert results
-
- # 'a' should have been added to local keys cache, but not 'b'
- results = t.side_effects.keys()
- assert len(results) == 1
- a, = results
- assert a == 'a'
-
- # Try to get 'b' directly from server, should throw error
- with pytest.raises(Exception):
- t.side_effects.value_lambda('b')
-
- def test_promise(self, remote_connection):
- g = traversal().withRemote(remote_connection)
- future = g.V().aggregate('a').promise()
- t = future.result()
- assert len(t.toList()) == 6
- a, = t.side_effects.keys()
- assert a == 'a'
- results = t.side_effects.get('a')
- assert results
- results = t.side_effects.close()
- assert not results
-
def test_in_tornado_app(remote_connection):
# Make sure nothing weird with loops
[tinkerpop] 12/33: Added Graph/Vertex/VertexProperty to python
graphbinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit f498886acd091d65e62973f3d3abffc45f477007
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Wed Jul 31 14:06:23 2019 -0400
Added Graph/Vertex/VertexProperty to python graphbinary
---
.../gremlin_python/structure/io/graphbinaryV1.py | 62 +++++++++++++++++++++-
.../tests/structure/io/test_graphbinaryV1.py | 10 ++++
2 files changed, 71 insertions(+), 1 deletion(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index c12ffca..551db60 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -37,7 +37,7 @@ from isodate import parse_duration, duration_isoformat
from gremlin_python import statics
from gremlin_python.statics import FloatType, FunctionType, IntType, LongType, TypeType, DictType, ListType, SetType, SingleByte, ByteBufferType, SingleChar
from gremlin_python.process.traversal import Binding, Bytecode, P, TextP, Traversal, Traverser, TraversalStrategy, T
-from gremlin_python.structure.graph import Edge, Property, Vertex, VertexProperty, Path
+from gremlin_python.structure.graph import Graph, Edge, Property, Vertex, VertexProperty, Path
log = logging.getLogger(__name__)
@@ -65,6 +65,9 @@ class DataType(Enum):
edge = 0x0d
path = 0x0e
property = 0x0f
+ tinkergraph = 0x10
+ vertex = 0x11
+ vertexproperty = 0x12
class GraphBinaryTypeType(type):
@@ -444,3 +447,60 @@ class PropertyIO(_GraphBinaryTypeIO):
p = Property(cls.read_string(b), reader.readObject(b), None)
b.read(1)
return p
+
+
+class TinkerGraphIO(_GraphBinaryTypeIO):
+
+ python_type = Graph
+ graphbinary_type = DataType.tinkergraph
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ raise AttributeError("TinkerGraph serialization is not currently supported by gremlin-python")
+
+ @classmethod
+ def objectify(cls, b, reader):
+ raise AttributeError("TinkerGraph deserialization is not currently supported by gremlin-python")
+
+
+class VertexIO(_GraphBinaryTypeIO):
+
+ python_type = Vertex
+ graphbinary_type = DataType.vertex
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(writer.writeObject(obj.id))
+ ba.extend(cls.string_as_bytes(obj.label))
+ ba.extend([DataType.null.value])
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ vertex = Vertex(reader.readObject(b), cls.read_string(b))
+ b.read(1)
+ return vertex
+
+
+class VertexPropertyIO(_GraphBinaryTypeIO):
+
+ python_type = VertexProperty
+ graphbinary_type = DataType.vertexproperty
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(writer.writeObject(obj.id))
+ ba.extend(cls.string_as_bytes(obj.label))
+ ba.extend(writer.writeObject(obj.value))
+ ba.extend([DataType.null.value])
+ ba.extend([DataType.null.value])
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ vp = VertexProperty(reader.readObject(b), cls.read_string(b), reader.readObject(b), None)
+ b.read(1)
+ b.read(1)
+ return vp
\ No newline at end of file
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 3c4ef34..cdf52fc 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -139,3 +139,13 @@ class TestGraphSONWriter(object):
x = Property("name", "stephen", None)
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+
+ def test_vertex(self):
+ x = Vertex(123, "person")
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ def test_vertexproperty(self):
+ x = VertexProperty(123, "name", "stephen", None)
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
[tinkerpop] 22/33: Improved null handling in graphbinary for python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 8543b87100dd12d24663518ccdd94005445aff25
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Thu Aug 8 13:42:12 2019 -0400
Improved null handling in graphbinary for python
---
.../jython/gremlin_python/driver/serializer.py | 30 +---
.../gremlin_python/structure/io/graphbinaryV1.py | 158 +++++++++++----------
2 files changed, 88 insertions(+), 100 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py b/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
index be28140..f805220 100644
--- a/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
+++ b/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
@@ -16,6 +16,7 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
"""
+
try:
import ujson as json
except ImportError:
@@ -243,34 +244,13 @@ class GraphBinaryMessageSerializerV1(object):
def deserialize_message(self, message):
b = io.BytesIO(message)
- #TODO: lots of hardcode null checks need better resolution
-
b.read(1) # version
- b.read(1) # requestid nullable
- request_id = str(uuid.UUID(bytes=b.read(16))) # result queue uses string as a key
-
+ request_id = str(self._graphbinary_reader.toObject(b, graphbinaryV1.DataType.uuid))
status_code = struct.unpack(">i", b.read(4))[0]
-
- b.read(1) # status message nullable
- status_msg = b.read(struct.unpack(">i", b.read(4))[0]).decode("utf-8")
-
- attr_count = struct.unpack(">i", b.read(4))[0]
- status_attrs = {}
- while attr_count > 0:
- k = self._graphbinary_reader.toObject(b)
- v = self._graphbinary_reader.toObject(b)
- status_attrs[k] = v
- attr_count = attr_count - 1
-
- meta_count = struct.unpack(">i", b.read(4))[0]
- meta_attrs = {}
- while meta_count > 0:
- k = self._graphbinary_reader.toObject(b)
- v = self._graphbinary_reader.toObject(b)
- meta_attrs[k] = v
- meta_count = meta_count - 1
-
+ status_msg = self._graphbinary_reader.toObject(b, graphbinaryV1.DataType.string)
+ status_attrs = self._graphbinary_reader.toObject(b, graphbinaryV1.DataType.map, nullable=False)
+ meta_attrs = self._graphbinary_reader.toObject(b, graphbinaryV1.DataType.map, nullable=False)
result = self._graphbinary_reader.toObject(b)
msg = {'requestId': request_id,
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 045c7ed..f34a3d4 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -151,13 +151,14 @@ class GraphBinaryReader(object):
elif isinstance(b, io.BufferedIOBase):
return self.toObject(b)
- def toObject(self, buff):
- bt = buff.read(1)
- if bt[0] == DataType.null.value:
- return None
-
- bt_value = struct.unpack('>b', bt)[0]
- return self.deserializers[DataType(bt_value)].objectify(buff, self)
+ def toObject(self, buff, data_type=None, nullable=True):
+ if data_type is None:
+ bt = buff.read(1)
+ if bt[0] == DataType.null.value:
+ return None
+ return self.deserializers[DataType(struct.unpack('>b', bt)[0])].objectify(buff, self, nullable)
+ else:
+ return self.deserializers[data_type].objectify(buff, self, nullable)
@six.add_metaclass(GraphBinaryTypeType)
@@ -211,13 +212,13 @@ class _GraphBinaryTypeIO(object):
return None if as_value else graph_binary_type
@classmethod
- def is_null(cls, buff, reader, else_opt):
- return None if buff.read(1)[0] == 0x01 else else_opt(buff, reader)
+ def is_null(cls, buff, reader, else_opt, nullable=True):
+ return None if nullable and buff.read(1)[0] == 0x01 else else_opt(buff, reader)
def dictify(self, obj, writer, as_value=False):
raise NotImplementedError()
- def objectify(self, d, reader, as_value=False):
+ def objectify(self, d, reader, nullable=True):
raise NotImplementedError()
@@ -236,8 +237,8 @@ class LongIO(_GraphBinaryTypeIO):
None, struct.pack(cls.byte_format, obj))
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(8))[0])
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(8))[0], nullable)
class IntIO(LongIO):
@@ -247,8 +248,8 @@ class IntIO(LongIO):
byte_format = ">i"
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: cls.read_int(b))
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, lambda b, r: cls.read_int(b), nullable)
class DateIO(_GraphBinaryTypeIO):
@@ -263,15 +264,16 @@ class DateIO(_GraphBinaryTypeIO):
pts = timestamp_seconds * 1e3 + getattr(obj, 'microsecond', 0) / 1e3
except AttributeError:
pts = calendar.timegm(obj.timetuple()) * 1e3
-
+
ts = int(round(pts))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
None, struct.pack(">q", ts))
@classmethod
- def objectify(cls, buff, reader, as_value=False):
+ def objectify(cls, buff, reader, nullable=True):
return cls.is_null(buff, reader,
- lambda b, r: datetime.datetime.utcfromtimestamp(struct.unpack(">q", b.read(8))[0] / 1000.0))
+ lambda b, r: datetime.datetime.utcfromtimestamp(struct.unpack(">q", b.read(8))[0] / 1000.0),
+ nullable)
# Based on current implementation, this class must always be declared before FloatIO.
@@ -284,13 +286,14 @@ class TimestampIO(_GraphBinaryTypeIO):
def dictify(cls, obj, writer, as_value=False):
# Java timestamp expects milliseconds integer - Have to use int because of legacy Python
ts = int(round(obj * 1000))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
None, struct.pack(">q", ts))
@classmethod
- def objectify(cls, buff, reader):
+ def objectify(cls, buff, reader, nullable=True):
# Python timestamp expects seconds
- return cls.is_null(buff, reader, lambda b, r: statics.timestamp(struct.unpack(">q", b.read(8))[0] / 1000.0))
+ return cls.is_null(buff, reader, lambda b, r: statics.timestamp(struct.unpack(">q", b.read(8))[0] / 1000.0),
+ nullable)
def _long_bits_to_double(bits):
@@ -312,35 +315,35 @@ class FloatIO(LongIO):
@classmethod
def dictify(cls, obj, writer, as_value=False):
if math.isnan(obj):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
None, struct.pack(cls.byte_format, NAN))
elif math.isinf(obj) and obj > 0:
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
None, struct.pack(cls.byte_format, POSITIVE_INFINITY))
elif math.isinf(obj) and obj < 0:
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
None, struct.pack(cls.byte_format, NEGATIVE_INFINITY))
else:
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
None, struct.pack(cls.byte_format, obj))
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(4))[0])
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(4))[0], nullable)
class DoubleIO(FloatIO):
"""
Floats basically just fall through to double serialization.
"""
-
+
graphbinary_type = DataType.double
graphbinary_base_type = DataType.double
byte_format = ">d"
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(8))[0])
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(8))[0], nullable)
class TypeSerializer(_GraphBinaryTypeIO):
@@ -358,12 +361,12 @@ class StringIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
len(obj), obj.encode("utf-8"))
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: b.read(cls.read_int(b)).decode("utf-8"))
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, lambda b, r: b.read(cls.read_int(b)).decode("utf-8"), nullable)
class ListIO(_GraphBinaryTypeIO):
@@ -377,12 +380,12 @@ class ListIO(_GraphBinaryTypeIO):
for item in obj:
list_data.extend(writer.writeObject(item))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
len(obj), list_data)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, cls._read_list)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_list, nullable)
@classmethod
def _read_list(cls, b, r):
@@ -401,8 +404,8 @@ class SetIO(ListIO):
graphbinary_type = DataType.set
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return set(ListIO.objectify(buff, reader, as_value))
+ def objectify(cls, buff, reader, nullable=True):
+ return set(ListIO.objectify(buff, reader, nullable))
class MapIO(_GraphBinaryTypeIO):
@@ -417,12 +420,12 @@ class MapIO(_GraphBinaryTypeIO):
map_data.extend(writer.writeObject(k))
map_data.extend(writer.writeObject(v))
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
len(obj), map_data)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, cls._read_map)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_map, nullable)
@classmethod
def _read_map(cls, b, r):
@@ -448,8 +451,8 @@ class UuidIO(_GraphBinaryTypeIO):
None, obj.bytes)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: uuid.UUID(bytes=b.read(16)))
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, lambda b, r: uuid.UUID(bytes=b.read(16)), nullable)
class EdgeIO(_GraphBinaryTypeIO):
@@ -471,8 +474,8 @@ class EdgeIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, cls._read_edge)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_edge, nullable)
@classmethod
def _read_edge(cls, b, r):
@@ -497,8 +500,8 @@ class PathIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: Path(r.readObject(b), r.readObject(b)))
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, lambda b, r: Path(r.readObject(b), r.readObject(b)), nullable)
class PropertyIO(_GraphBinaryTypeIO):
@@ -515,8 +518,8 @@ class PropertyIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, cls._read_property)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_property, nullable)
@classmethod
def _read_property(cls, b, r):
@@ -553,8 +556,8 @@ class VertexIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, cls._read_vertex)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_vertex, nullable)
@classmethod
def _read_vertex(cls, b, r):
@@ -570,7 +573,7 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False):
- ba = bytearray()
+ ba = bytearray()
ba.extend(writer.writeObject(obj.id))
ba.extend(cls.string_as_bytes(obj.label))
ba.extend(writer.writeObject(obj.value))
@@ -579,8 +582,8 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, cls._read_vertexproperty)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_vertexproperty, nullable)
@classmethod
def _read_vertexproperty(cls, b, r):
@@ -599,8 +602,8 @@ class _EnumIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: cls.python_type[cls.read_string(b)])
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, lambda b, r: cls.python_type[cls.read_string(b)], nullable)
class BarrierIO(_EnumIO):
@@ -644,7 +647,7 @@ class PopIO(_EnumIO):
class BindingIO(_GraphBinaryTypeIO):
-
+
python_type = Binding
graphbinary_type = DataType.binding
@@ -654,16 +657,16 @@ class BindingIO(_GraphBinaryTypeIO):
ba.extend(cls.string_as_bytes(obj.key))
ba.extend(writer.writeObject(obj.value))
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
-
+
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: Binding(cls.read_string(b), reader.readObject(b)))
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, lambda b, r: Binding(cls.read_string(b), reader.readObject(b)), nullable)
class BytecodeIO(_GraphBinaryTypeIO):
python_type = Bytecode
graphbinary_type = DataType.bytecode
-
+
@classmethod
def dictify(cls, obj, writer, as_value=False):
ba = bytearray()
@@ -686,13 +689,13 @@ class BytecodeIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, cls._read_bytecode)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_bytecode, nullable)
@classmethod
def _read_bytecode(cls, b, r):
bytecode = Bytecode()
-
+
step_count = cls.read_int(b)
ix = 0
while ix < step_count:
@@ -787,8 +790,8 @@ class TraverserIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, cls._read_traverser)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_traverser, nullable)
@classmethod
def _read_traverser(cls, b, r):
@@ -806,8 +809,10 @@ class ByteIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, struct.pack(">b", obj))
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: int.__new__(SingleByte, struct.unpack_from(">b", b.read(1))[0]))
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader,
+ lambda b, r: int.__new__(SingleByte, struct.unpack_from(">b", b.read(1))[0]),
+ nullable)
class ByteBufferIO(_GraphBinaryTypeIO):
@@ -819,8 +824,8 @@ class ByteBufferIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), len(obj), obj)
@classmethod
- def objectify(cls, buff, reader):
- return cls.is_null(buff, reader, cls._read_bytebuffer)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_bytebuffer, nullable)
@classmethod
def _read_bytebuffer(cls, b, r):
@@ -834,11 +839,14 @@ class BooleanIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False):
- return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, struct.pack(">b", 0x01 if obj else 0x00))
+ return cls.as_bytes(cls.write_as_value(
+ cls.graphbinary_type, as_value), None, struct.pack(">b", 0x01 if obj else 0x00))
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, lambda b, r: True if struct.unpack_from(">b", b.read(1))[0] == 0x01 else False)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader,
+ lambda b, r: True if struct.unpack_from(">b", b.read(1))[0] == 0x01 else False,
+ nullable)
class TextPIO(_GraphBinaryTypeIO):
@@ -863,8 +871,8 @@ class BulkSetIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.bulkset
@classmethod
- def objectify(cls, buff, reader, as_value=False):
- return cls.is_null(buff, reader, cls._read_bulkset)
+ def objectify(cls, buff, reader, nullable=True):
+ return cls.is_null(buff, reader, cls._read_bulkset, nullable)
@classmethod
def _read_bulkset(cls, b, r):
[tinkerpop] 21/33: Major refactoring of GraphBinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 65da362817df2b2623c034c1d6075740bd090f8a
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Wed Aug 7 10:31:28 2019 -0400
Major refactoring of GraphBinary
Better support for null/value flag. Basic traversal connectivity to Gremlin Server working. Still a bit of a working state at this point.
---
.../main/jython/gremlin_python/driver/protocol.py | 5 +-
.../jython/gremlin_python/driver/serializer.py | 125 +++++++-
.../gremlin_python/structure/io/graphbinaryV1.py | 331 +++++++++++++--------
gremlin-python/src/main/jython/tests/conftest.py | 32 +-
.../main/jython/tests/driver/test_serializer.py | 20 +-
.../tests/structure/io/test_functionalityio.py | 64 ++++
.../tests/structure/io/test_graphbinaryV1.py | 2 +-
.../jython/tests/structure/io/test_graphsonV3d0.py | 42 ---
8 files changed, 426 insertions(+), 195 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/driver/protocol.py b/gremlin-python/src/main/jython/gremlin_python/driver/protocol.py
index c033fe1..bb3ab24 100644
--- a/gremlin-python/src/main/jython/gremlin_python/driver/protocol.py
+++ b/gremlin-python/src/main/jython/gremlin_python/driver/protocol.py
@@ -76,9 +76,10 @@ class GremlinServerWSProtocol(AbstractBaseProtocol):
def data_received(self, message, results_dict):
# if Gremlin Server cuts off then we get a None for the message
if message is None:
- raise GremlinServerError({'code': 500, 'message':'Server disconnected - please try to reconnect', 'attributes': {}})
+ raise GremlinServerError({'code': 500,
+ 'message': 'Server disconnected - please try to reconnect', 'attributes': {}})
- message = self._message_serializer.deserialize_message(json.loads(message.decode('utf-8')))
+ message = self._message_serializer.deserialize_message(message)
request_id = message['requestId']
result_set = results_dict[request_id]
status_code = message['status']['code']
diff --git a/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py b/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
index 007e162..be28140 100644
--- a/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
+++ b/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
@@ -20,7 +20,11 @@ try:
import ujson as json
except ImportError:
import json
+import struct
+import uuid
+import io
+from gremlin_python.structure.io import graphbinaryV1
from gremlin_python.structure.io import graphsonV2d0
from gremlin_python.structure.io import graphsonV3d0
@@ -31,7 +35,7 @@ class Processor:
"""Base class for OpProcessor serialization system."""
def __init__(self, writer):
- self._graphson_writer = writer
+ self._writer = writer
def get_op_args(self, op, args):
op_method = getattr(self, op, None)
@@ -56,7 +60,7 @@ class Traversal(Processor):
def bytecode(self, args):
gremlin = args['gremlin']
- args['gremlin'] = self._graphson_writer.toDict(gremlin)
+ args['gremlin'] = self._writer.toDict(gremlin)
aliases = args.get('aliases', '')
if not aliases:
aliases = {'g': 'g'}
@@ -143,7 +147,8 @@ class GraphSONMessageSerializer(object):
return message
def deserialize_message(self, message):
- return self._graphson_reader.toObject(message)
+ msg = json.loads(message.decode('utf-8'))
+ return self._graphson_reader.toObject(msg)
class GraphSONSerializersV2d0(GraphSONMessageSerializer):
@@ -162,3 +167,117 @@ class GraphSONSerializersV3d0(GraphSONMessageSerializer):
writer = graphsonV3d0.GraphSONWriter()
version = b"application/vnd.gremlin-v3.0+json"
super(GraphSONSerializersV3d0, self).__init__(reader, writer, version)
+
+
+class GraphBinaryMessageSerializerV1(object):
+ DEFAULT_READER_CLASS = graphbinaryV1.GraphBinaryReader
+ DEFAULT_WRITER_CLASS = graphbinaryV1.GraphBinaryWriter
+ DEFAULT_VERSION = b"application/vnd.graphbinary-v1.0"
+
+ def __init__(self, reader=None, writer=None, version=None):
+ if not version:
+ version = self.DEFAULT_VERSION
+ self._version = version
+ if not reader:
+ reader = self.DEFAULT_READER_CLASS()
+ self._graphbinary_reader = reader
+ if not writer:
+ writer = self.DEFAULT_WRITER_CLASS()
+ self._graphbinary_writer = writer
+ self.standard = Standard(writer)
+ self.traversal = Traversal(writer)
+
+ @property
+ def version(self):
+ """Read only property"""
+ return self._version
+
+ def get_processor(self, processor):
+ processor = getattr(self, processor, None)
+ if not processor:
+ raise Exception("Unknown processor")
+ return processor
+
+ def serialize_message(self, request_id, request_message):
+ processor = request_message.processor
+ op = request_message.op
+ args = request_message.args
+ if not processor:
+ processor_obj = self.get_processor('standard')
+ else:
+ processor_obj = self.get_processor(processor)
+ args = processor_obj.get_op_args(op, args)
+ message = self.build_message(request_id, processor, op, args)
+ return message
+
+ def build_message(self, request_id, processor, op, args):
+ message = {
+ 'requestId': request_id,
+ 'processor': processor,
+ 'op': op,
+ 'args': args
+ }
+ return self.finalize_message(message, 0x20, self.version)
+
+ def finalize_message(self, message, mime_len, mime_type):
+ ba = bytearray()
+ ba.extend(struct.pack(">b", mime_len))
+ ba.extend(mime_type)
+ ba.extend([0x81])
+ ba.extend(uuid.UUID(message['requestId']).bytes)
+
+ ba.extend(struct.pack(">i", len(message['op'])))
+ ba.extend(message['op'].encode("utf-8"))
+
+ ba.extend(struct.pack(">i", len(message['processor'])))
+ ba.extend(message['processor'].encode("utf-8"))
+
+ args = message["args"]
+ ba.extend(struct.pack(">i", len(args)))
+ for k, v in args.items():
+ ba.extend(self._graphbinary_writer.writeObject(k))
+ ba.extend(self._graphbinary_writer.writeObject(v))
+
+ return bytes(ba)
+
+ def deserialize_message(self, message):
+ b = io.BytesIO(message)
+
+ #TODO: lots of hardcode null checks need better resolution
+
+ b.read(1) # version
+
+ b.read(1) # requestid nullable
+ request_id = str(uuid.UUID(bytes=b.read(16))) # result queue uses string as a key
+
+ status_code = struct.unpack(">i", b.read(4))[0]
+
+ b.read(1) # status message nullable
+ status_msg = b.read(struct.unpack(">i", b.read(4))[0]).decode("utf-8")
+
+ attr_count = struct.unpack(">i", b.read(4))[0]
+ status_attrs = {}
+ while attr_count > 0:
+ k = self._graphbinary_reader.toObject(b)
+ v = self._graphbinary_reader.toObject(b)
+ status_attrs[k] = v
+ attr_count = attr_count - 1
+
+ meta_count = struct.unpack(">i", b.read(4))[0]
+ meta_attrs = {}
+ while meta_count > 0:
+ k = self._graphbinary_reader.toObject(b)
+ v = self._graphbinary_reader.toObject(b)
+ meta_attrs[k] = v
+ meta_count = meta_count - 1
+
+ result = self._graphbinary_reader.toObject(b)
+
+ msg = {'requestId': request_id,
+ 'status': {'code': status_code,
+ 'message': status_msg,
+ 'attributes': status_attrs},
+ 'result': {'meta': meta_attrs,
+ 'data': result}}
+
+ return msg
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index e282dda..045c7ed 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -95,10 +95,10 @@ class DataType(Enum):
textp = 0x28
traversalstrategy = 0x29 #todo
bulkset = 0x2a
- tree = 0x2b
- metrics = 0x2c
- traversalmetrics = 0x2d
- custom = 0x00
+ tree = 0x2b #todo
+ metrics = 0x2c #todo
+ traversalmetrics = 0x2d #todo
+ custom = 0x00 #todo
class GraphBinaryTypeType(type):
@@ -153,6 +153,9 @@ class GraphBinaryReader(object):
def toObject(self, buff):
bt = buff.read(1)
+ if bt[0] == DataType.null.value:
+ return None
+
bt_value = struct.unpack('>b', bt)[0]
return self.deserializers[DataType(bt_value)].objectify(buff, self)
@@ -170,6 +173,10 @@ class _GraphBinaryTypeIO(object):
@classmethod
def as_bytes(cls, graphbin_type=None, size=None, *args):
ba = bytearray() if graphbin_type is None else bytearray([graphbin_type.value])
+
+ # todo: empty value flag just hardcoded in
+ ba.extend(struct.pack(">b", 0))
+
if size is not None:
ba.extend(struct.pack(">i", size))
@@ -182,7 +189,10 @@ class _GraphBinaryTypeIO(object):
@classmethod
def string_as_bytes(cls, s):
- return cls.as_bytes(None, len(s), s.encode("utf-8"))
+ ba = bytearray()
+ ba.extend(struct.pack(">i", len(s)))
+ ba.extend(s.encode("utf-8"))
+ return ba
@classmethod
def read_int(cls, buff):
@@ -195,13 +205,21 @@ class _GraphBinaryTypeIO(object):
@classmethod
def unmangleKeyword(cls, symbol):
return cls.symbolMap.get(symbol, symbol)
+
+ @classmethod
+ def write_as_value(cls, graph_binary_type, as_value):
+ return None if as_value else graph_binary_type
- def dictify(self, obj, writer):
- raise NotImplementedError()
+ @classmethod
+ def is_null(cls, buff, reader, else_opt):
+ return None if buff.read(1)[0] == 0x01 else else_opt(buff, reader)
- def objectify(self, d, reader):
+ def dictify(self, obj, writer, as_value=False):
raise NotImplementedError()
+ def objectify(self, d, reader, as_value=False):
+ raise NotImplementedError()
+
class LongIO(_GraphBinaryTypeIO):
@@ -210,15 +228,16 @@ class LongIO(_GraphBinaryTypeIO):
byte_format = ">q"
@classmethod
- def dictify(cls, obj, writer):
+ def dictify(cls, obj, writer, as_value=False):
if obj < -9223372036854775808 or obj > 9223372036854775807:
raise Exception("TODO: don't forget bigint")
else:
- return cls.as_bytes(cls.graphbinary_type, None, struct.pack(cls.byte_format, obj))
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ None, struct.pack(cls.byte_format, obj))
@classmethod
- def objectify(cls, buff, reader):
- return struct.unpack(cls.byte_format, buff.read(8))[0]
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(8))[0])
class IntIO(LongIO):
@@ -228,8 +247,8 @@ class IntIO(LongIO):
byte_format = ">i"
@classmethod
- def objectify(cls, buff, reader):
- return cls.read_int(buff)
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: cls.read_int(b))
class DateIO(_GraphBinaryTypeIO):
@@ -238,19 +257,21 @@ class DateIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.date
@classmethod
- def dictify(cls, obj, writer):
+ def dictify(cls, obj, writer, as_value=False):
try:
timestamp_seconds = calendar.timegm(obj.utctimetuple())
pts = timestamp_seconds * 1e3 + getattr(obj, 'microsecond', 0) / 1e3
except AttributeError:
pts = calendar.timegm(obj.timetuple()) * 1e3
- ts = int(round(pts * 100))
- return cls.as_bytes(cls.graphbinary_type, None, struct.pack(">q", ts))
+ ts = int(round(pts))
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ None, struct.pack(">q", ts))
@classmethod
- def objectify(cls, buff, reader):
- return datetime.datetime.utcfromtimestamp(struct.unpack(">q", buff.read(8))[0] / 1000.0)
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader,
+ lambda b, r: datetime.datetime.utcfromtimestamp(struct.unpack(">q", b.read(8))[0] / 1000.0))
# Based on current implementation, this class must always be declared before FloatIO.
@@ -260,15 +281,16 @@ class TimestampIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.timestamp
@classmethod
- def dictify(cls, obj, writer):
+ def dictify(cls, obj, writer, as_value=False):
# Java timestamp expects milliseconds integer - Have to use int because of legacy Python
ts = int(round(obj * 1000))
- return cls.as_bytes(cls.graphbinary_type, None, struct.pack(">q", ts))
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ None, struct.pack(">q", ts))
@classmethod
def objectify(cls, buff, reader):
# Python timestamp expects seconds
- return statics.timestamp(struct.unpack(">q", buff.read(8))[0] / 1000.0)
+ return cls.is_null(buff, reader, lambda b, r: statics.timestamp(struct.unpack(">q", b.read(8))[0] / 1000.0))
def _long_bits_to_double(bits):
@@ -288,19 +310,23 @@ class FloatIO(LongIO):
byte_format = ">f"
@classmethod
- def dictify(cls, obj, writer):
+ def dictify(cls, obj, writer, as_value=False):
if math.isnan(obj):
- return cls.as_bytes(cls.graphbinary_base_type, None, struct.pack(cls.byte_format, NAN))
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ None, struct.pack(cls.byte_format, NAN))
elif math.isinf(obj) and obj > 0:
- return cls.as_bytes(cls.graphbinary_base_type, None, struct.pack(cls.byte_format, POSITIVE_INFINITY))
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ None, struct.pack(cls.byte_format, POSITIVE_INFINITY))
elif math.isinf(obj) and obj < 0:
- return cls.as_bytes(cls.graphbinary_base_type, None, struct.pack(cls.byte_format, NEGATIVE_INFINITY))
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ None, struct.pack(cls.byte_format, NEGATIVE_INFINITY))
else:
- return cls.as_bytes(cls.graphbinary_base_type, None, struct.pack(cls.byte_format, obj))
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ None, struct.pack(cls.byte_format, obj))
@classmethod
- def objectify(cls, buff, reader):
- return struct.unpack(cls.byte_format, buff.read(4))[0]
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(4))[0])
class DoubleIO(FloatIO):
@@ -313,15 +339,15 @@ class DoubleIO(FloatIO):
byte_format = ">d"
@classmethod
- def objectify(cls, buff, reader):
- return struct.unpack(cls.byte_format, buff.read(8))[0]
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: struct.unpack(cls.byte_format, b.read(8))[0])
class TypeSerializer(_GraphBinaryTypeIO):
python_type = TypeType
@classmethod
- def dictify(cls, typ, writer):
+ def dictify(cls, typ, writer, as_value=False):
return writer.toDict(typ())
@@ -331,12 +357,13 @@ class StringIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.string
@classmethod
- def dictify(cls, obj, writer):
- return cls.as_bytes(cls.graphbinary_type, len(obj), obj.encode("utf-8"))
+ def dictify(cls, obj, writer, as_value=False):
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ len(obj), obj.encode("utf-8"))
@classmethod
- def objectify(cls, b, reader):
- return cls.read_string(b)
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: b.read(cls.read_int(b)).decode("utf-8"))
class ListIO(_GraphBinaryTypeIO):
@@ -345,19 +372,24 @@ class ListIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.list
@classmethod
- def dictify(cls, obj, writer):
+ def dictify(cls, obj, writer, as_value=False):
list_data = bytearray()
for item in obj:
list_data.extend(writer.writeObject(item))
- return cls.as_bytes(cls.graphbinary_type, len(obj), list_data)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ len(obj), list_data)
@classmethod
- def objectify(cls, buff, reader):
- size = cls.read_int(buff)
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, cls._read_list)
+
+ @classmethod
+ def _read_list(cls, b, r):
+ size = cls.read_int(b)
the_list = []
while size > 0:
- the_list.append(reader.readObject(buff))
+ the_list.append(r.readObject(b))
size = size - 1
return the_list
@@ -369,8 +401,8 @@ class SetIO(ListIO):
graphbinary_type = DataType.set
@classmethod
- def objectify(cls, buff, reader):
- return set(ListIO.objectify(buff, reader))
+ def objectify(cls, buff, reader, as_value=False):
+ return set(ListIO.objectify(buff, reader, as_value))
class MapIO(_GraphBinaryTypeIO):
@@ -379,21 +411,26 @@ class MapIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.map
@classmethod
- def dictify(cls, obj, writer):
+ def dictify(cls, obj, writer, as_value=False):
map_data = bytearray()
for k, v in obj.items():
map_data.extend(writer.writeObject(k))
map_data.extend(writer.writeObject(v))
- return cls.as_bytes(cls.graphbinary_type, len(obj), map_data)
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ len(obj), map_data)
@classmethod
- def objectify(cls, buff, reader):
- size = cls.read_int(buff)
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, cls._read_map)
+
+ @classmethod
+ def _read_map(cls, b, r):
+ size = cls.read_int(b)
the_dict = {}
while size > 0:
- k = reader.readObject(buff)
- v = reader.readObject(buff)
+ k = r.readObject(b)
+ v = r.readObject(b)
the_dict[k] = v
size = size - 1
@@ -406,14 +443,13 @@ class UuidIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.uuid
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
- ba.extend(obj.bytes)
- return ba
+ def dictify(cls, obj, writer, as_value=False):
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value),
+ None, obj.bytes)
@classmethod
- def objectify(cls, b, reader):
- return uuid.UUID(bytes=b.read(16))
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: uuid.UUID(bytes=b.read(16)))
class EdgeIO(_GraphBinaryTypeIO):
@@ -422,8 +458,8 @@ class EdgeIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.edge
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray()
ba.extend(writer.writeObject(obj.id))
ba.extend(cls.string_as_bytes(obj.label))
ba.extend(writer.writeObject(obj.inV.id))
@@ -432,14 +468,18 @@ class EdgeIO(_GraphBinaryTypeIO):
ba.extend(cls.string_as_bytes(obj.outV.label))
ba.extend([DataType.null.value])
ba.extend([DataType.null.value])
- return ba
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, b, reader):
- edgeid = reader.readObject(b)
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, cls._read_edge)
+
+ @classmethod
+ def _read_edge(cls, b, r):
+ edgeid = r.readObject(b)
edgelbl = cls.read_string(b)
- edge = Edge(edgeid, Vertex(reader.readObject(b), cls.read_string(b)),
- edgelbl, Vertex(reader.readObject(b), cls.read_string(b)))
+ edge = Edge(edgeid, Vertex(r.readObject(b), cls.read_string(b)),
+ edgelbl, Vertex(r.readObject(b), cls.read_string(b)))
b.read(2)
return edge
@@ -450,15 +490,15 @@ class PathIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.path
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray()
ba.extend(writer.writeObject(obj.labels))
ba.extend(writer.writeObject(obj.objects))
- return ba
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, b, reader):
- return Path(reader.readObject(b), reader.readObject(b))
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: Path(r.readObject(b), r.readObject(b)))
class PropertyIO(_GraphBinaryTypeIO):
@@ -467,16 +507,20 @@ class PropertyIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.property
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray()
ba.extend(cls.string_as_bytes(obj.key))
ba.extend(writer.writeObject(obj.value))
ba.extend([DataType.null.value])
- return ba
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, b, reader):
- p = Property(cls.read_string(b), reader.readObject(b), None)
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, cls._read_property)
+
+ @classmethod
+ def _read_property(cls, b, r):
+ p = Property(cls.read_string(b), r.readObject(b), None)
b.read(1)
return p
@@ -487,11 +531,11 @@ class TinkerGraphIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.graph
@classmethod
- def dictify(cls, obj, writer):
+ def dictify(cls, obj, writer, as_value=False):
raise AttributeError("TinkerGraph serialization is not currently supported by gremlin-python")
@classmethod
- def objectify(cls, b, reader):
+ def objectify(cls, b, reader, as_value=False):
raise AttributeError("TinkerGraph deserialization is not currently supported by gremlin-python")
@@ -501,16 +545,20 @@ class VertexIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.vertex
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray()
ba.extend(writer.writeObject(obj.id))
ba.extend(cls.string_as_bytes(obj.label))
ba.extend([DataType.null.value])
- return ba
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+
+ @classmethod
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, cls._read_vertex)
@classmethod
- def objectify(cls, b, reader):
- vertex = Vertex(reader.readObject(b), cls.read_string(b))
+ def _read_vertex(cls, b, r):
+ vertex = Vertex(r.readObject(b), cls.read_string(b))
b.read(1)
return vertex
@@ -521,18 +569,22 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.vertexproperty
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray()
ba.extend(writer.writeObject(obj.id))
ba.extend(cls.string_as_bytes(obj.label))
ba.extend(writer.writeObject(obj.value))
ba.extend([DataType.null.value])
ba.extend([DataType.null.value])
- return ba
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+
+ @classmethod
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, cls._read_vertexproperty)
@classmethod
- def objectify(cls, b, reader):
- vp = VertexProperty(reader.readObject(b), cls.read_string(b), reader.readObject(b), None)
+ def _read_vertexproperty(cls, b, r):
+ vp = VertexProperty(r.readObject(b), cls.read_string(b), r.readObject(b), None)
b.read(1)
b.read(1)
return vp
@@ -541,14 +593,14 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
class _EnumIO(_GraphBinaryTypeIO):
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray()
ba.extend(cls.string_as_bytes(cls.unmangleKeyword(str(obj.name))))
- return ba
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, b, reader):
- return cls.python_type[cls.read_string(b)]
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: cls.python_type[cls.read_string(b)])
class BarrierIO(_EnumIO):
@@ -597,15 +649,15 @@ class BindingIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.binding
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray()
ba.extend(cls.string_as_bytes(obj.key))
ba.extend(writer.writeObject(obj.value))
- return ba
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
@classmethod
- def objectify(cls, b, reader):
- return Binding(cls.read_string(b), reader.readObject(b))
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: Binding(cls.read_string(b), reader.readObject(b)))
class BytecodeIO(_GraphBinaryTypeIO):
@@ -613,8 +665,8 @@ class BytecodeIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.bytecode
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray()
ba.extend(struct.pack(">i", len(obj.step_instructions)))
for inst in obj.step_instructions:
inst_name, inst_args = inst[0], inst[1:] if len(inst) > 1 else []
@@ -630,11 +682,15 @@ class BytecodeIO(_GraphBinaryTypeIO):
ba.extend(struct.pack(">i", len(inst_args)))
for arg in inst_args:
ba.extend(writer.writeObject(arg))
-
- return ba
-
+
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
+
@classmethod
- def objectify(cls, b, reader):
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, cls._read_bytecode)
+
+ @classmethod
+ def _read_bytecode(cls, b, r):
bytecode = Bytecode()
step_count = cls.read_int(b)
@@ -644,7 +700,7 @@ class BytecodeIO(_GraphBinaryTypeIO):
inst_ct = cls.read_int(b)
iy = 0
while iy < inst_ct:
- inst.append(reader.readObject(b))
+ inst.append(r.readObject(b))
iy += 1
bytecode.step_instructions.append(inst)
ix += 1
@@ -656,7 +712,7 @@ class BytecodeIO(_GraphBinaryTypeIO):
inst_ct = cls.read_int(b)
iy = 0
while iy < inst_ct:
- inst.append(reader.readObject(b))
+ inst.append(r.readObject(b))
iy += 1
bytecode.source_instructions.append(inst)
ix += 1
@@ -670,8 +726,8 @@ class LambdaIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.lambda_
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray() if as_value else bytearray([cls.graphbinary_type.value])
lambda_result = obj()
script = lambda_result if isinstance(lambda_result, str) else lambda_result[0]
language = statics.default_lambda_language if isinstance(lambda_result, str) else lambda_result[1]
@@ -697,8 +753,8 @@ class PIO(_GraphBinaryTypeIO):
python_type = P
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray() if as_value else bytearray([cls.graphbinary_type.value])
ba.extend(cls.string_as_bytes(obj.operator))
additional = [writer.writeObject(obj.value), writer.writeObject(obj.other)] \
if obj.other is not None else [writer.writeObject(obj.value)]
@@ -724,12 +780,21 @@ class TraverserIO(_GraphBinaryTypeIO):
python_type = Traverser
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
- ba.extend(struct.pack(">i", obj.bulk))
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray()
+ ba.extend(struct.pack(">q", obj.bulk))
ba.extend(writer.writeObject(obj.object))
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, ba)
- return ba
+ @classmethod
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, cls._read_traverser)
+
+ @classmethod
+ def _read_traverser(cls, b, r):
+ bulk = struct.unpack(">q", b.read(8))[0]
+ obj = r.readObject(b)
+ return Traverser(obj, bulk=bulk)
class ByteIO(_GraphBinaryTypeIO):
@@ -737,14 +802,12 @@ class ByteIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.byte
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
- ba.extend(struct.pack(">b", obj))
- return ba
+ def dictify(cls, obj, writer, as_value=False):
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, struct.pack(">b", obj))
@classmethod
- def objectify(cls, b, reader):
- return int.__new__(SingleByte, struct.unpack_from(">b", b.read(1))[0])
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: int.__new__(SingleByte, struct.unpack_from(">b", b.read(1))[0]))
class ByteBufferIO(_GraphBinaryTypeIO):
@@ -752,11 +815,15 @@ class ByteBufferIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.bytebuffer
@classmethod
- def dictify(cls, obj, writer):
- return cls.as_bytes(cls.graphbinary_type, len(obj), obj)
+ def dictify(cls, obj, writer, as_value=False):
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), len(obj), obj)
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ return cls.is_null(buff, reader, cls._read_bytebuffer)
@classmethod
- def objectify(cls, b, reader):
+ def _read_bytebuffer(cls, b, r):
size = cls.read_int(b)
return ByteBufferType(b.read(size))
@@ -766,14 +833,12 @@ class BooleanIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.boolean
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
- ba.extend(struct.pack(">b", 0x01 if obj else 0x00))
- return ba
+ def dictify(cls, obj, writer, as_value=False):
+ return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, struct.pack(">b", 0x01 if obj else 0x00))
@classmethod
- def objectify(cls, b, reader):
- return True if struct.unpack_from(">b", b.read(1))[0] == 0x01 else False
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, lambda b, r: True if struct.unpack_from(">b", b.read(1))[0] == 0x01 else False)
class TextPIO(_GraphBinaryTypeIO):
@@ -781,8 +846,8 @@ class TextPIO(_GraphBinaryTypeIO):
python_type = TextP
@classmethod
- def dictify(cls, obj, writer):
- ba = bytearray([cls.graphbinary_type.value])
+ def dictify(cls, obj, writer, as_value=False):
+ ba = bytearray() if as_value else bytearray([cls.graphbinary_type.value])
ba.extend(cls.string_as_bytes(obj.operator))
additional = [writer.writeObject(obj.value), writer.writeObject(obj.other)] \
if obj.other is not None else [writer.writeObject(obj.value)]
@@ -798,12 +863,16 @@ class BulkSetIO(_GraphBinaryTypeIO):
graphbinary_type = DataType.bulkset
@classmethod
- def objectify(cls, buff, reader):
- size = cls.read_int(buff)
+ def objectify(cls, buff, reader, as_value=False):
+ return cls.is_null(buff, reader, cls._read_bulkset)
+
+ @classmethod
+ def _read_bulkset(cls, b, r):
+ size = cls.read_int(b)
the_list = []
while size > 0:
- itm = reader.readObject(buff)
- bulk = cls.read_int(buff)
+ itm = r.readObject(b)
+ bulk = cls.read_int(b)
for y in range(bulk):
the_list.append(itm)
size = size - 1
diff --git a/gremlin-python/src/main/jython/tests/conftest.py b/gremlin-python/src/main/jython/tests/conftest.py
index 408c469..fb31c31 100644
--- a/gremlin-python/src/main/jython/tests/conftest.py
+++ b/gremlin-python/src/main/jython/tests/conftest.py
@@ -28,10 +28,13 @@ from gremlin_python.driver.driver_remote_connection import (
DriverRemoteConnection)
from gremlin_python.driver.protocol import GremlinServerWSProtocol
from gremlin_python.driver.serializer import (
- GraphSONMessageSerializer, GraphSONSerializersV2d0,
- GraphSONSerializersV3d0)
+ GraphSONMessageSerializer, GraphSONSerializersV2d0, GraphSONSerializersV3d0,
+ GraphBinaryMessageSerializerV1)
from gremlin_python.driver.tornado.transport import TornadoTransport
+gremlin_server_host = "localhost"
+gremlin_server_url = 'ws://' + gremlin_server_host + ':45940/gremlin'
+
@pytest.fixture
def connection(request):
@@ -41,7 +44,7 @@ def connection(request):
executor = concurrent.futures.ThreadPoolExecutor(5)
pool = queue.Queue()
try:
- conn = Connection('ws://localhost:45941/gremlin', 'gmodern', protocol,
+ conn = Connection(gremlin_server_url, 'gmodern', protocol,
lambda: TornadoTransport(), executor, pool)
except OSError:
executor.shutdown()
@@ -57,7 +60,7 @@ def connection(request):
@pytest.fixture
def client(request):
try:
- client = Client('ws://localhost:45940/gremlin', 'gmodern')
+ client = Client(gremlin_server_url, 'gmodern')
except OSError:
pytest.skip('Gremlin Server is not running')
else:
@@ -70,7 +73,7 @@ def client(request):
@pytest.fixture
def secure_client(request):
try:
- client = Client('ws://localhost:45941/gremlin', 'gmodern', username='stephen', password='password')
+ client = Client('ws://' + gremlin_server_host + ':45941/gremlin', 'gmodern', username='stephen', password='password')
except OSError:
pytest.skip('Gremlin Server is not running')
else:
@@ -80,14 +83,17 @@ def secure_client(request):
return client
-@pytest.fixture(params=['v2', 'v3'])
+@pytest.fixture(params=['graphsonv2', 'graphsonv3', 'graphbinaryv1'])
def remote_connection(request):
try:
- if request.param == 'v2':
- remote_conn = DriverRemoteConnection('ws://localhost:45940/gremlin', 'gmodern',
+ if request.param == 'graphbinaryv1':
+ remote_conn = DriverRemoteConnection(gremlin_server_url, 'gmodern',
+ message_serializer=serializer.GraphBinaryMessageSerializerV1())
+ elif request.param == 'graphsonv2':
+ remote_conn = DriverRemoteConnection(gremlin_server_url, 'gmodern',
message_serializer=serializer.GraphSONSerializersV2d0())
else:
- remote_conn = DriverRemoteConnection('ws://localhost:45940/gremlin', 'gmodern')
+ remote_conn = DriverRemoteConnection(gremlin_server_url, 'gmodern')
except OSError:
pytest.skip('Gremlin Server is not running')
else:
@@ -100,7 +106,8 @@ def remote_connection(request):
@pytest.fixture
def remote_connection_v2(request):
try:
- remote_conn = DriverRemoteConnection('ws://localhost:45940/gremlin', 'g', message_serializer=serializer.GraphSONSerializersV2d0())
+ remote_conn = DriverRemoteConnection(gremlin_server_url, 'g',
+ message_serializer=serializer.GraphSONSerializersV2d0())
except OSError:
pytest.skip('Gremlin Server is not running')
else:
@@ -118,3 +125,8 @@ def graphson_serializer_v2(request):
@pytest.fixture
def graphson_serializer_v3(request):
return GraphSONSerializersV3d0()
+
+
+@pytest.fixture
+def graphbinary_serializer_v1(request):
+ return GraphBinaryMessageSerializerV1()
diff --git a/gremlin-python/src/main/jython/tests/driver/test_serializer.py b/gremlin-python/src/main/jython/tests/driver/test_serializer.py
index f1b4ccb..80c5966 100644
--- a/gremlin-python/src/main/jython/tests/driver/test_serializer.py
+++ b/gremlin-python/src/main/jython/tests/driver/test_serializer.py
@@ -18,20 +18,28 @@ under the License.
'''
from gremlin_python.structure.io import graphsonV2d0
from gremlin_python.structure.io import graphsonV3d0
+from gremlin_python.structure.io import graphbinaryV1
__author__ = 'David M. Brown'
-def test_graphson_serialzier_v2(graphson_serializer_v2):
+def test_graphson_serializer_v2(graphson_serializer_v2):
assert graphson_serializer_v2.version == b"application/vnd.gremlin-v2.0+json"
assert isinstance(graphson_serializer_v2._graphson_reader, graphsonV2d0.GraphSONReader)
- assert isinstance(graphson_serializer_v2.standard._graphson_writer, graphsonV2d0.GraphSONWriter)
- assert isinstance(graphson_serializer_v2.traversal._graphson_writer, graphsonV2d0.GraphSONWriter)
+ assert isinstance(graphson_serializer_v2.standard._writer, graphsonV2d0.GraphSONWriter)
+ assert isinstance(graphson_serializer_v2.traversal._writer, graphsonV2d0.GraphSONWriter)
-def test_graphson_serialzier_v3(graphson_serializer_v3):
+def test_graphson_serializer_v3(graphson_serializer_v3):
assert graphson_serializer_v3.version == b"application/vnd.gremlin-v3.0+json"
assert isinstance(graphson_serializer_v3._graphson_reader, graphsonV3d0.GraphSONReader)
- assert isinstance(graphson_serializer_v3.standard._graphson_writer, graphsonV3d0.GraphSONWriter)
- assert isinstance(graphson_serializer_v3.traversal._graphson_writer, graphsonV3d0.GraphSONWriter)
+ assert isinstance(graphson_serializer_v3.standard._writer, graphsonV3d0.GraphSONWriter)
+ assert isinstance(graphson_serializer_v3.traversal._writer, graphsonV3d0.GraphSONWriter)
+
+
+def test_graphbinary_serializer_v1(graphbinary_serializer_v1):
+ assert graphbinary_serializer_v1.version == b"application/vnd.graphbinary-v1.0"
+ assert isinstance(graphbinary_serializer_v1._graphbinary_reader, graphbinaryV1.GraphBinaryReader)
+ assert isinstance(graphbinary_serializer_v1.standard._writer, graphbinaryV1.GraphBinaryWriter)
+ assert isinstance(graphbinary_serializer_v1.traversal._writer, graphbinaryV1.GraphBinaryWriter)
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_functionalityio.py b/gremlin-python/src/main/jython/tests/structure/io/test_functionalityio.py
new file mode 100644
index 0000000..54dd35e
--- /dev/null
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_functionalityio.py
@@ -0,0 +1,64 @@
+'''
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+'''
+
+import datetime
+import uuid
+
+from gremlin_python.structure.graph import Graph
+from gremlin_python.statics import *
+
+
+def test_timestamp(remote_connection):
+ g = Graph().traversal().withRemote(remote_connection)
+ ts = timestamp(1481750076295 / 1000)
+ resp = g.addV('test_vertex').property('ts', ts)
+ resp = resp.toList()
+ vid = resp[0].id
+ try:
+ ts_prop = g.V(vid).properties('ts').toList()[0]
+ assert isinstance(ts_prop.value, timestamp)
+ assert ts_prop.value == ts
+ finally:
+ g.V(vid).drop().iterate()
+
+
+def test_datetime(remote_connection):
+ g = Graph().traversal().withRemote(remote_connection)
+ dt = datetime.datetime.utcfromtimestamp(1481750076295 / 1000)
+ resp = g.addV('test_vertex').property('dt', dt).toList()
+ vid = resp[0].id
+ try:
+ dt_prop = g.V(vid).properties('dt').toList()[0]
+ assert isinstance(dt_prop.value, datetime.datetime)
+ assert dt_prop.value == dt
+ finally:
+ g.V(vid).drop().iterate()
+
+
+def test_uuid(remote_connection):
+ g = Graph().traversal().withRemote(remote_connection)
+ uid = uuid.UUID("41d2e28a-20a4-4ab0-b379-d810dede3786")
+ resp = g.addV('test_vertex').property('uuid', uid).toList()
+ vid = resp[0].id
+ try:
+ uid_prop = g.V(vid).properties('uuid').toList()[0]
+ assert isinstance(uid_prop.value, uuid.UUID)
+ assert uid_prop.value == uid
+ finally:
+ g.V(vid).drop().iterate()
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 620c3fc..bcaad3d 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -77,7 +77,7 @@ class TestGraphSONWriter(object):
assert x == output
def test_date(self):
- x = calendar.timegm(datetime.datetime(2016, 12, 14, 16, 14, 36, 295000).utctimetuple())
+ x = datetime.datetime(2016, 12, 14, 16, 14, 36, 295000)
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphsonV3d0.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphsonV3d0.py
index 7cc8d68..17bb177 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphsonV3d0.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphsonV3d0.py
@@ -88,7 +88,6 @@ class TestGraphSONReader(object):
assert x.count("marko") == 1
assert x.count("josh") == 3
-
def test_number_input(self):
x = self.graphson_reader.readObject(json.dumps({
"@type": "gx:Byte",
@@ -535,44 +534,3 @@ class TestGraphSONWriter(object):
c = str.__new__(SingleChar, chr(76))
output = self.graphson_writer.writeObject(c)
assert expected == output
-
-
-class TestFunctionalGraphSONIO(object):
- """Functional IO tests"""
-
- def test_timestamp(self, remote_connection):
- g = Graph().traversal().withRemote(remote_connection)
- ts = timestamp(1481750076295 / 1000)
- resp = g.addV('test_vertex').property('ts', ts)
- resp = resp.toList()
- vid = resp[0].id
- try:
- ts_prop = g.V(vid).properties('ts').toList()[0]
- assert isinstance(ts_prop.value, timestamp)
- assert ts_prop.value == ts
- finally:
- g.V(vid).drop().iterate()
-
- def test_datetime(self, remote_connection):
- g = Graph().traversal().withRemote(remote_connection)
- dt = datetime.datetime.utcfromtimestamp(1481750076295 / 1000)
- resp = g.addV('test_vertex').property('dt', dt).toList()
- vid = resp[0].id
- try:
- dt_prop = g.V(vid).properties('dt').toList()[0]
- assert isinstance(dt_prop.value, datetime.datetime)
- assert dt_prop.value == dt
- finally:
- g.V(vid).drop().iterate()
-
- def test_uuid(self, remote_connection):
- g = Graph().traversal().withRemote(remote_connection)
- uid = uuid.UUID("41d2e28a-20a4-4ab0-b379-d810dede3786")
- resp = g.addV('test_vertex').property('uuid', uid).toList()
- vid = resp[0].id
- try:
- uid_prop = g.V(vid).properties('uuid').toList()[0]
- assert isinstance(uid_prop.value, uuid.UUID)
- assert uid_prop.value == uid
- finally:
- g.V(vid).drop().iterate()
[tinkerpop] 18/33: Added boolean and textp to python graphbinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit e2cf80fc51875652467e7b2565d0ce527216d52c
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Aug 2 07:03:40 2019 -0400
Added boolean and textp to python graphbinary
---
.../gremlin_python/structure/io/graphbinaryV1.py | 39 ++++++++++++++++++++++
.../tests/structure/io/test_graphbinaryV1.py | 9 +++++
2 files changed, 48 insertions(+)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 208a1e3..2f64ba2 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -90,6 +90,15 @@ class DataType(Enum):
biginteger = 0x23 #todo
byte = 0x24
bytebuffer = 0x25
+ short = 0x26 #todo?
+ boolean = 0x27
+ textp = 0x28
+ traversalstrategy = 0x29
+ bulkset = 0x2a
+ tree = 0x2b
+ metrics = 0x2c
+ traversalmetrics = 0x2d
+ custom = 0x00
class GraphBinaryTypeType(type):
@@ -752,3 +761,33 @@ class ByteBufferIO(_GraphBinaryTypeIO):
return ByteBufferType(b.read(size))
+class BooleanIO(_GraphBinaryTypeIO):
+ python_type = bool
+ graphbinary_type = DataType.boolean
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(struct.pack(">b", 0x01 if obj else 0x00))
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ return True if struct.unpack_from(">b", b.read(1))[0] == 0x01 else False
+
+
+class TextPIO(_GraphBinaryTypeIO):
+ graphbinary_type = DataType.textp
+ python_type = TextP
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(cls.string_as_bytes(obj.operator))
+ additional = [writer.writeObject(obj.value), writer.writeObject(obj.other)] \
+ if obj.other is not None else [writer.writeObject(obj.value)]
+ ba.extend(struct.pack(">i", len(additional)))
+ for a in additional:
+ ba.extend(a)
+
+ return ba
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 4c827ad..620c3fc 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -178,4 +178,13 @@ class TestGraphSONWriter(object):
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+ def test_boolean(self):
+ x = True
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ x = False
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
[tinkerpop] 13/33: Add enum graphbinary support in python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit bbc9bfa34e5e6e7e1b546b364ed05624437aa2cd
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Thu Aug 1 10:22:47 2019 -0400
Add enum graphbinary support in python
---
.../gremlin_python/structure/io/graphbinaryV1.py | 82 ++++++++++++++++++++--
.../tests/structure/io/test_graphbinaryV1.py | 7 +-
2 files changed, 82 insertions(+), 7 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 551db60..7a91edf 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -35,8 +35,10 @@ from aenum import Enum
from isodate import parse_duration, duration_isoformat
from gremlin_python import statics
-from gremlin_python.statics import FloatType, FunctionType, IntType, LongType, TypeType, DictType, ListType, SetType, SingleByte, ByteBufferType, SingleChar
-from gremlin_python.process.traversal import Binding, Bytecode, P, TextP, Traversal, Traverser, TraversalStrategy, T
+from gremlin_python.statics import FloatType, FunctionType, IntType, LongType, TypeType, DictType, ListType, SetType, \
+ SingleByte, ByteBufferType, SingleChar
+from gremlin_python.process.traversal import Barrier, Binding, Bytecode, Cardinality, Column, Direction, Operator, \
+ Order, Pick, Pop, P, TextP, Traversal, Traverser, TraversalStrategy, T
from gremlin_python.structure.graph import Graph, Edge, Property, Vertex, VertexProperty, Path
log = logging.getLogger(__name__)
@@ -65,9 +67,19 @@ class DataType(Enum):
edge = 0x0d
path = 0x0e
property = 0x0f
- tinkergraph = 0x10
+ graph = 0x10
vertex = 0x11
vertexproperty = 0x12
+ barrier = 0x13
+ binding = 0x14 # todo
+ bytecode = 0x15 # todo
+ cardinality = 0x16
+ column = 0x17
+ direction = 0x18
+ operator = 0x19
+ order = 0x1a
+ pick = 0x1b
+ pop = 0x1c
class GraphBinaryTypeType(type):
@@ -133,7 +145,8 @@ class _GraphBinaryTypeIO(object):
symbolMap = {"global_": "global", "as_": "as", "in_": "in", "and_": "and",
"or_": "or", "is_": "is", "not_": "not", "from_": "from",
- "set_": "set", "list_": "list", "all_": "all", "with_": "with"}
+ "set_": "set", "list_": "list", "all_": "all", "with_": "with",
+ "filter_": "filter", "id_": "id", "max_": "max", "min_": "min", "sum_": "sum"}
@classmethod
def as_bytes(cls, graphbin_type=None, size=None, *args):
@@ -452,7 +465,7 @@ class PropertyIO(_GraphBinaryTypeIO):
class TinkerGraphIO(_GraphBinaryTypeIO):
python_type = Graph
- graphbinary_type = DataType.tinkergraph
+ graphbinary_type = DataType.graph
@classmethod
def dictify(cls, obj, writer):
@@ -503,4 +516,61 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
vp = VertexProperty(reader.readObject(b), cls.read_string(b), reader.readObject(b), None)
b.read(1)
b.read(1)
- return vp
\ No newline at end of file
+ return vp
+
+
+class _EnumIO(_GraphBinaryTypeIO):
+
+ python_type = Enum
+ base_enum = None
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(cls.string_as_bytes(cls.unmangleKeyword(str(obj.name))))
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ return cls.base_enum[cls.read_string(b)]
+
+
+class BarrierIO(_EnumIO):
+ graphbinary_type = DataType.barrier
+ base_enum = Barrier
+
+
+class CardinalityIO(_EnumIO):
+ graphbinary_type = DataType.cardinality
+ base_enum = Cardinality
+
+
+class ColumnIO(_EnumIO):
+ graphbinary_type = DataType.column
+ base_enum = Column
+
+
+class DirectionIO(_EnumIO):
+ graphbinary_type = DataType.direction
+ base_enum = Direction
+
+
+class OperatorIO(_EnumIO):
+ graphbinary_type = DataType.operator
+ base_enum = Operator
+
+
+class OrderIO(_EnumIO):
+ graphbinary_type = DataType.order
+ base_enum = Order
+
+
+class PickIO(_EnumIO):
+ graphbinary_type = DataType.pick
+ base_enum = Pick
+
+
+class PopIO(_EnumIO):
+ graphbinary_type = DataType.pop
+ base_enum = Pop
+
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index cdf52fc..10c77c1 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -31,7 +31,7 @@ import six
from gremlin_python.statics import timestamp, long
from gremlin_python.structure.graph import Vertex, Edge, Property, VertexProperty, Graph, Path
from gremlin_python.structure.io.graphbinaryV1 import GraphBinaryWriter, GraphBinaryReader, DataType
-from gremlin_python.process.traversal import P
+from gremlin_python.process.traversal import P, Barrier
from gremlin_python.process.strategies import SubgraphStrategy
from gremlin_python.process.graph_traversal import __
@@ -149,3 +149,8 @@ class TestGraphSONWriter(object):
x = VertexProperty(123, "name", "stephen", None)
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+
+ def test_barrier(self):
+ x = Barrier.normSack
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
[tinkerpop] 11/33: Added property/path to python graphbinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 2623402501976f2798fc4b9388ffef0de879c1b6
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Wed Jul 31 13:55:52 2019 -0400
Added property/path to python graphbinary
---
.../gremlin_python/structure/io/graphbinaryV1.py | 44 +++++++++++++++++++++-
.../tests/structure/io/test_graphbinaryV1.py | 10 +++++
2 files changed, 52 insertions(+), 2 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 76dfaee..c12ffca 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -49,6 +49,7 @@ _deserializers = {}
class DataType(Enum):
+ null = 0xfe
int = 0x01
long = 0x02
string = 0x03
@@ -62,6 +63,8 @@ class DataType(Enum):
set = 0x0b
uuid = 0x0c
edge = 0x0d
+ path = 0x0e
+ property = 0x0f
class GraphBinaryTypeType(type):
@@ -392,8 +395,8 @@ class EdgeIO(_GraphBinaryTypeIO):
ba.extend(cls.string_as_bytes(obj.inV.label))
ba.extend(writer.writeObject(obj.outV.id))
ba.extend(cls.string_as_bytes(obj.outV.label))
- ba.extend([0xfe])
- ba.extend([0xfe])
+ ba.extend([DataType.null.value])
+ ba.extend([DataType.null.value])
return ba
@classmethod
@@ -404,3 +407,40 @@ class EdgeIO(_GraphBinaryTypeIO):
edgelbl, Vertex(reader.readObject(b), cls.read_string(b)))
b.read(2)
return edge
+
+
+class PathIO(_GraphBinaryTypeIO):
+
+ python_type = Path
+ graphbinary_type = DataType.path
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(writer.writeObject(obj.labels))
+ ba.extend(writer.writeObject(obj.objects))
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ return Path(reader.readObject(b), reader.readObject(b))
+
+
+class PropertyIO(_GraphBinaryTypeIO):
+
+ python_type = Property
+ graphbinary_type = DataType.property
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(cls.string_as_bytes(obj.key))
+ ba.extend(writer.writeObject(obj.value))
+ ba.extend([DataType.null.value])
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ p = Property(cls.read_string(b), reader.readObject(b), None)
+ b.read(1)
+ return p
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index f287c01..3c4ef34 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -129,3 +129,13 @@ class TestGraphSONWriter(object):
x = Edge(123, Vertex(1, 'person'), "developed", Vertex(10, "software"))
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+
+ def test_path(self):
+ x = Path(["x", "y", "z"], [1, 2, 3])
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ def test_property(self):
+ x = Property("name", "stephen", None)
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
[tinkerpop] 01/33: Initial setup for GraphBinary in Python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 391efe0678ceb5e856f68aa15d0d1818bf65b6b0
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Mon Jul 22 09:35:05 2019 -0400
Initial setup for GraphBinary in Python
---
.../gremlin_python/structure/io/graphbinaryV1.py | 151 +++++++++++++++++++++
.../tests/structure/io/test_graphbinaryV1.py | 49 +++++++
2 files changed, 200 insertions(+)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
new file mode 100644
index 0000000..4eff156
--- /dev/null
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -0,0 +1,151 @@
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+"""
+import datetime
+import struct
+import time
+import uuid
+import math
+import base64
+import numbers
+from collections import OrderedDict
+from decimal import *
+import logging
+from datetime import timedelta
+
+import six
+from aenum import Enum
+from isodate import parse_duration, duration_isoformat
+
+from gremlin_python import statics
+from gremlin_python.statics import FloatType, FunctionType, IntType, LongType, TypeType, DictType, ListType, SetType, SingleByte, ByteBufferType, SingleChar
+from gremlin_python.process.traversal import Binding, Bytecode, P, TextP, Traversal, Traverser, TraversalStrategy, T
+from gremlin_python.structure.graph import Edge, Property, Vertex, VertexProperty, Path
+
+log = logging.getLogger(__name__)
+
+# When we fall back to a superclass's serializer, we iterate over this map.
+# We want that iteration order to be consistent, so we use an OrderedDict,
+# not a dict.
+_serializers = OrderedDict()
+_deserializers = {}
+
+
+class DataType(Enum):
+ string = 0x03
+
+
+class GraphBinaryTypeType(type):
+ def __new__(mcs, name, bases, dct):
+ cls = super(GraphBinaryTypeType, mcs).__new__(mcs, name, bases, dct)
+ if not name.startswith('_'):
+ if cls.python_type:
+ _serializers[cls.python_type] = cls
+ if cls.graphbinary_type:
+ _deserializers[cls.graphbinary_type] = cls
+ return cls
+
+
+class GraphBinaryWriter(object):
+ def __init__(self, serializer_map=None):
+ self.serializers = _serializers.copy()
+ if serializer_map:
+ self.serializers.update(serializer_map)
+
+ def writeObject(self, objectData):
+ return self.toDict(objectData)
+
+ def toDict(self, obj):
+ try:
+ return self.serializers[type(obj)].dictify(obj, self)
+ except KeyError:
+ for key, serializer in self.serializers.items():
+ if isinstance(obj, key):
+ return serializer.dictify(obj, self)
+
+ if isinstance(obj, dict):
+ return dict((self.toDict(k), self.toDict(v)) for k, v in obj.items())
+ elif isinstance(obj, set):
+ return set([self.toDict(o) for o in obj])
+ elif isinstance(obj, list):
+ return [self.toDict(o) for o in obj]
+ else:
+ return obj
+
+
+class GraphBinaryReader(object):
+ def __init__(self, deserializer_map=None):
+ self.deserializers = _deserializers.copy()
+ if deserializer_map:
+ self.deserializers.update(deserializer_map)
+
+ def readObject(self, b):
+ return self.toObject(b)
+
+ def toObject(self, obj):
+ binary_type, rest = DataType(obj[0]), obj[1:]
+ return self.deserializers[binary_type].objectify(rest, self)
+
+
+@six.add_metaclass(GraphBinaryTypeType)
+class _GraphBinaryTypeIO(object):
+ python_type = None
+ graphbinary_type = None
+
+ symbolMap = {"global_": "global", "as_": "as", "in_": "in", "and_": "and",
+ "or_": "or", "is_": "is", "not_": "not", "from_": "from",
+ "set_": "set", "list_": "list", "all_": "all", "with_": "with"}
+
+ @classmethod
+ def as_bytes(cls, graphbin_type, *args):
+ ba = bytearray([graphbin_type.value])
+ for arg in args:
+ if isinstance(arg, str):
+ ba.extend(arg.encode("utf-8"))
+ elif isinstance(arg, numbers.Number):
+ ba.extend(bytes([arg]))
+ elif isinstance(arg, (bytes, bytearray)):
+ ba.extend(arg)
+ return ba
+
+ @classmethod
+ def unmangleKeyword(cls, symbol):
+ return cls.symbolMap.get(symbol, symbol)
+
+ def dictify(self, obj, writer):
+ raise NotImplementedError()
+
+ def objectify(self, d, reader):
+ raise NotImplementedError()
+
+
+class StringIO(_GraphBinaryTypeIO):
+
+ python_type = str
+ graphbinary_type = DataType.string
+
+ @classmethod
+ def dictify(cls, n, writer):
+ return cls.as_bytes(cls.graphbinary_type, len(n), n)
+
+ @classmethod
+ def objectify(cls, b, reader):
+ len_block, rest = b[0:4], b[4:]
+ size = struct.unpack("<L", len_block)[0]
+ string_block, b = rest[0:size], rest[size:]
+ return string_block
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
new file mode 100644
index 0000000..9eb82cf
--- /dev/null
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -0,0 +1,49 @@
+"""
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+"""
+
+import datetime
+import time
+import uuid
+import math
+from decimal import *
+
+from mock import Mock
+
+import six
+
+from gremlin_python.statics import *
+from gremlin_python.structure.graph import Vertex, Edge, Property, VertexProperty, Graph, Path
+from gremlin_python.structure.io.graphbinaryV1 import GraphBinaryWriter, GraphBinaryReader, DataType
+from gremlin_python.process.traversal import P
+from gremlin_python.process.strategies import SubgraphStrategy
+from gremlin_python.process.graph_traversal import __
+
+
+class TestGraphBinaryReader(object):
+ graphbinary_reader = GraphBinaryReader()
+
+
+class TestGraphSONWriter(object):
+ graphbinary_writer = GraphBinaryWriter()
+ graphbinary_reader = GraphBinaryReader()
+
+ def test_string(self):
+ x = "serialize this!"
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
[tinkerpop] 10/33: Added uuid/edge to python graphbinary derser
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 9cd7f1847dc932007d2c3ebe619e8afae3943dbb
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Wed Jul 31 13:19:05 2019 -0400
Added uuid/edge to python graphbinary derser
---
.../gremlin_python/structure/io/graphbinaryV1.py | 56 +++++++++++++++++++++-
.../tests/structure/io/test_graphbinaryV1.py | 10 ++++
2 files changed, 65 insertions(+), 1 deletion(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 3d7d7da..76dfaee 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -60,6 +60,8 @@ class DataType(Enum):
list = 0x09
map = 0x0a
set = 0x0b
+ uuid = 0x0c
+ edge = 0x0d
class GraphBinaryTypeType(type):
@@ -141,10 +143,18 @@ class _GraphBinaryTypeIO(object):
return ba
@classmethod
+ def string_as_bytes(cls, s):
+ return cls.as_bytes(None, len(s), s.encode("utf-8"))
+
+ @classmethod
def read_int(cls, buff):
return struct.unpack(">i", buff.read(4))[0]
@classmethod
+ def read_string(cls, buff):
+ return buff.read(cls.read_int(buff)).decode("utf-8")
+
+ @classmethod
def unmangleKeyword(cls, symbol):
return cls.symbolMap.get(symbol, symbol)
@@ -288,7 +298,7 @@ class StringIO(_GraphBinaryTypeIO):
@classmethod
def objectify(cls, b, reader):
- return b.read(cls.read_int(b)).decode("utf-8")
+ return cls.read_string(b)
class ListIO(_GraphBinaryTypeIO):
@@ -350,3 +360,47 @@ class MapIO(_GraphBinaryTypeIO):
size = size - 1
return the_dict
+
+
+class UuidIO(_GraphBinaryTypeIO):
+
+ python_type = uuid.UUID
+ graphbinary_type = DataType.uuid
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(obj.bytes)
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ return uuid.UUID(bytes=b.read(16))
+
+
+class EdgeIO(_GraphBinaryTypeIO):
+
+ python_type = Edge
+ graphbinary_type = DataType.edge
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(writer.writeObject(obj.id))
+ ba.extend(cls.string_as_bytes(obj.label))
+ ba.extend(writer.writeObject(obj.inV.id))
+ ba.extend(cls.string_as_bytes(obj.inV.label))
+ ba.extend(writer.writeObject(obj.outV.id))
+ ba.extend(cls.string_as_bytes(obj.outV.label))
+ ba.extend([0xfe])
+ ba.extend([0xfe])
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ edgeid = reader.readObject(b)
+ edgelbl = cls.read_string(b)
+ edge = Edge(edgeid, Vertex(reader.readObject(b), cls.read_string(b)),
+ edgelbl, Vertex(reader.readObject(b), cls.read_string(b)))
+ b.read(2)
+ return edge
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 54acb55..f287c01 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -119,3 +119,13 @@ class TestGraphSONWriter(object):
987: ["go", "deep", {"here": "!"}]}
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+
+ def test_uuid(self):
+ x = uuid.UUID("41d2e28a-20a4-4ab0-b379-d810dede3786")
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ def test_edge(self):
+ x = Edge(123, Vertex(1, 'person'), "developed", Vertex(10, "software"))
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
[tinkerpop] 07/33: Fixed the graphbinary g;
Date serialization - wasn't account for timezone stuff properly
(still)
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 04b091b2d23d5cecfef2d864d805ab9044b76fb4
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Jul 26 11:43:47 2019 -0400
Fixed the graphbinary g;Date serialization - wasn't account for timezone stuff properly (still)
---
.../jython/gremlin_python/structure/io/graphbinaryV1.py | 17 ++++++++---------
.../jython/tests/structure/io/test_graphbinaryV1.py | 3 ++-
2 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 33342de..f84d2a0 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -17,6 +17,7 @@ specific language governing permissions and limitations
under the License.
"""
import datetime
+import calendar
import struct
import time
import uuid
@@ -185,15 +186,13 @@ class DateIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer):
- if six.PY3:
- pts = (obj - datetime.datetime(1970, 1, 1)) / datetime.timedelta(seconds=1)
- else:
- # Hack for legacy Python - timestamp() in Python 3.3
- pts = (time.mktime(obj.timetuple()) + obj.microsecond / 1e6) - \
- (time.mktime(datetime.datetime(1970, 1, 1).timetuple()))
-
- # Java timestamp expects milliseconds - have to use int because of legacy Python
- ts = int(round(pts * 1000))
+ try:
+ timestamp_seconds = calendar.timegm(obj.utctimetuple())
+ pts = timestamp_seconds * 1e3 + getattr(obj, 'microsecond', 0) / 1e3
+ except AttributeError:
+ pts = calendar.timegm(obj.timetuple()) * 1e3
+
+ ts = int(round(pts * 100))
return cls.as_bytes(cls.graphbinary_type, None, struct.pack(">q", ts))
@classmethod
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 5775f3a..db58545 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -18,6 +18,7 @@ under the License.
"""
import datetime
+import calendar
import time
import uuid
import math
@@ -54,7 +55,7 @@ class TestGraphSONWriter(object):
assert x == output
def test_date(self):
- x = datetime.datetime(2016, 12, 14, 16, 14, 36, 295000)
+ x = calendar.timegm(datetime.datetime(2016, 12, 14, 16, 14, 36, 295000).utctimetuple())
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
[tinkerpop] 14/33: Aded Binding and fixed up enum derser in python
graphbinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit b0fa0fb181f85637156f58d4f7ed8cb19f960675
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Thu Aug 1 10:36:45 2019 -0400
Aded Binding and fixed up enum derser in python graphbinary
---
.../gremlin_python/structure/io/graphbinaryV1.py | 40 +++++++++++++++-------
.../tests/structure/io/test_graphbinaryV1.py | 7 +++-
2 files changed, 33 insertions(+), 14 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 7a91edf..8eb323b 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -71,7 +71,7 @@ class DataType(Enum):
vertex = 0x11
vertexproperty = 0x12
barrier = 0x13
- binding = 0x14 # todo
+ binding = 0x14
bytecode = 0x15 # todo
cardinality = 0x16
column = 0x17
@@ -521,9 +521,6 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
class _EnumIO(_GraphBinaryTypeIO):
- python_type = Enum
- base_enum = None
-
@classmethod
def dictify(cls, obj, writer):
ba = bytearray([cls.graphbinary_type.value])
@@ -532,45 +529,62 @@ class _EnumIO(_GraphBinaryTypeIO):
@classmethod
def objectify(cls, b, reader):
- return cls.base_enum[cls.read_string(b)]
+ return cls.python_type[cls.read_string(b)]
class BarrierIO(_EnumIO):
graphbinary_type = DataType.barrier
- base_enum = Barrier
+ python_type = Barrier
class CardinalityIO(_EnumIO):
graphbinary_type = DataType.cardinality
- base_enum = Cardinality
+ python_type = Cardinality
class ColumnIO(_EnumIO):
graphbinary_type = DataType.column
- base_enum = Column
+ python_type = Column
class DirectionIO(_EnumIO):
graphbinary_type = DataType.direction
- base_enum = Direction
+ python_type = Direction
class OperatorIO(_EnumIO):
graphbinary_type = DataType.operator
- base_enum = Operator
+ python_type = Operator
class OrderIO(_EnumIO):
graphbinary_type = DataType.order
- base_enum = Order
+ python_type = Order
class PickIO(_EnumIO):
graphbinary_type = DataType.pick
- base_enum = Pick
+ python_type = Pick
class PopIO(_EnumIO):
graphbinary_type = DataType.pop
- base_enum = Pop
+ python_type = Pop
+
+
+class BindingIO(_GraphBinaryTypeIO):
+
+ python_type = Binding
+ graphbinary_type = DataType.binding
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(cls.string_as_bytes(obj.key))
+ ba.extend(writer.writeObject(obj.value))
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ return Binding(cls.read_string(b), reader.readObject(b))
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 10c77c1..f3ee2c8 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -31,7 +31,7 @@ import six
from gremlin_python.statics import timestamp, long
from gremlin_python.structure.graph import Vertex, Edge, Property, VertexProperty, Graph, Path
from gremlin_python.structure.io.graphbinaryV1 import GraphBinaryWriter, GraphBinaryReader, DataType
-from gremlin_python.process.traversal import P, Barrier
+from gremlin_python.process.traversal import P, Barrier, Binding
from gremlin_python.process.strategies import SubgraphStrategy
from gremlin_python.process.graph_traversal import __
@@ -154,3 +154,8 @@ class TestGraphSONWriter(object):
x = Barrier.normSack
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+
+ def test_binding(self):
+ x = Binding("name", "marko")
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
[tinkerpop] 08/33: Added double and float graphbinary serialization
in python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 7c01e7a44dce57b84de0be487ed9b7577085c084
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Jul 26 12:37:58 2019 -0400
Added double and float graphbinary serialization in python
---
.../GraphBinaryReaderWriterRoundTripTest.java | 3 ++
.../gremlin_python/structure/io/graphbinaryV1.py | 61 +++++++++++++++++++++-
.../tests/structure/io/test_graphbinaryV1.py | 22 ++++++++
3 files changed, 84 insertions(+), 2 deletions(-)
diff --git a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/binary/GraphBinaryReaderWriterRoundTripTest.java b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/binary/GraphBinaryReaderWriterRoundTripTest.java
index d278602..48e324c 100644
--- a/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/binary/GraphBinaryReaderWriterRoundTripTest.java
+++ b/gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/binary/GraphBinaryReaderWriterRoundTripTest.java
@@ -165,6 +165,9 @@ public class GraphBinaryReaderWriterRoundTripTest {
new Object[] {"Integer", 1, null},
new Object[] {"Float", 2f, null},
new Object[] {"Double", 3.1d, null},
+ new Object[] {"Double", Double.NaN, null},
+ new Object[] {"Double", Double.POSITIVE_INFINITY, null},
+ new Object[] {"Double", Double.NEGATIVE_INFINITY, null},
new Object[] {"Long", 10122L, null},
new Object[] {"IntegerZero", 0, null},
new Object[] {"FloatZero", 0f, null},
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index f84d2a0..4a5dd0e 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -54,6 +54,9 @@ class DataType(Enum):
string = 0x03
date = 0x04
timestamp = 0x05
+ clazz = 0x06
+ double = 0x07
+ float = 0x08
list = 0x09
@@ -123,8 +126,8 @@ class _GraphBinaryTypeIO(object):
"set_": "set", "list_": "list", "all_": "all", "with_": "with"}
@classmethod
- def as_bytes(cls, graphbin_type, size=None, *args):
- ba = bytearray([graphbin_type.value])
+ def as_bytes(cls, graphbin_type=None, size=None, *args):
+ ba = bytearray() if graphbin_type is None else bytearray([graphbin_type.value])
if size is not None:
ba.extend(struct.pack(">i", size))
@@ -218,6 +221,60 @@ class TimestampIO(_GraphBinaryTypeIO):
return statics.timestamp(struct.unpack(">q", buff.read(8))[0] / 1000.0)
+def _long_bits_to_double(bits):
+ return struct.unpack('d', struct.pack('Q', bits))[0]
+
+
+NAN = _long_bits_to_double(0x7ff8000000000000)
+POSITIVE_INFINITY = _long_bits_to_double(0x7ff0000000000000)
+NEGATIVE_INFINITY = _long_bits_to_double(0xFff0000000000000)
+
+
+class FloatIO(LongIO):
+
+ python_type = FloatType
+ graphbinary_type = DataType.float
+ graphbinary_base_type = DataType.float
+ byte_format = ">f"
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ if math.isnan(obj):
+ return cls.as_bytes(cls.graphbinary_base_type, None, struct.pack(cls.byte_format, NAN))
+ elif math.isinf(obj) and obj > 0:
+ return cls.as_bytes(cls.graphbinary_base_type, None, struct.pack(cls.byte_format, POSITIVE_INFINITY))
+ elif math.isinf(obj) and obj < 0:
+ return cls.as_bytes(cls.graphbinary_base_type, None, struct.pack(cls.byte_format, NEGATIVE_INFINITY))
+ else:
+ return cls.as_bytes(cls.graphbinary_base_type, None, struct.pack(cls.byte_format, obj))
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ return struct.unpack(cls.byte_format, buff.read(4))[0]
+
+
+class DoubleIO(FloatIO):
+ """
+ Floats basically just fall through to double serialization.
+ """
+
+ graphbinary_type = DataType.double
+ graphbinary_base_type = DataType.double
+ byte_format = ">d"
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ return struct.unpack(cls.byte_format, buff.read(8))[0]
+
+
+class TypeSerializer(_GraphBinaryTypeIO):
+ python_type = TypeType
+
+ @classmethod
+ def dictify(cls, typ, writer):
+ return writer.toDict(typ())
+
+
class StringIO(_GraphBinaryTypeIO):
python_type = str
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index db58545..75b4392 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -54,6 +54,28 @@ class TestGraphSONWriter(object):
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+ def test_float(self):
+ x = float(100.001)
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ x = float('nan')
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert math.isnan(output)
+
+ x = float('-inf')
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert math.isinf(output) and output < 0
+
+ x = float('inf')
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert math.isinf(output) and output > 0
+
+ def test_double(self):
+ x = 100.001
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
def test_date(self):
x = calendar.timegm(datetime.datetime(2016, 12, 14, 16, 14, 36, 295000).utctimetuple())
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
[tinkerpop] 24/33: Added fqcn to TraversalStrategy in python to
allow GraphBinary serialization to work properly
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 602edf8029a56430017c7a9047b3f4a741bbae9b
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Mon Aug 12 18:31:52 2019 -0400
Added fqcn to TraversalStrategy in python to allow GraphBinary serialization to work properly
---
gremlin-python/glv/TraversalSource.template | 3 ++-
gremlin-python/src/main/jython/gremlin_python/process/traversal.py | 2 +-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/gremlin-python/glv/TraversalSource.template b/gremlin-python/glv/TraversalSource.template
index 8f776d2..2b021ac 100644
--- a/gremlin-python/glv/TraversalSource.template
+++ b/gremlin-python/glv/TraversalSource.template
@@ -297,7 +297,8 @@ class TraversalStrategies(object):
class TraversalStrategy(object):
- def __init__(self, strategy_name=None, configuration=None):
+ def __init__(self, strategy_name=None, configuration=None, fqcn=None):
+ self.fqcn = fqcn
self.strategy_name = type(self).__name__ if strategy_name is None else strategy_name
self.configuration = {} if configuration is None else configuration
diff --git a/gremlin-python/src/main/jython/gremlin_python/process/traversal.py b/gremlin-python/src/main/jython/gremlin_python/process/traversal.py
index 5a73702..7253df6 100644
--- a/gremlin-python/src/main/jython/gremlin_python/process/traversal.py
+++ b/gremlin-python/src/main/jython/gremlin_python/process/traversal.py
@@ -582,8 +582,8 @@ class TraversalStrategies(object):
class TraversalStrategy(object):
def __init__(self, strategy_name=None, configuration=None, fqcn=None):
- self.strategy_name = type(self).__name__ if strategy_name is None else strategy_name
self.fqcn = fqcn
+ self.strategy_name = type(self).__name__ if strategy_name is None else strategy_name
self.configuration = {} if configuration is None else configuration
def apply(self, traversal):
[tinkerpop] 03/33: Added int graphbinary serialization for python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 255c7c22c945e1492fdc6b4e78375cb6508da511
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Mon Jul 22 13:51:55 2019 -0400
Added int graphbinary serialization for python
Included tests for heterogenous lists now that we have two primitives that work.
---
.../gremlin_python/structure/io/graphbinaryV1.py | 25 ++++++++++++++++++----
.../tests/structure/io/test_graphbinaryV1.py | 10 +++++++++
2 files changed, 31 insertions(+), 4 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 5e06da4..9c9ab30 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -48,6 +48,7 @@ _deserializers = {}
class DataType(Enum):
+ int = 0x01
string = 0x03
list = 0x09
@@ -131,6 +132,10 @@ class _GraphBinaryTypeIO(object):
return ba
@classmethod
+ def read_int(cls, buff):
+ return struct.unpack(">i", buff.read(4))[0]
+
+ @classmethod
def unmangleKeyword(cls, symbol):
return cls.symbolMap.get(symbol, symbol)
@@ -141,6 +146,20 @@ class _GraphBinaryTypeIO(object):
raise NotImplementedError()
+class IntIO(_GraphBinaryTypeIO):
+
+ python_type = IntType
+ graphbinary_type = DataType.int
+
+ @classmethod
+ def dictify(cls, n, writer):
+ return cls.as_bytes(cls.graphbinary_type, None, struct.pack(">i", n))
+
+ @classmethod
+ def objectify(cls, b, reader):
+ return cls.read_int(b)
+
+
class StringIO(_GraphBinaryTypeIO):
python_type = str
@@ -152,8 +171,7 @@ class StringIO(_GraphBinaryTypeIO):
@classmethod
def objectify(cls, b, reader):
- size = struct.unpack(">i", b.read(4))[0]
- return b.read(size)
+ return b.read(cls.read_int(b))
class ListIO(_GraphBinaryTypeIO):
@@ -171,8 +189,7 @@ class ListIO(_GraphBinaryTypeIO):
@classmethod
def objectify(cls, buff, reader):
- x = buff.read(4)
- size = struct.unpack(">i", x)[0]
+ size = cls.read_int(buff)
the_list = []
while size > 0:
the_list.append(reader.readObject(buff))
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index 1fd2a19..b95b0e0 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -43,6 +43,11 @@ class TestGraphSONWriter(object):
graphbinary_writer = GraphBinaryWriter()
graphbinary_reader = GraphBinaryReader()
+ def test_int(self):
+ x = 100
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
def test_string(self):
x = "serialize this!"
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
@@ -52,3 +57,8 @@ class TestGraphSONWriter(object):
x = ["serialize this!", "serialize that!", "stop telling me what to serialize"]
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+
+ def test_heterogeneous_list(self):
+ x = ["serialize this!", 0, "serialize that!", 1, "stop telling me what to serialize", 2]
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
[tinkerpop] 04/33: Added Long graphbinary support to python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit c4c38d385ff369f47bb838dd331adb3a9c8139c2
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Mon Jul 22 14:14:01 2019 -0400
Added Long graphbinary support to python
Fixed bug in string serialization that only occurred on python 3.x
---
.../gremlin_python/structure/io/graphbinaryV1.py | 30 +++++++++++++++++-----
.../tests/structure/io/test_graphbinaryV1.py | 5 ++++
2 files changed, 28 insertions(+), 7 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 9c9ab30..225b399 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -49,6 +49,7 @@ _deserializers = {}
class DataType(Enum):
int = 0x01
+ long = 0x02
string = 0x03
list = 0x09
@@ -146,18 +147,33 @@ class _GraphBinaryTypeIO(object):
raise NotImplementedError()
-class IntIO(_GraphBinaryTypeIO):
+class LongIO(_GraphBinaryTypeIO):
- python_type = IntType
- graphbinary_type = DataType.int
+ python_type = LongType
+ graphbinary_type = DataType.long
+ byte_format = ">q"
@classmethod
def dictify(cls, n, writer):
- return cls.as_bytes(cls.graphbinary_type, None, struct.pack(">i", n))
+ if n < -9223372036854775808 or n > 9223372036854775807:
+ raise Exception("TODO: don't forget bigint")
+ else:
+ return cls.as_bytes(cls.graphbinary_type, None, struct.pack(cls.byte_format, n))
@classmethod
- def objectify(cls, b, reader):
- return cls.read_int(b)
+ def objectify(cls, buff, reader):
+ return struct.unpack(">q", buff.read(8))[0]
+
+
+class IntIO(LongIO):
+
+ python_type = IntType
+ graphbinary_type = DataType.int
+ byte_format = ">i"
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ return cls.read_int(buff)
class StringIO(_GraphBinaryTypeIO):
@@ -171,7 +187,7 @@ class StringIO(_GraphBinaryTypeIO):
@classmethod
def objectify(cls, b, reader):
- return b.read(cls.read_int(b))
+ return b.read(cls.read_int(b)).decode("utf-8")
class ListIO(_GraphBinaryTypeIO):
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index b95b0e0..766544d 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -48,6 +48,11 @@ class TestGraphSONWriter(object):
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+ def test_long(self):
+ x = long(100)
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
def test_string(self):
x = "serialize this!"
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
[tinkerpop] 29/33: Fixed up BulkSet python graphbinary serialization
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 9fd5b565ab99c67af9f94b7d84f0f8a142b295b6
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Tue Aug 13 17:16:09 2019 -0400
Fixed up BulkSet python graphbinary serialization
---
.../src/main/jython/gremlin_python/structure/io/graphbinaryV1.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 5261952..ce03f40 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -908,7 +908,7 @@ class BulkSetIO(_GraphBinaryTypeIO):
the_list = []
while size > 0:
itm = r.readObject(b)
- bulk = cls.read_int(b)
+ bulk = struct.unpack(">q", b.read(8))[0]
for y in range(bulk):
the_list.append(itm)
size = size - 1
[tinkerpop] 26/33: Fixed handling of the null type in python
graphbinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit af5a2994e25ab439bd9dce6273fdf82cafee2821
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Tue Aug 13 14:58:49 2019 -0400
Fixed handling of the null type in python graphbinary
---
.../src/main/jython/gremlin_python/structure/io/graphbinaryV1.py | 8 +++++---
gremlin-python/src/main/jython/tests/conftest.py | 3 ++-
.../src/main/jython/tests/driver/test_driver_remote_connection.py | 5 ++---
3 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index e730c8e..a8eb853 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -157,10 +157,12 @@ class GraphBinaryReader(object):
def toObject(self, buff, data_type=None, nullable=True):
if data_type is None:
- bt = buff.read(1)
- if bt[0] == DataType.null.value:
+ bt = struct.unpack('>B', buff.read(1))[0]
+ if bt == DataType.null.value:
+ if nullable:
+ buff.read(1)
return None
- return self.deserializers[DataType(struct.unpack('>b', bt)[0])].objectify(buff, self, nullable)
+ return self.deserializers[DataType(bt)].objectify(buff, self, nullable)
else:
return self.deserializers[data_type].objectify(buff, self, nullable)
diff --git a/gremlin-python/src/main/jython/tests/conftest.py b/gremlin-python/src/main/jython/tests/conftest.py
index a1224ac..7d10109 100644
--- a/gremlin-python/src/main/jython/tests/conftest.py
+++ b/gremlin-python/src/main/jython/tests/conftest.py
@@ -74,7 +74,8 @@ def client(request):
@pytest.fixture
def secure_client(request):
try:
- client = Client('ws://' + gremlin_server_host + ':45941/gremlin', 'gmodern', username='stephen', password='password')
+ client = Client('ws://' + gremlin_server_host + ':45941/gremlin', 'gmodern',
+ username='stephen', password='password')
except OSError:
pytest.skip('Gremlin Server is not running')
else:
diff --git a/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py b/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py
index b2b4a7d..1c689d9 100644
--- a/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py
+++ b/gremlin-python/src/main/jython/tests/driver/test_driver_remote_connection.py
@@ -154,12 +154,11 @@ class TestDriverRemoteConnection(object):
assert 6 == g.E().count().next()
-def test_in_tornado_app(remote_connection):
+def test_in_tornado_app():
# Make sure nothing weird with loops
@gen.coroutine
def go():
- conn = DriverRemoteConnection(
- 'ws://localhost:45940/gremlin', 'gmodern', pool_size=4)
+ conn = DriverRemoteConnection('ws://localhost:45940/gremlin', 'gmodern', pool_size=4)
g = traversal().withRemote(conn)
yield gen.sleep(0)
assert len(g.V().toList()) == 6
[tinkerpop] 33/33: Refactored python graphbinary to stop using
special means of string conversion
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 34d26b0aa8f449e7d8b6b8968bb352a66c8af182
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Aug 16 14:49:56 2019 -0400
Refactored python graphbinary to stop using special means of string conversion
---
.../gremlin_python/structure/io/graphbinaryV1.py | 31 +++++++++-------------
1 file changed, 12 insertions(+), 19 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 6fc73ab..5fd2fe5 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -191,13 +191,6 @@ class _GraphBinaryTypeIO(object):
return ba
@classmethod
- def string_as_bytes(cls, s):
- ba = bytearray()
- ba.extend(struct.pack(">i", len(s)))
- ba.extend(s.encode("utf-8"))
- return ba
-
- @classmethod
def read_int(cls, buff):
return struct.unpack(">i", buff.read(4))[0]
@@ -454,11 +447,11 @@ class EdgeIO(_GraphBinaryTypeIO):
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(writer.writeObject(obj.id))
- ba.extend(cls.string_as_bytes(obj.label))
+ ba.extend(StringIO.dictify(obj.label, writer, True, False))
ba.extend(writer.writeObject(obj.inV.id))
- ba.extend(cls.string_as_bytes(obj.inV.label))
+ ba.extend(StringIO.dictify(obj.inV.label, writer, True, False))
ba.extend(writer.writeObject(obj.outV.id))
- ba.extend(cls.string_as_bytes(obj.outV.label))
+ ba.extend(StringIO.dictify(obj.outV.label, writer, True, False))
ba.extend(NULL_BYTES)
ba.extend(NULL_BYTES)
return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@@ -502,7 +495,7 @@ class PropertyIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
- ba.extend(cls.string_as_bytes(obj.key))
+ ba.extend(StringIO.dictify(obj.key, writer, True, False))
ba.extend(writer.writeObject(obj.value))
ba.extend(NULL_BYTES)
return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@@ -541,7 +534,7 @@ class VertexIO(_GraphBinaryTypeIO):
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(writer.writeObject(obj.id))
- ba.extend(cls.string_as_bytes(obj.label))
+ ba.extend(StringIO.dictify(obj.label, writer, True, False))
ba.extend(NULL_BYTES)
return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@@ -565,7 +558,7 @@ class VertexPropertyIO(_GraphBinaryTypeIO):
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
ba.extend(writer.writeObject(obj.id))
- ba.extend(cls.string_as_bytes(obj.label))
+ ba.extend(StringIO.dictify(obj.label, writer, True, False))
ba.extend(writer.writeObject(obj.value))
ba.extend(NULL_BYTES)
ba.extend(NULL_BYTES)
@@ -648,7 +641,7 @@ class BindingIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
- ba.extend(cls.string_as_bytes(obj.key))
+ ba.extend(StringIO.dictify(obj.key, writer, True, False))
ba.extend(writer.writeObject(obj.value))
return cls.as_bytes(cls.graphbinary_type, as_value, nullable, ba)
@@ -668,7 +661,7 @@ class BytecodeIO(_GraphBinaryTypeIO):
ba.extend(struct.pack(">i", len(bc.step_instructions)))
for inst in bc.step_instructions:
inst_name, inst_args = inst[0], inst[1:] if len(inst) > 1 else []
- ba.extend(cls.string_as_bytes(inst_name))
+ ba.extend(StringIO.dictify(inst_name, writer, True, False))
ba.extend(struct.pack(">i", len(inst_args)))
for arg in inst_args:
ba.extend(writer.writeObject(arg))
@@ -676,7 +669,7 @@ class BytecodeIO(_GraphBinaryTypeIO):
ba.extend(struct.pack(">i", len(bc.source_instructions)))
for inst in bc.source_instructions:
inst_name, inst_args = inst[0], inst[1:] if len(inst) > 1 else []
- ba.extend(cls.string_as_bytes(inst_name))
+ ba.extend(StringIO.dictify(inst_name, writer, True, False))
ba.extend(struct.pack(">i", len(inst_args)))
for arg in inst_args:
if isinstance(arg, TypeType):
@@ -760,7 +753,7 @@ class PSerializer(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
- ba.extend(cls.string_as_bytes(obj.operator))
+ ba.extend(StringIO.dictify(obj.operator, writer, True, False))
args = []
if obj.other is None:
@@ -869,7 +862,7 @@ class TextPSerializer(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
- ba.extend(cls.string_as_bytes(obj.operator))
+ ba.extend(StringIO.dictify(obj.operator, writer, True, False))
args = []
if obj.other is None:
@@ -959,7 +952,7 @@ class ClassSerializer(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
return cls.as_bytes(cls.graphbinary_type, as_value, nullable,
- StringIO.dictify(obj.gremlin_type, writer, True, False))
+ StringIO.dictify(obj.gremlin_type, writer, True, False))
class TraversalStrategySerializer(_GraphBinaryTypeIO):
[tinkerpop] 31/33: Renamed python graphbinary "IO" to der or ser
depending on function
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit eb5e239345b72df0e089a0c19aeb21d792fdf590
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Aug 16 11:16:03 2019 -0400
Renamed python graphbinary "IO" to der or ser depending on function
Makes it a little easier to see what function of io the class supports
---
CHANGELOG.asciidoc | 1 +
docs/src/dev/io/graphbinary.asciidoc | 2 +-
.../gremlin_python/structure/io/graphbinaryV1.py | 20 ++++++++++----------
3 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index 368c161..648a994 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -23,6 +23,7 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima
[[release-3-4-4]]
=== TinkerPop 3.4.4 (Release Date: NOT OFFICIALLY RELEASED YET)
+* Added GraphBinary support for Python.
[[release-3-4-3]]
=== TinkerPop 3.4.3 (Release Date: August 5, 2019)
diff --git a/docs/src/dev/io/graphbinary.asciidoc b/docs/src/dev/io/graphbinary.asciidoc
index 0262387..40d0ac0 100644
--- a/docs/src/dev/io/graphbinary.asciidoc
+++ b/docs/src/dev/io/graphbinary.asciidoc
@@ -32,7 +32,7 @@ It describes arbitrary object graphs with a fully-qualified format:
Where:
-* `{type_code}` is a single byte representing the type number.
+* `{type_code}` is a single unsigned byte representing the type number.
* `{type_info}` is an optional sequence of bytes providing additional information of the type represented. This is
specially useful for representing complex and custom types.
* `{value_flag}` is a single byte providing information about the value. Flags have the following meaning:
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index 738cfe7..cbdda22 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -403,7 +403,7 @@ class ListIO(_GraphBinaryTypeIO):
return the_list
-class SetIO(ListIO):
+class SetDeserializer(ListIO):
python_type = SetType
graphbinary_type = DataType.set
@@ -740,7 +740,7 @@ class TraversalIO(BytecodeIO):
python_type = GraphTraversal
-class LambdaIO(_GraphBinaryTypeIO):
+class LambdaSerializer(_GraphBinaryTypeIO):
python_type = FunctionType
graphbinary_type = DataType.lambda_
@@ -768,7 +768,7 @@ class LambdaIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
-class PIO(_GraphBinaryTypeIO):
+class PSerializer(_GraphBinaryTypeIO):
graphbinary_type = DataType.p
python_type = P
@@ -875,7 +875,7 @@ class BooleanIO(_GraphBinaryTypeIO):
nullable)
-class TextPIO(_GraphBinaryTypeIO):
+class TextPSerializer(_GraphBinaryTypeIO):
graphbinary_type = DataType.textp
python_type = TextP
@@ -901,7 +901,7 @@ class TextPIO(_GraphBinaryTypeIO):
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
-class BulkSetIO(_GraphBinaryTypeIO):
+class BulkSetDeserializer(_GraphBinaryTypeIO):
graphbinary_type = DataType.bulkset
@@ -923,7 +923,7 @@ class BulkSetIO(_GraphBinaryTypeIO):
return the_list
-class MetricsIO(_GraphBinaryTypeIO):
+class MetricsDeserializer(_GraphBinaryTypeIO):
graphbinary_type = DataType.metrics
@@ -948,7 +948,7 @@ class MetricsIO(_GraphBinaryTypeIO):
"metrics": metrics}
-class TraversalMetricsIO(_GraphBinaryTypeIO):
+class TraversalMetricsDeserializer(_GraphBinaryTypeIO):
graphbinary_type = DataType.traversalmetrics
@@ -965,7 +965,7 @@ class TraversalMetricsIO(_GraphBinaryTypeIO):
"metrics": metrics}
-class ClassIO(_GraphBinaryTypeIO):
+class ClassSerializer(_GraphBinaryTypeIO):
graphbinary_type = DataType.clazz
python_type = GremlinType
@@ -975,14 +975,14 @@ class ClassIO(_GraphBinaryTypeIO):
cls.graphbinary_type, as_value), None, nullable, StringIO.dictify(obj.gremlin_type, writer, True, False))
-class TraversalStrategyIO(_GraphBinaryTypeIO):
+class TraversalStrategySerializer(_GraphBinaryTypeIO):
graphbinary_type = DataType.traversalstrategy
python_type = TraversalStrategy
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
ba = bytearray()
- ba.extend(ClassIO.dictify(GremlinType(obj.fqcn), writer, True, False))
+ ba.extend(ClassSerializer.dictify(GremlinType(obj.fqcn), writer, True, False))
conf = {k: cls._convert(v) for k, v in obj.configuration.items()}
ba.extend(MapIO.dictify(conf, writer, True, False))
return cls.as_bytes(cls.write_as_value(cls.graphbinary_type, as_value), None, nullable, ba)
[tinkerpop] 25/33: Fixed a basic integration test runs for python 2.
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit f14606b8b06bc0cd45da5e5e7d67213909b8da65
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Tue Aug 13 11:46:50 2019 -0400
Fixed a basic integration test runs for python 2.
---
.../src/main/jython/gremlin_python/driver/serializer.py | 8 +++++++-
.../jython/gremlin_python/structure/io/graphbinaryV1.py | 14 ++++++++++----
gremlin-python/src/main/jython/tests/conftest.py | 6 +++++-
3 files changed, 22 insertions(+), 6 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py b/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
index a95d292..a0493f4 100644
--- a/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
+++ b/gremlin-python/src/main/jython/gremlin_python/driver/serializer.py
@@ -237,7 +237,13 @@ class GraphBinarySerializersV1(object):
ba.extend(struct.pack(">i", len(args)))
for k, v in args.items():
ba.extend(self._graphbinary_writer.writeObject(k))
- ba.extend(self._graphbinary_writer.writeObject(v))
+
+ # processor_obj.get_op_args in serialize_message() seems to already handle bytecode. in python 3
+ # because bytearray isn't bound to a type in graphbinary it falls through the writeObject() and
+ # just works but python 2 bytearray is bound to ByteBufferType so it writes DataType.bytebuffer
+ # rather than DataType.bytecode and the server gets confused. special casing this for now until
+ # it can be refactored
+ ba.extend(v if k == "gremlin" else self._graphbinary_writer.writeObject(v))
return bytes(ba)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index dd829cf..e730c8e 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -40,6 +40,7 @@ from gremlin_python.statics import FloatType, FunctionType, IntType, LongType, T
from gremlin_python.process.traversal import Barrier, Binding, Bytecode, Cardinality, Column, Direction, Operator, \
Order, Pick, Pop, P, Scope, TextP, Traversal, Traverser, \
TraversalStrategy, T
+from gremlin_python.process.graph_traversal import GraphTraversal
from gremlin_python.structure.graph import Graph, Edge, Property, Vertex, VertexProperty, Path
log = logging.getLogger(__name__)
@@ -663,17 +664,18 @@ class BytecodeIO(_GraphBinaryTypeIO):
@classmethod
def dictify(cls, obj, writer, as_value=False, nullable=True):
+ bc = obj.bytecode if isinstance(obj, Traversal) else obj
ba = bytearray()
- ba.extend(struct.pack(">i", len(obj.step_instructions)))
- for inst in obj.step_instructions:
+ ba.extend(struct.pack(">i", len(bc.step_instructions)))
+ for inst in bc.step_instructions:
inst_name, inst_args = inst[0], inst[1:] if len(inst) > 1 else []
ba.extend(cls.string_as_bytes(inst_name))
ba.extend(struct.pack(">i", len(inst_args)))
for arg in inst_args:
ba.extend(writer.writeObject(arg))
- ba.extend(struct.pack(">i", len(obj.source_instructions)))
- for inst in obj.source_instructions:
+ ba.extend(struct.pack(">i", len(bc.source_instructions)))
+ for inst in bc.source_instructions:
inst_name, inst_args = inst[0], inst[1:] if len(inst) > 1 else []
ba.extend(cls.string_as_bytes(inst_name))
ba.extend(struct.pack(">i", len(inst_args)))
@@ -720,6 +722,10 @@ class BytecodeIO(_GraphBinaryTypeIO):
return bytecode
+class TraversalIO(BytecodeIO):
+ python_type = GraphTraversal
+
+
class LambdaIO(_GraphBinaryTypeIO):
python_type = FunctionType
diff --git a/gremlin-python/src/main/jython/tests/conftest.py b/gremlin-python/src/main/jython/tests/conftest.py
index 9597f4d..a1224ac 100644
--- a/gremlin-python/src/main/jython/tests/conftest.py
+++ b/gremlin-python/src/main/jython/tests/conftest.py
@@ -32,6 +32,7 @@ from gremlin_python.driver.serializer import (
GraphBinarySerializersV1)
from gremlin_python.driver.tornado.transport import TornadoTransport
+# docker Gremlin Server = 172.17.0.2
gremlin_server_host = "localhost"
gremlin_server_url = 'ws://' + gremlin_server_host + ':45940/gremlin'
@@ -92,8 +93,11 @@ def remote_connection(request):
elif request.param == 'graphsonv2':
remote_conn = DriverRemoteConnection(gremlin_server_url, 'gmodern',
message_serializer=serializer.GraphSONSerializersV2d0())
+ elif request.param == 'graphsonv3':
+ remote_conn = DriverRemoteConnection(gremlin_server_url, 'gmodern',
+ message_serializer=serializer.GraphSONSerializersV3d0())
else:
- remote_conn = DriverRemoteConnection(gremlin_server_url, 'gmodern')
+ raise ValueError("Invalid serializer option - " + request.param)
except OSError:
pytest.skip('Gremlin Server is not running')
else:
[tinkerpop] 06/33: Added timestamp for graphbinary in python
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 6693c9c32b8b82a49183803bbcd398bb87a5b395
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Mon Jul 22 18:45:58 2019 -0400
Added timestamp for graphbinary in python
---
.../gremlin_python/structure/io/graphbinaryV1.py | 19 +++++++++++++++++++
.../jython/tests/structure/io/test_graphbinaryV1.py | 7 ++++++-
2 files changed, 25 insertions(+), 1 deletion(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index a3c7bc9..33342de 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -52,6 +52,7 @@ class DataType(Enum):
long = 0x02
string = 0x03
date = 0x04
+ timestamp = 0x05
list = 0x09
@@ -200,6 +201,24 @@ class DateIO(_GraphBinaryTypeIO):
return datetime.datetime.utcfromtimestamp(struct.unpack(">q", buff.read(8))[0] / 1000.0)
+# Based on current implementation, this class must always be declared before FloatIO.
+# Seems pretty fragile for future maintainers. Maybe look into this.
+class TimestampIO(_GraphBinaryTypeIO):
+ python_type = statics.timestamp
+ graphbinary_type = DataType.timestamp
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ # Java timestamp expects milliseconds integer - Have to use int because of legacy Python
+ ts = int(round(obj * 1000))
+ return cls.as_bytes(cls.graphbinary_type, None, struct.pack(">q", ts))
+
+ @classmethod
+ def objectify(cls, buff, reader):
+ # Python timestamp expects seconds
+ return statics.timestamp(struct.unpack(">q", buff.read(8))[0] / 1000.0)
+
+
class StringIO(_GraphBinaryTypeIO):
python_type = str
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index a0a5be1..5775f3a 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -27,7 +27,7 @@ from mock import Mock
import six
-from gremlin_python.statics import *
+from gremlin_python.statics import timestamp, long
from gremlin_python.structure.graph import Vertex, Edge, Property, VertexProperty, Graph, Path
from gremlin_python.structure.io.graphbinaryV1 import GraphBinaryWriter, GraphBinaryReader, DataType
from gremlin_python.process.traversal import P
@@ -58,6 +58,11 @@ class TestGraphSONWriter(object):
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+ def test_timestamp(self):
+ x = timestamp(1481750076295 / 1000)
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
def test_string(self):
x = "serialize this!"
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
[tinkerpop] 17/33: Added traverser, byte,
bytearray to python graphbinary
Posted by sp...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch TINKERPOP-2279
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 89e9dd74ab8fc0edefb9e56f9f029bfa9b774a9b
Author: Stephen Mallette <sp...@genoprime.com>
AuthorDate: Fri Aug 2 06:39:56 2019 -0400
Added traverser, byte, bytearray to python graphbinary
---
.../gremlin_python/structure/io/graphbinaryV1.py | 51 +++++++++++++++++++++-
.../tests/structure/io/test_graphbinaryV1.py | 12 ++++-
2 files changed, 61 insertions(+), 2 deletions(-)
diff --git a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
index edb1136..208a1e3 100644
--- a/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/gremlin_python/structure/io/graphbinaryV1.py
@@ -85,6 +85,11 @@ class DataType(Enum):
p = 0x1e
scope = 0x1f
t = 0x20
+ traverser = 0x21
+ bigdecimal = 0x22 #todo
+ biginteger = 0x23 #todo
+ byte = 0x24
+ bytebuffer = 0x25
class GraphBinaryTypeType(type):
@@ -702,4 +707,48 @@ class ScopeIO(_EnumIO):
class TIO(_EnumIO):
graphbinary_type = DataType.t
- python_type = T
\ No newline at end of file
+ python_type = T
+
+
+class TraverserIO(_GraphBinaryTypeIO):
+ graphbinary_type = DataType.traverser
+ python_type = Traverser
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(struct.pack(">i", obj.bulk))
+ ba.extend(writer.writeObject(obj.object))
+
+ return ba
+
+
+class ByteIO(_GraphBinaryTypeIO):
+ python_type = SingleByte
+ graphbinary_type = DataType.byte
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ ba = bytearray([cls.graphbinary_type.value])
+ ba.extend(struct.pack(">b", obj))
+ return ba
+
+ @classmethod
+ def objectify(cls, b, reader):
+ return int.__new__(SingleByte, struct.unpack_from(">b", b.read(1))[0])
+
+
+class ByteBufferIO(_GraphBinaryTypeIO):
+ python_type = ByteBufferType
+ graphbinary_type = DataType.bytebuffer
+
+ @classmethod
+ def dictify(cls, obj, writer):
+ return cls.as_bytes(cls.graphbinary_type, len(obj), obj)
+
+ @classmethod
+ def objectify(cls, b, reader):
+ size = cls.read_int(b)
+ return ByteBufferType(b.read(size))
+
+
diff --git a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
index c9ce87e..4c827ad 100644
--- a/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
+++ b/gremlin-python/src/main/jython/tests/structure/io/test_graphbinaryV1.py
@@ -28,7 +28,7 @@ from mock import Mock
import six
-from gremlin_python.statics import timestamp, long
+from gremlin_python.statics import timestamp, long, SingleByte, SingleChar, ByteBufferType
from gremlin_python.structure.graph import Vertex, Edge, Property, VertexProperty, Graph, Path
from gremlin_python.structure.io.graphbinaryV1 import GraphBinaryWriter, GraphBinaryReader, DataType
from gremlin_python.process.traversal import P, Barrier, Binding, Bytecode
@@ -168,4 +168,14 @@ class TestGraphSONWriter(object):
output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
assert x == output
+ def test_byte(self):
+ x = int.__new__(SingleByte, 1)
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+
+ def test_bytebuffer(self):
+ x = ByteBufferType("c29tZSBieXRlcyBmb3IgeW91", "utf8")
+ output = self.graphbinary_reader.readObject(self.graphbinary_writer.writeObject(x))
+ assert x == output
+