[Discuss] Plans for Graph Systems : MongoDB specifically

[Fred Eisele <fr...@gmail.com>]

We have a graph-centric project which uses MongoDB as its backing store.
We would like to use the TinkerPop3 API to provide access to our graph.

Q1: Is there any plan to provide a TinkerPop3 wrapper for MongoDB?

Q2: Would recommended path be to update
https://github.com/datablend/blueprints-mongodb-graph ?

Q3: What is the best TinkerPop3 reference for constructing a TinkerPop3
wrapper around a graph-system?

Thanks
Fred

Re: [Discuss] Plans for Graph Systems : MongoDB specifically

[Marko Rodriguez <ok...@gmail.com>]

Hi,

> Q1: Is there any plan to provide a TinkerPop3 wrapper for MongoDB?

No. Apache TinkerPop will not be a home for implementations like TinkerPop2 was. All we distribute are "reference implementations."
	TinkerGraph -- in-memory POJO (OLTP and OLAP)
	Neo4j -- transactional graph database (OLTP)
	Spark/Giraph -- distributed graph processors (OLAP).

> Q2: Would recommended path be to update
> https://github.com/datablend/blueprints-mongodb-graph ?

No. I would start from scratch. TinkerPop3 is significantly different from TinkerPop2, though you could study their code to get ideas for how they represent their graph in MongoDB.
> 
> Q3: What is the best TinkerPop3 reference for constructing a TinkerPop3
> wrapper around a graph-system?

I think Pieter Martin would be the best to answer this question as he is a TinkerPop committer and develops Sqlg (http://umlg.org/sqlg.html).

However, in the meantime, I would study the TinkerGraph or Neo4j implementations that are distributed with Apache TinkerPop.
	https://github.com/apache/incubator-tinkerpop/tree/master/tinkergraph-gremlin/src/main/java/org/apache/tinkerpop/gremlin/tinkergraph/structure
	https://github.com/apache/incubator-tinkerpop/tree/master/neo4j-gremlin/src/main/java/org/apache/tinkerpop/gremlin/neo4j/structure

Finally, we have this documentation for providers:
	http://tinkerpop.apache.org/docs/3.1.2-incubating/dev/provider/

The above might be a little clunky. It would be great, @Pieter, if you could take all your experience with Sqlg and make the above provider-documentaiton better.

Anywho, hope that helps,
Marko.

http://markorodriguez.com

On Apr 8, 2016, at 10:04 AM, Fred Eisele <fr...@gmail.com> wrote:

> We have a graph-centric project which uses MongoDB as its backing store.
> We would like to use the TinkerPop3 API to provide access to our graph.
> 
> Thanks
> Fred

Re: [Discuss] Plans for Graph Systems : MongoDB specifically

[Fred Eisele <fr...@gmail.com>]

It looks like I should start here?
http://tinkerpop.apache.org/docs/3.0.2-incubating/#implementations


On Fri, Apr 8, 2016 at 11:04 AM, Fred Eisele <fr...@gmail.com>
wrote:

> We have a graph-centric project which uses MongoDB as its backing store.
> We would like to use the TinkerPop3 API to provide access to our graph.
>
> Q1: Is there any plan to provide a TinkerPop3 wrapper for MongoDB?
>
> Q2: Would recommended path be to update
> https://github.com/datablend/blueprints-mongodb-graph ?
>
> Q3: What is the best TinkerPop3 reference for constructing a TinkerPop3
> wrapper around a graph-system?
>
> Thanks
> Fred
>
>
>

Re: [Discuss] Plans for Graph Systems : MongoDB specifically

[pieter-gmail <pi...@gmail.com>]

Hi,

Do you want to use gremlin to query an existing graphy mongo db?

If so that will probably be a difficult task as TinkerPop assumes the
underlying graph is modeled according to the property graph model.

If not then its not that difficult. First you'll have to work out how
you are going to model the property graph model in MongoDb (This at
least for me is the fun part). Once that is done TinkerPop allows the
rest to fall into place.

The difficulty after that is to optimize gremlin queries.
eg.
g.V(vertex1).out().out()

If the first out() returns 1 000 000 vertices then the second out() will
result in 1 000 000 calls to mongodb. MongoDb is fast but a million
separate individual calls kills any applications performance. So you
have to optimize traversing edges to execute on the db itself as oppose
to doing it from the application (i.e. your TinkerPop mongo java
implementation). I have not recently googled this but I understand
(hope) that mongo will be implementing edges/links/joins natively.

Either way the primary challenge is to optimize the two "out().out()"
calls to be only one call to the db.

As an aside, TinkerPop has no support at the moment to query a JSON
property. In fact TinkerPop has no notion of type so it does not even
know nor care that a particular property's type is JSON.
Recent versions of Postgresql support JSON natively and Sqlg supports
storing JSON blobs. However at present it is unqueryable from Sqlg
(gremlin). Postgresql has sophisticated JSON query abilities that I
intend to make available via custom TinkerPop predicates. TinkerPop uses
the notion of Predicates to select data. It is fairly easy to extend
TinkerPop to expose custom db features. I imagine that doing something
similar will make a MongoDb implementation able to use gremlin to query
arbitrary JSON documents. My understanding of the joy of mongo is that
there is no need to transform data from one representation to another to
yet another. JSON from browser to something(Node) to db and all is
simple. So I imagine that using MongoDb as a graph db should do its best
not to obfuscate this benefit.

Cheers
Pieter


On 08/04/2016 18:04, Fred Eisele wrote:
> We have a graph-centric project which uses MongoDB as its backing store.
> We would like to use the TinkerPop3 API to provide access to our graph.
>
> Q1: Is there any plan to provide a TinkerPop3 wrapper for MongoDB?
>
> Q2: Would recommended path be to update
> https://github.com/datablend/blueprints-mongodb-graph ?
>
> Q3: What is the best TinkerPop3 reference for constructing a TinkerPop3
> wrapper around a graph-system?
>
> Thanks
> Fred
>