You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by Marko Rodriguez <ok...@gmail.com> on 2015/06/09 16:21:43 UTC
[TinkerPop3] Neo4j-Gremlin is now available in master/
Hello everyone,
With Michael Hunger's help, NeoTechnology has published the following two artifacts to Apache's Maven Central Repo:
http://repo1.maven.org/maven2/org/neo4j/neo4j-tinkerpop-api/ (Apache2 licensed API)
http://repo1.maven.org/maven2/org/neo4j/neo4j-tinkerpop-api-impl/ (AGPL licensed API implementation)
Note that TinkerPop3's Neo4j-Gremlin only <depends/> on the Apache2 licensed neo4j-tinkerpop-api artifact. When a user wishes to :install Neo4j-Gremlin (e.g. GremlinServer or GremlinConsole), then the AGPL binaries will be downloaded from Maven Central to the user's machine.
Neo4j-Gremlin was originally in TinkerPop3 M1 through M7. However, once TinkerPop went Apache Software Foundation, we had to gut it until the above ASF permitted model was implemented. Note that the current Neo4jGraph implementation has seen a lot of updates since M7 so please give it a test drive and find any problems issues.
http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#neo4j-gremlin
A collection of notes:
1. Neo4j 2.2+ has done a lot to ensure transaction consistency between indices and global graph operations.
- All the "isDeleted()" checks in Neo4jGraph are gone.
2. The entire TinkerPop3 test suite passes except for TransactionTest test around graph.close() semantics.
- Michael Hunger is looking into the problem. Right now its OPT_OUT as its not a critical test.
3. Neo4jGraph no longer supports legacy indices -- only schema indices.
- This greatly simplified the code and ensured no @Deprecated references.
- Creating indices in Neo4jGraph is done via Cypher: graph.cypher("CREATE INDEX on :person(name)").
- http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#_indices
4. Neo4jGraph supports Neo4j multi-labels both at the Neo4jVertex API level and at the Neo4jGraphStep index lookup level.
- We now have LabelP.of() which allows for g.V.has(label,of('person')).
- http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#_multi_label
5. Neo4jGraph supports multi/meta-properties though this is considered an experimental feature until it is more fully tested at scale and high concurrency.
- The fear is that multi/meta-properties diverges from the native Neo4j representation and until we are comfortable with the embedding, its not safe for production use.
- http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#_multi_meta_properties
5. And of course, you can still go Cypher->Gremlin which is really cool.
- http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#_cypher
Thanks again to NeoTechnology for working with TinkerPop and releasing an Apache2 licensed version of their API so the community can enjoy Neo4j-Gremlin.
Take care,
Marko.
http://markorodriguez.com
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Stephen Mallette <sp...@gmail.com>.
Mentors, as Marko mentions the "ASF permitted model" for Neo4j, i was
wondering if any of you have had a chance to consider our development model
around neo4j-gremlin. I outlined it here:
http://bit.ly/1QwKqDt
in a previous post. Is that an acceptable way to do this?
Thanks,
Stephen
On Tue, Jun 9, 2015 at 10:21 AM, Marko Rodriguez <ok...@gmail.com>
wrote:
> Hello everyone,
>
> With Michael Hunger's help, NeoTechnology has published the following two
> artifacts to Apache's Maven Central Repo:
>
> http://repo1.maven.org/maven2/org/neo4j/neo4j-tinkerpop-api/ (Apache2
> licensed API)
> http://repo1.maven.org/maven2/org/neo4j/neo4j-tinkerpop-api-impl/ (AGPL
> licensed API implementation)
>
> Note that TinkerPop3's Neo4j-Gremlin only <depends/> on the Apache2
> licensed neo4j-tinkerpop-api artifact. When a user wishes to :install
> Neo4j-Gremlin (e.g. GremlinServer or GremlinConsole), then the AGPL
> binaries will be downloaded from Maven Central to the user's machine.
>
> Neo4j-Gremlin was originally in TinkerPop3 M1 through M7. However, once
> TinkerPop went Apache Software Foundation, we had to gut it until the above
> ASF permitted model was implemented. Note that the current Neo4jGraph
> implementation has seen a lot of updates since M7 so *please give it a
> test drive and find any problems issues*.
>
> http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#neo4j-gremlin
>
> A collection of notes:
>
> 1. Neo4j 2.2+ has done a lot to ensure transaction consistency between
> indices and global graph operations.
> - All the "isDeleted()" checks in Neo4jGraph are gone.
> 2. The entire TinkerPop3 test suite passes except for TransactionTest test
> around graph.close() semantics.
> - Michael Hunger is looking into the problem. Right now its OPT_OUT as its
> not a critical test.
> 3. Neo4jGraph no longer supports legacy indices -- only schema indices.
> - This greatly simplified the code and ensured no @Deprecated references.
> - Creating indices in Neo4jGraph is done via Cypher: graph.cypher("CREATE
> INDEX on :person(name)").
> - http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#_indices
> 4. Neo4jGraph supports Neo4j multi-labels both at the Neo4jVertex API
> level and at the Neo4jGraphStep index lookup level.
> - We now have LabelP.of() which allows for g.V.has(label,of('person')).
> - http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#_multi_label
> 5. Neo4jGraph supports multi/meta-properties though this is considered an
> experimental feature until it is more fully tested at scale and high
> concurrency.
> - The fear is that multi/meta-properties diverges from the native Neo4j
> representation and until we are comfortable with the embedding, its not
> safe for production use.
> -
> http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#_multi_meta_properties
> 5. And of course, you can still go Cypher->Gremlin which is really cool.
> - http://tinkerpop.incubator.apache.org/docs/3.0.0-SNAPSHOT/#_cypher
>
> Thanks again to NeoTechnology for working with TinkerPop and releasing an
> Apache2 licensed version of their API so the community can enjoy
> Neo4j-Gremlin.
>
> Take care,
> Marko.
>
> http://markorodriguez.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gremlin-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/A57B78DF-0F69-441E-9AB1-36F543DCF0AC%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/A57B78DF-0F69-441E-9AB1-36F543DCF0AC%40gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Stephen Mallette <sp...@gmail.com>.
wow - i didn't expect that to make such a difference. i may need to look
at the code again to see why there is such a big difference considering i
dismissed it so easily as impacting performance.
On Fri, Jun 12, 2015 at 8:25 PM, Michael Pollmeier <
michael@michaelpollmeier.com> wrote:
> That's a good tip! I just tried with a different example in neo4j and the
> graph creation time went from 50s using `sg.addVertex("some
> label").setProperty(s"property ${math.random}", math.random)`
> to 40s using `sg.addVertex("some label", Map(s"property ${math.random}",
> math.random))`
>
> On 12/06/15 22:06, Stephen Mallette wrote:
>
>> Not sure how ScalaGraph works, but in reference to your code if this:
>>
>> sg.addVertex("some label").setProperty("property ${math.random}",
>> math.random)
>>
>> is not doing this
>>
>> sg.addVertex(label, "some label", "property ${math.random}", math.random)
>>
>> your getting some extra transactional checks that aren't necessary if you
>> know all the properties up-front at the time the vertex is added.
>> changing
>> that probably isn't going to help your load times all that much, but i
>> thought i'd mention it.
>>
>> After a quick review, I think a "batch trait" might be possible. Recall
>> that Neo4jBatchGraph used the Neo4j BatchInserter and not a Neo4jGraphAPI
>> instance - again unsure of how that fits here at the moment. I'm guessing
>> that BatchInserter is not exposed via apache licensed neo4j interfaces at
>> this time so that would be another problem.
>>
>>
>>
>> On Thu, Jun 11, 2015 at 7:22 PM, Michael Pollmeier <
>> michael@michaelpollmeier.com> wrote:
>>
>> Yup that's what I meant.
>>>
>>>
>>> On 12/06/15 10:54, Marko Rodriguez wrote:
>>>
>>> Hey Michael,
>>>>
>>>> I'll let Stephen confirm the path forward, but I bet you could use a
>>>> Neo4jTrait implementation to do the TinkerPop2-style Neo4jBatchGraph.
>>>> Checkout Neo4jTrait in TinkerPop3.
>>>>
>>>> Marko.
>>>>
>>>> http://markorodriguez.com
>>>>
>>>> On Jun 11, 2015, at 4:07 PM, Michael Pollmeier <
>>>> michael@michaelpollmeier.com> wrote:
>>>>
>>>> Here you go - all pretty straightforward:
>>>>
>>>>> https://gist.github.com/mpollmeier/108ab8998e3b0321f020
>>>>>
>>>>> Without a bulk api for neo4j it takes nearly 70s to create 30k
>>>>> vertices.
>>>>>
>>>>> There is currently no publicly available build artifact of tinkerpop3
>>>>> that contains neo4j-gremlin, so this all depends on custom local
>>>>> builds.
>>>>>
>>>>> The main thing is the obvious stuff I guess: disable indexes and
>>>>> integrity constraints. Should be the same as the good old
>>>>> Neo4jBatchGraph,
>>>>> no?
>>>>>
>>>>>
>>>>> https://github.com/tinkerpop/blueprints/tree/master/blueprints-neo4j-graph/src/main/java/com/tinkerpop/blueprints/impls/neo4j/batch
>>>>> I guess for license reasons we can't just copy that but have to
>>>>> reimplement it?
>>>>>
>>>>> Cheers
>>>>> Michael
>>>>>
>>>>> On 12/06/15 01:19, Marko Rodriguez wrote:
>>>>>
>>>>> Michael,
>>>>>>
>>>>>> If there are any areas that can be sped up in the Neo4jGraph codebase,
>>>>>> please identify. There hasn't been much manual testing of Neo4jGraph
>>>>>> so any
>>>>>> help/tickets/PRs you could provide would be greatly appreciated.
>>>>>>
>>>>>> Also, could you share your data loading code via a gist?
>>>>>>
>>>>>> Thanks,
>>>>>> Marko.
>>>>>>
>>>>>> http://markorodriguez.com
>>>>>>
>>>>>> On Jun 11, 2015, at 6:37 AM, Stephen Mallette <sp...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> General support for bulk loading is still under development and
>>>>>> won't
>>>>>>
>>>>>>> be available for GA. Here's some related issues:
>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-319
>>>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-320
>>>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-550
>>>>>>>
>>>>>>> As for just straight loading, I'm not sure we've evaluated
>>>>>>> performance
>>>>>>> of neo4j-gremlin at this point. Not sure what improvements are to
>>>>>>> be had,
>>>>>>> if any.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier <
>>>>>>> michael@michaelpollmeier.com> wrote:
>>>>>>> What's the best way to insert many elements at the same time, is
>>>>>>> there
>>>>>>> some sort of bulk mode that doesn't check for constraints? I didn't
>>>>>>> find
>>>>>>> anything fitting in the code/documentation in TP3.
>>>>>>>
>>>>>>> Creating 25k vertices takes around 40s, and about 50s if I wrap it in
>>>>>>> a transaction - that's a bit slow.
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "Gremlin-users" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send
>>>>>>> an email to gremlin-users+unsubscribe@googlegroups.com.
>>>>>>> To view this discussion on the web visit
>>>>>>>
>>>>>>> https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com
>>>>>>> .
>>>>>>>
>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "Gremlin-users" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send
>>>>>>> an email to gremlin-users+unsubscribe@googlegroups.com.
>>>>>>> To view this discussion on the web visit
>>>>>>>
>>>>>>> https://groups.google.com/d/msgid/gremlin-users/CAA-H438DLd-K%2Baj%3DufCx5w3EtY6r32bTTVznTRgoNYwYOMXxCA%40mail.gmail.com
>>>>>>> .
>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Gremlin-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to gremlin-users+unsubscribe@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>>
>>>>> https://groups.google.com/d/msgid/gremlin-users/557A0695.9090807%40michaelpollmeier.com
>>>>> .
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Gremlin-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to gremlin-users+unsubscribe@googlegroups.com.
>>> To view this discussion on the web visit
>>>
>>> https://groups.google.com/d/msgid/gremlin-users/557A1819.2010108%40michaelpollmeier.com
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gremlin-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/557B786B.2040504%40michaelpollmeier.com
> .
> For more options, visit https://groups.google.com/d/optout.
>
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Michael Pollmeier <mi...@michaelpollmeier.com>.
That's a good tip! I just tried with a different example in neo4j and
the graph creation time went from 50s using `sg.addVertex("some
label").setProperty(s"property ${math.random}", math.random)`
to 40s using `sg.addVertex("some label", Map(s"property ${math.random}",
math.random))`
On 12/06/15 22:06, Stephen Mallette wrote:
> Not sure how ScalaGraph works, but in reference to your code if this:
>
> sg.addVertex("some label").setProperty("property ${math.random}",
> math.random)
>
> is not doing this
>
> sg.addVertex(label, "some label", "property ${math.random}", math.random)
>
> your getting some extra transactional checks that aren't necessary if you
> know all the properties up-front at the time the vertex is added. changing
> that probably isn't going to help your load times all that much, but i
> thought i'd mention it.
>
> After a quick review, I think a "batch trait" might be possible. Recall
> that Neo4jBatchGraph used the Neo4j BatchInserter and not a Neo4jGraphAPI
> instance - again unsure of how that fits here at the moment. I'm guessing
> that BatchInserter is not exposed via apache licensed neo4j interfaces at
> this time so that would be another problem.
>
>
>
> On Thu, Jun 11, 2015 at 7:22 PM, Michael Pollmeier <
> michael@michaelpollmeier.com> wrote:
>
>> Yup that's what I meant.
>>
>>
>> On 12/06/15 10:54, Marko Rodriguez wrote:
>>
>>> Hey Michael,
>>>
>>> I'll let Stephen confirm the path forward, but I bet you could use a
>>> Neo4jTrait implementation to do the TinkerPop2-style Neo4jBatchGraph.
>>> Checkout Neo4jTrait in TinkerPop3.
>>>
>>> Marko.
>>>
>>> http://markorodriguez.com
>>>
>>> On Jun 11, 2015, at 4:07 PM, Michael Pollmeier <
>>> michael@michaelpollmeier.com> wrote:
>>>
>>> Here you go - all pretty straightforward:
>>>> https://gist.github.com/mpollmeier/108ab8998e3b0321f020
>>>>
>>>> Without a bulk api for neo4j it takes nearly 70s to create 30k vertices.
>>>>
>>>> There is currently no publicly available build artifact of tinkerpop3
>>>> that contains neo4j-gremlin, so this all depends on custom local builds.
>>>>
>>>> The main thing is the obvious stuff I guess: disable indexes and
>>>> integrity constraints. Should be the same as the good old Neo4jBatchGraph,
>>>> no?
>>>>
>>>> https://github.com/tinkerpop/blueprints/tree/master/blueprints-neo4j-graph/src/main/java/com/tinkerpop/blueprints/impls/neo4j/batch
>>>> I guess for license reasons we can't just copy that but have to
>>>> reimplement it?
>>>>
>>>> Cheers
>>>> Michael
>>>>
>>>> On 12/06/15 01:19, Marko Rodriguez wrote:
>>>>
>>>>> Michael,
>>>>>
>>>>> If there are any areas that can be sped up in the Neo4jGraph codebase,
>>>>> please identify. There hasn't been much manual testing of Neo4jGraph so any
>>>>> help/tickets/PRs you could provide would be greatly appreciated.
>>>>>
>>>>> Also, could you share your data loading code via a gist?
>>>>>
>>>>> Thanks,
>>>>> Marko.
>>>>>
>>>>> http://markorodriguez.com
>>>>>
>>>>> On Jun 11, 2015, at 6:37 AM, Stephen Mallette <sp...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> General support for bulk loading is still under development and won't
>>>>>> be available for GA. Here's some related issues:
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-319
>>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-320
>>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-550
>>>>>>
>>>>>> As for just straight loading, I'm not sure we've evaluated performance
>>>>>> of neo4j-gremlin at this point. Not sure what improvements are to be had,
>>>>>> if any.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier <
>>>>>> michael@michaelpollmeier.com> wrote:
>>>>>> What's the best way to insert many elements at the same time, is there
>>>>>> some sort of bulk mode that doesn't check for constraints? I didn't find
>>>>>> anything fitting in the code/documentation in TP3.
>>>>>>
>>>>>> Creating 25k vertices takes around 40s, and about 50s if I wrap it in
>>>>>> a transaction - that's a bit slow.
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Gremlin-users" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>> an email to gremlin-users+unsubscribe@googlegroups.com.
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com
>>>>>> .
>>>>>>
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Gremlin-users" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>> an email to gremlin-users+unsubscribe@googlegroups.com.
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/gremlin-users/CAA-H438DLd-K%2Baj%3DufCx5w3EtY6r32bTTVznTRgoNYwYOMXxCA%40mail.gmail.com
>>>>>> .
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Gremlin-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to gremlin-users+unsubscribe@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/gremlin-users/557A0695.9090807%40michaelpollmeier.com
>>>> .
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to gremlin-users+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/557A1819.2010108%40michaelpollmeier.com
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Stephen Mallette <sp...@gmail.com>.
Not sure how ScalaGraph works, but in reference to your code if this:
sg.addVertex("some label").setProperty("property ${math.random}",
math.random)
is not doing this
sg.addVertex(label, "some label", "property ${math.random}", math.random)
your getting some extra transactional checks that aren't necessary if you
know all the properties up-front at the time the vertex is added. changing
that probably isn't going to help your load times all that much, but i
thought i'd mention it.
After a quick review, I think a "batch trait" might be possible. Recall
that Neo4jBatchGraph used the Neo4j BatchInserter and not a Neo4jGraphAPI
instance - again unsure of how that fits here at the moment. I'm guessing
that BatchInserter is not exposed via apache licensed neo4j interfaces at
this time so that would be another problem.
On Thu, Jun 11, 2015 at 7:22 PM, Michael Pollmeier <
michael@michaelpollmeier.com> wrote:
> Yup that's what I meant.
>
>
> On 12/06/15 10:54, Marko Rodriguez wrote:
>
>> Hey Michael,
>>
>> I'll let Stephen confirm the path forward, but I bet you could use a
>> Neo4jTrait implementation to do the TinkerPop2-style Neo4jBatchGraph.
>> Checkout Neo4jTrait in TinkerPop3.
>>
>> Marko.
>>
>> http://markorodriguez.com
>>
>> On Jun 11, 2015, at 4:07 PM, Michael Pollmeier <
>> michael@michaelpollmeier.com> wrote:
>>
>> Here you go - all pretty straightforward:
>>> https://gist.github.com/mpollmeier/108ab8998e3b0321f020
>>>
>>> Without a bulk api for neo4j it takes nearly 70s to create 30k vertices.
>>>
>>> There is currently no publicly available build artifact of tinkerpop3
>>> that contains neo4j-gremlin, so this all depends on custom local builds.
>>>
>>> The main thing is the obvious stuff I guess: disable indexes and
>>> integrity constraints. Should be the same as the good old Neo4jBatchGraph,
>>> no?
>>>
>>> https://github.com/tinkerpop/blueprints/tree/master/blueprints-neo4j-graph/src/main/java/com/tinkerpop/blueprints/impls/neo4j/batch
>>> I guess for license reasons we can't just copy that but have to
>>> reimplement it?
>>>
>>> Cheers
>>> Michael
>>>
>>> On 12/06/15 01:19, Marko Rodriguez wrote:
>>>
>>>> Michael,
>>>>
>>>> If there are any areas that can be sped up in the Neo4jGraph codebase,
>>>> please identify. There hasn't been much manual testing of Neo4jGraph so any
>>>> help/tickets/PRs you could provide would be greatly appreciated.
>>>>
>>>> Also, could you share your data loading code via a gist?
>>>>
>>>> Thanks,
>>>> Marko.
>>>>
>>>> http://markorodriguez.com
>>>>
>>>> On Jun 11, 2015, at 6:37 AM, Stephen Mallette <sp...@gmail.com>
>>>> wrote:
>>>>
>>>> General support for bulk loading is still under development and won't
>>>>> be available for GA. Here's some related issues:
>>>>>
>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-319
>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-320
>>>>> https://issues.apache.org/jira/browse/TINKERPOP3-550
>>>>>
>>>>> As for just straight loading, I'm not sure we've evaluated performance
>>>>> of neo4j-gremlin at this point. Not sure what improvements are to be had,
>>>>> if any.
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier <
>>>>> michael@michaelpollmeier.com> wrote:
>>>>> What's the best way to insert many elements at the same time, is there
>>>>> some sort of bulk mode that doesn't check for constraints? I didn't find
>>>>> anything fitting in the code/documentation in TP3.
>>>>>
>>>>> Creating 25k vertices takes around 40s, and about 50s if I wrap it in
>>>>> a transaction - that's a bit slow.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Gremlin-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to gremlin-users+unsubscribe@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com
>>>>> .
>>>>>
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Gremlin-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to gremlin-users+unsubscribe@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/gremlin-users/CAA-H438DLd-K%2Baj%3DufCx5w3EtY6r32bTTVznTRgoNYwYOMXxCA%40mail.gmail.com
>>>>> .
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>>
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Gremlin-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to gremlin-users+unsubscribe@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/gremlin-users/557A0695.9090807%40michaelpollmeier.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gremlin-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/557A1819.2010108%40michaelpollmeier.com
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Michael Pollmeier <mi...@michaelpollmeier.com>.
Yup that's what I meant.
On 12/06/15 10:54, Marko Rodriguez wrote:
> Hey Michael,
>
> I'll let Stephen confirm the path forward, but I bet you could use a Neo4jTrait implementation to do the TinkerPop2-style Neo4jBatchGraph. Checkout Neo4jTrait in TinkerPop3.
>
> Marko.
>
> http://markorodriguez.com
>
> On Jun 11, 2015, at 4:07 PM, Michael Pollmeier <mi...@michaelpollmeier.com> wrote:
>
>> Here you go - all pretty straightforward:
>> https://gist.github.com/mpollmeier/108ab8998e3b0321f020
>>
>> Without a bulk api for neo4j it takes nearly 70s to create 30k vertices.
>>
>> There is currently no publicly available build artifact of tinkerpop3 that contains neo4j-gremlin, so this all depends on custom local builds.
>>
>> The main thing is the obvious stuff I guess: disable indexes and integrity constraints. Should be the same as the good old Neo4jBatchGraph, no?
>> https://github.com/tinkerpop/blueprints/tree/master/blueprints-neo4j-graph/src/main/java/com/tinkerpop/blueprints/impls/neo4j/batch
>> I guess for license reasons we can't just copy that but have to reimplement it?
>>
>> Cheers
>> Michael
>>
>> On 12/06/15 01:19, Marko Rodriguez wrote:
>>> Michael,
>>>
>>> If there are any areas that can be sped up in the Neo4jGraph codebase, please identify. There hasn't been much manual testing of Neo4jGraph so any help/tickets/PRs you could provide would be greatly appreciated.
>>>
>>> Also, could you share your data loading code via a gist?
>>>
>>> Thanks,
>>> Marko.
>>>
>>> http://markorodriguez.com
>>>
>>> On Jun 11, 2015, at 6:37 AM, Stephen Mallette <sp...@gmail.com> wrote:
>>>
>>>> General support for bulk loading is still under development and won't be available for GA. Here's some related issues:
>>>>
>>>> https://issues.apache.org/jira/browse/TINKERPOP3-319
>>>> https://issues.apache.org/jira/browse/TINKERPOP3-320
>>>> https://issues.apache.org/jira/browse/TINKERPOP3-550
>>>>
>>>> As for just straight loading, I'm not sure we've evaluated performance of neo4j-gremlin at this point. Not sure what improvements are to be had, if any.
>>>>
>>>>
>>>>
>>>> On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier <mi...@michaelpollmeier.com> wrote:
>>>> What's the best way to insert many elements at the same time, is there some sort of bulk mode that doesn't check for constraints? I didn't find anything fitting in the code/documentation in TP3.
>>>>
>>>> Creating 25k vertices takes around 40s, and about 50s if I wrap it in a transaction - that's a bit slow.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
>>>> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com.
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
>>>> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H438DLd-K%2Baj%3DufCx5w3EtY6r32bTTVznTRgoNYwYOMXxCA%40mail.gmail.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/557A0695.9090807%40michaelpollmeier.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Marko Rodriguez <ok...@gmail.com>.
Hey Michael,
I'll let Stephen confirm the path forward, but I bet you could use a Neo4jTrait implementation to do the TinkerPop2-style Neo4jBatchGraph. Checkout Neo4jTrait in TinkerPop3.
Marko.
http://markorodriguez.com
On Jun 11, 2015, at 4:07 PM, Michael Pollmeier <mi...@michaelpollmeier.com> wrote:
> Here you go - all pretty straightforward:
> https://gist.github.com/mpollmeier/108ab8998e3b0321f020
>
> Without a bulk api for neo4j it takes nearly 70s to create 30k vertices.
>
> There is currently no publicly available build artifact of tinkerpop3 that contains neo4j-gremlin, so this all depends on custom local builds.
>
> The main thing is the obvious stuff I guess: disable indexes and integrity constraints. Should be the same as the good old Neo4jBatchGraph, no?
> https://github.com/tinkerpop/blueprints/tree/master/blueprints-neo4j-graph/src/main/java/com/tinkerpop/blueprints/impls/neo4j/batch
> I guess for license reasons we can't just copy that but have to reimplement it?
>
> Cheers
> Michael
>
> On 12/06/15 01:19, Marko Rodriguez wrote:
>> Michael,
>>
>> If there are any areas that can be sped up in the Neo4jGraph codebase, please identify. There hasn't been much manual testing of Neo4jGraph so any help/tickets/PRs you could provide would be greatly appreciated.
>>
>> Also, could you share your data loading code via a gist?
>>
>> Thanks,
>> Marko.
>>
>> http://markorodriguez.com
>>
>> On Jun 11, 2015, at 6:37 AM, Stephen Mallette <sp...@gmail.com> wrote:
>>
>>> General support for bulk loading is still under development and won't be available for GA. Here's some related issues:
>>>
>>> https://issues.apache.org/jira/browse/TINKERPOP3-319
>>> https://issues.apache.org/jira/browse/TINKERPOP3-320
>>> https://issues.apache.org/jira/browse/TINKERPOP3-550
>>>
>>> As for just straight loading, I'm not sure we've evaluated performance of neo4j-gremlin at this point. Not sure what improvements are to be had, if any.
>>>
>>>
>>>
>>> On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier <mi...@michaelpollmeier.com> wrote:
>>> What's the best way to insert many elements at the same time, is there some sort of bulk mode that doesn't check for constraints? I didn't find anything fitting in the code/documentation in TP3.
>>>
>>> Creating 25k vertices takes around 40s, and about 50s if I wrap it in a transaction - that's a bit slow.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H438DLd-K%2Baj%3DufCx5w3EtY6r32bTTVznTRgoNYwYOMXxCA%40mail.gmail.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/557A0695.9090807%40michaelpollmeier.com.
> For more options, visit https://groups.google.com/d/optout.
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Michael Pollmeier <mi...@michaelpollmeier.com>.
Here you go - all pretty straightforward:
https://gist.github.com/mpollmeier/108ab8998e3b0321f020
Without a bulk api for neo4j it takes nearly 70s to create 30k vertices.
There is currently no publicly available build artifact of tinkerpop3
that contains neo4j-gremlin, so this all depends on custom local builds.
The main thing is the obvious stuff I guess: disable indexes and
integrity constraints. Should be the same as the good old
Neo4jBatchGraph, no?
https://github.com/tinkerpop/blueprints/tree/master/blueprints-neo4j-graph/src/main/java/com/tinkerpop/blueprints/impls/neo4j/batch
I guess for license reasons we can't just copy that but have to
reimplement it?
Cheers
Michael
On 12/06/15 01:19, Marko Rodriguez wrote:
> Michael,
>
> If there are any areas that can be sped up in the Neo4jGraph codebase, please identify. There hasn't been much manual testing of Neo4jGraph so any help/tickets/PRs you could provide would be greatly appreciated.
>
> Also, could you share your data loading code via a gist?
>
> Thanks,
> Marko.
>
> http://markorodriguez.com
>
> On Jun 11, 2015, at 6:37 AM, Stephen Mallette <sp...@gmail.com> wrote:
>
>> General support for bulk loading is still under development and won't be available for GA. Here's some related issues:
>>
>> https://issues.apache.org/jira/browse/TINKERPOP3-319
>> https://issues.apache.org/jira/browse/TINKERPOP3-320
>> https://issues.apache.org/jira/browse/TINKERPOP3-550
>>
>> As for just straight loading, I'm not sure we've evaluated performance of neo4j-gremlin at this point. Not sure what improvements are to be had, if any.
>>
>>
>>
>> On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier <mi...@michaelpollmeier.com> wrote:
>> What's the best way to insert many elements at the same time, is there some sort of bulk mode that doesn't check for constraints? I didn't find anything fitting in the code/documentation in TP3.
>>
>> Creating 25k vertices takes around 40s, and about 50s if I wrap it in a transaction - that's a bit slow.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H438DLd-K%2Baj%3DufCx5w3EtY6r32bTTVznTRgoNYwYOMXxCA%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Marko Rodriguez <ok...@gmail.com>.
Michael,
If there are any areas that can be sped up in the Neo4jGraph codebase, please identify. There hasn't been much manual testing of Neo4jGraph so any help/tickets/PRs you could provide would be greatly appreciated.
Also, could you share your data loading code via a gist?
Thanks,
Marko.
http://markorodriguez.com
On Jun 11, 2015, at 6:37 AM, Stephen Mallette <sp...@gmail.com> wrote:
> General support for bulk loading is still under development and won't be available for GA. Here's some related issues:
>
> https://issues.apache.org/jira/browse/TINKERPOP3-319
> https://issues.apache.org/jira/browse/TINKERPOP3-320
> https://issues.apache.org/jira/browse/TINKERPOP3-550
>
> As for just straight loading, I'm not sure we've evaluated performance of neo4j-gremlin at this point. Not sure what improvements are to be had, if any.
>
>
>
> On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier <mi...@michaelpollmeier.com> wrote:
> What's the best way to insert many elements at the same time, is there some sort of bulk mode that doesn't check for constraints? I didn't find anything fitting in the code/documentation in TP3.
>
> Creating 25k vertices takes around 40s, and about 50s if I wrap it in a transaction - that's a bit slow.
>
> --
> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com.
>
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H438DLd-K%2Baj%3DufCx5w3EtY6r32bTTVznTRgoNYwYOMXxCA%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Stephen Mallette <sp...@gmail.com>.
General support for bulk loading is still under development and won't be
available for GA. Here's some related issues:
https://issues.apache.org/jira/browse/TINKERPOP3-319
https://issues.apache.org/jira/browse/TINKERPOP3-320
https://issues.apache.org/jira/browse/TINKERPOP3-550
As for just straight loading, I'm not sure we've evaluated performance of
neo4j-gremlin at this point. Not sure what improvements are to be had, if
any.
On Wed, Jun 10, 2015 at 8:01 PM, Michael Pollmeier <
michael@michaelpollmeier.com> wrote:
> What's the best way to insert many elements at the same time, is there
> some sort of bulk mode that doesn't check for constraints? I didn't find
> anything fitting in the code/documentation in TP3.
>
> Creating 25k vertices takes around 40s, and about 50s if I wrap it in a
> transaction - that's a bit slow.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gremlin-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/5578CFDB.5070603%40michaelpollmeier.com
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Michael Pollmeier <mi...@michaelpollmeier.com>.
What's the best way to insert many elements at the same time, is there
some sort of bulk mode that doesn't check for constraints? I didn't find
anything fitting in the code/documentation in TP3.
Creating 25k vertices takes around 40s, and about 50s if I wrap it in a
transaction - that's a bit slow.
Re: [TinkerPop] [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Marko Rodriguez <ok...@gmail.com>.
Hi Michael,
> I have tested a few basic things with gremlin-scala and Neo4jGraph, and it works fine. Thanks everybody!
Great.
> I have tested it with locally built snapshots as neo4j-gremlin isn't part of M9-incubating.
Yep.
> Will there be a separate release of tinkerpop3 that includes neo4j-gremlin, so that I can share this in my examples project?
The next official release of TinkerPop3 will be 3.0.0-incubating (i.e. GA).
We will go through 1 or 2 RCs that are basically tags in Apache Git.
HTH,
Marko.
http://markorodriguez.com
Re: [TinkerPop3] Neo4j-Gremlin is now available in master/
Posted by Michael Pollmeier <mi...@michaelpollmeier.com>.
I have tested a few basic things with gremlin-scala and Neo4jGraph, and
it works fine. Thanks everybody!
I have tested it with locally built snapshots as neo4j-gremlin isn't
part of M9-incubating. Will there be a separate release of tinkerpop3
that includes neo4j-gremlin, so that I can share this in my examples
project?