Re: Code Reviews - for committers

[Robert Dale <ro...@gmail.com>]

One day I'll figure out why docker always fails for me.  It's usually
around some java.lang.OutOfMemoryError: unable to create new native
thread.  Usually happens in tinkergraph-gremlin. Sometimes it makes it to
spark or hadoop, but inevitably fails miserably there.  Tried latest master
and TINKERPOP-1434 but no joy. Even rebuilt docker containers from scratch.

Robert Dale

On Fri, Oct 28, 2016 at 6:22 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> That's probably fair. Perhaps that's a good rule. At least one person
> should use CPU to do the docker build and mention that on the PR as part of
> their VOTE. I also suppose some PRs won't even need the docker run at all.
> We can just use some common sense with that "rule".
>
> On Fri, Oct 28, 2016 at 5:29 PM, Daniel Kuppitz <me...@gremlin.guru> wrote:
>
> > Talking about sparing CPU cycles: docker/build.sh -t -i -n takes time,
> lots
> > of time. I used to run it regardless of whether somebody else already did
> > it or not. I won't do that anymore and instead rely on idempotent results
> > (in the end that's why we originally created the Docker containers).
> Hence,
> > if anybody already ran the full build, I will only look through the code
> > changes and if they look good to me - VOTE: +1.
> >
> > Cheers,
> > Daniel
> >
> >
> > On Fri, Oct 28, 2016 at 9:28 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> > > We're backing up pretty heavily on pull request reviews/votes.
> > >
> > > https://github.com/apache/tinkerpop/pulls
> > >
> > > Can any committers help us along with some reviews/votes?
> > >
> > > Remember - you don't need to have full knowledge of the code to
> > participate
> > > in a code review. If the code comes from a core contributor (e.g. marko
> > > submitting a gremlin-spark bug fix), you mostly need to focus on the
> big
> > > picture and generalities of the change and then do a "mvn clean
> install"
> > or
> > > better yet "docker.build.sh -t -i -n". If it all works then "VOTE +1".
> > >
> > > Most pull requests for review are super simple, like this one:
> > >
> > > https://github.com/apache/tinkerpop/pull/465 (removed deprecated
> > classes)
> > >
> > > All you need to do is check if the classes are gone, if upgrade docs
> are
> > > updated and if the thing builds...done. VOTE +1.
> > >
> > > Doing reviews is a huge help to the project (even for those who are not
> > > committers and don't have binding votes, as it is a great way to get
> > > visibility in the project and learn more about how it works). It would
> be
> > > great if folks could make a habit of sparing a few CPU cycles for
> > TinkerPop
> > > at the end of a work day or while you sleep so that we can continue to
> > stay
> > > agile in our development.
> > >
> > > Thanks
> > >
> >
>

Re: Code Reviews - for committers

[Marko Rodriguez <ok...@gmail.com>]

Hello,

> Building all the way through now. It was IPv6 that bit me again but for
> hadoop. I have to disable IPv6 at the kernel level. I'll submit a patch for
> this and some other things later.

Huh. K.

> Why do the tests take 3 hours to go through giraph-gremlin? How can I speed
> that up?

If you can figure that out, you would be a god among men.

Marko.



> 
> Robert Dale
> 
> On Mon, Nov 7, 2016 at 12:18 PM, Robert Dale <ro...@gmail.com> wrote:
> 
>> There's some selinux/capability issue with docker 1.10. I'm not sure if
>> it's a bug in 1.10 or just missing some capability. There are some related
>> issues fixed in docker 1.12. Selinux doesn't complain.  However, turning
>> selinux off or to permissive gets around the issue.
>> 
>> I managed to get through the neo4j tests but docker 1.10 was terribly slow
>> so I ended up upgrading to docker 1.12 which is about 3-4x faster but still
>> slower than native.  I'll let it run tonight and see if it gets all the way
>> through.
>> 
>> Robert Dale
>> 
>> On Sun, Nov 6, 2016 at 6:37 PM, Stephen Mallette <sp...@gmail.com>
>> wrote:
>> 
>>> That stinks. Never got OOME with docker. Any other kinds of issues you can
>>> recall or is it pretty much just that?
>>> 
>>> On Nov 6, 2016 5:43 PM, "Robert Dale" <ro...@gmail.com> wrote:
>>> 
>>>> One day I'll figure out why docker always fails for me.  It's usually
>>>> around some java.lang.OutOfMemoryError: unable to create new native
>>>> thread.  Usually happens in tinkergraph-gremlin. Sometimes it makes it
>>> to
>>>> spark or hadoop, but inevitably fails miserably there.  Tried latest
>>> master
>>>> and TINKERPOP-1434 but no joy. Even rebuilt docker containers from
>>> scratch.
>>>> 
>>>> Robert Dale
>>>> 
>>>> On Fri, Oct 28, 2016 at 6:22 PM, Stephen Mallette <spmallette@gmail.com
>>>> 
>>>> wrote:
>>>> 
>>>>> That's probably fair. Perhaps that's a good rule. At least one person
>>>>> should use CPU to do the docker build and mention that on the PR as
>>> part
>>>> of
>>>>> their VOTE. I also suppose some PRs won't even need the docker run at
>>>> all.
>>>>> We can just use some common sense with that "rule".
>>>>> 
>>>>> On Fri, Oct 28, 2016 at 5:29 PM, Daniel Kuppitz <me...@gremlin.guru>
>>> wrote:
>>>>> 
>>>>>> Talking about sparing CPU cycles: docker/build.sh -t -i -n takes
>>> time,
>>>>> lots
>>>>>> of time. I used to run it regardless of whether somebody else
>>> already
>>>> did
>>>>>> it or not. I won't do that anymore and instead rely on idempotent
>>>> results
>>>>>> (in the end that's why we originally created the Docker containers).
>>>>> Hence,
>>>>>> if anybody already ran the full build, I will only look through the
>>>> code
>>>>>> changes and if they look good to me - VOTE: +1.
>>>>>> 
>>>>>> Cheers,
>>>>>> Daniel
>>>>>> 
>>>>>> 
>>>>>> On Fri, Oct 28, 2016 at 9:28 PM, Stephen Mallette <
>>>> spmallette@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> We're backing up pretty heavily on pull request reviews/votes.
>>>>>>> 
>>>>>>> https://github.com/apache/tinkerpop/pulls
>>>>>>> 
>>>>>>> Can any committers help us along with some reviews/votes?
>>>>>>> 
>>>>>>> Remember - you don't need to have full knowledge of the code to
>>>>>> participate
>>>>>>> in a code review. If the code comes from a core contributor (e.g.
>>>> marko
>>>>>>> submitting a gremlin-spark bug fix), you mostly need to focus on
>>> the
>>>>> big
>>>>>>> picture and generalities of the change and then do a "mvn clean
>>>>> install"
>>>>>> or
>>>>>>> better yet "docker.build.sh -t -i -n". If it all works then "VOTE
>>>> +1".
>>>>>>> 
>>>>>>> Most pull requests for review are super simple, like this one:
>>>>>>> 
>>>>>>> https://github.com/apache/tinkerpop/pull/465 (removed deprecated
>>>>>> classes)
>>>>>>> 
>>>>>>> All you need to do is check if the classes are gone, if upgrade
>>> docs
>>>>> are
>>>>>>> updated and if the thing builds...done. VOTE +1.
>>>>>>> 
>>>>>>> Doing reviews is a huge help to the project (even for those who
>>> are
>>>> not
>>>>>>> committers and don't have binding votes, as it is a great way to
>>> get
>>>>>>> visibility in the project and learn more about how it works). It
>>>> would
>>>>> be
>>>>>>> great if folks could make a habit of sparing a few CPU cycles for
>>>>>> TinkerPop
>>>>>>> at the end of a work day or while you sleep so that we can
>>> continue
>>>> to
>>>>>> stay
>>>>>>> agile in our development.
>>>>>>> 
>>>>>>> Thanks
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>> 
>> 
>> --
>> null
>> 

Re: Code Reviews - for committers

[Robert Dale <ro...@gmail.com>]

Building all the way through now. It was IPv6 that bit me again but for
hadoop. I have to disable IPv6 at the kernel level. I'll submit a patch for
this and some other things later.

Why do the tests take 3 hours to go through giraph-gremlin? How can I speed
that up?

Robert Dale

On Mon, Nov 7, 2016 at 12:18 PM, Robert Dale <ro...@gmail.com> wrote:

> There's some selinux/capability issue with docker 1.10. I'm not sure if
> it's a bug in 1.10 or just missing some capability. There are some related
> issues fixed in docker 1.12. Selinux doesn't complain.  However, turning
> selinux off or to permissive gets around the issue.
>
> I managed to get through the neo4j tests but docker 1.10 was terribly slow
> so I ended up upgrading to docker 1.12 which is about 3-4x faster but still
> slower than native.  I'll let it run tonight and see if it gets all the way
> through.
>
> Robert Dale
>
> On Sun, Nov 6, 2016 at 6:37 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
>> That stinks. Never got OOME with docker. Any other kinds of issues you can
>> recall or is it pretty much just that?
>>
>> On Nov 6, 2016 5:43 PM, "Robert Dale" <ro...@gmail.com> wrote:
>>
>> > One day I'll figure out why docker always fails for me.  It's usually
>> > around some java.lang.OutOfMemoryError: unable to create new native
>> > thread.  Usually happens in tinkergraph-gremlin. Sometimes it makes it
>> to
>> > spark or hadoop, but inevitably fails miserably there.  Tried latest
>> master
>> > and TINKERPOP-1434 but no joy. Even rebuilt docker containers from
>> scratch.
>> >
>> > Robert Dale
>> >
>> > On Fri, Oct 28, 2016 at 6:22 PM, Stephen Mallette <spmallette@gmail.com
>> >
>> > wrote:
>> >
>> > > That's probably fair. Perhaps that's a good rule. At least one person
>> > > should use CPU to do the docker build and mention that on the PR as
>> part
>> > of
>> > > their VOTE. I also suppose some PRs won't even need the docker run at
>> > all.
>> > > We can just use some common sense with that "rule".
>> > >
>> > > On Fri, Oct 28, 2016 at 5:29 PM, Daniel Kuppitz <me...@gremlin.guru>
>> wrote:
>> > >
>> > > > Talking about sparing CPU cycles: docker/build.sh -t -i -n takes
>> time,
>> > > lots
>> > > > of time. I used to run it regardless of whether somebody else
>> already
>> > did
>> > > > it or not. I won't do that anymore and instead rely on idempotent
>> > results
>> > > > (in the end that's why we originally created the Docker containers).
>> > > Hence,
>> > > > if anybody already ran the full build, I will only look through the
>> > code
>> > > > changes and if they look good to me - VOTE: +1.
>> > > >
>> > > > Cheers,
>> > > > Daniel
>> > > >
>> > > >
>> > > > On Fri, Oct 28, 2016 at 9:28 PM, Stephen Mallette <
>> > spmallette@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > We're backing up pretty heavily on pull request reviews/votes.
>> > > > >
>> > > > > https://github.com/apache/tinkerpop/pulls
>> > > > >
>> > > > > Can any committers help us along with some reviews/votes?
>> > > > >
>> > > > > Remember - you don't need to have full knowledge of the code to
>> > > > participate
>> > > > > in a code review. If the code comes from a core contributor (e.g.
>> > marko
>> > > > > submitting a gremlin-spark bug fix), you mostly need to focus on
>> the
>> > > big
>> > > > > picture and generalities of the change and then do a "mvn clean
>> > > install"
>> > > > or
>> > > > > better yet "docker.build.sh -t -i -n". If it all works then "VOTE
>> > +1".
>> > > > >
>> > > > > Most pull requests for review are super simple, like this one:
>> > > > >
>> > > > > https://github.com/apache/tinkerpop/pull/465 (removed deprecated
>> > > > classes)
>> > > > >
>> > > > > All you need to do is check if the classes are gone, if upgrade
>> docs
>> > > are
>> > > > > updated and if the thing builds...done. VOTE +1.
>> > > > >
>> > > > > Doing reviews is a huge help to the project (even for those who
>> are
>> > not
>> > > > > committers and don't have binding votes, as it is a great way to
>> get
>> > > > > visibility in the project and learn more about how it works). It
>> > would
>> > > be
>> > > > > great if folks could make a habit of sparing a few CPU cycles for
>> > > > TinkerPop
>> > > > > at the end of a work day or while you sleep so that we can
>> continue
>> > to
>> > > > stay
>> > > > > agile in our development.
>> > > > >
>> > > > > Thanks
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
>
> --
> null
>

Code Reviews - for committers

[Robert Dale <ro...@gmail.com>]

There's some selinux/capability issue with docker 1.10. I'm not sure if
it's a bug in 1.10 or just missing some capability. There are some related
issues fixed in docker 1.12. Selinux doesn't complain.  However, turning
selinux off or to permissive gets around the issue.

I managed to get through the neo4j tests but docker 1.10 was terribly slow
so I ended up upgrading to docker 1.12 which is about 3-4x faster but still
slower than native.  I'll let it run tonight and see if it gets all the way
through.

Robert Dale

On Sun, Nov 6, 2016 at 6:37 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> That stinks. Never got OOME with docker. Any other kinds of issues you can
> recall or is it pretty much just that?
>
> On Nov 6, 2016 5:43 PM, "Robert Dale" <ro...@gmail.com> wrote:
>
> > One day I'll figure out why docker always fails for me.  It's usually
> > around some java.lang.OutOfMemoryError: unable to create new native
> > thread.  Usually happens in tinkergraph-gremlin. Sometimes it makes it to
> > spark or hadoop, but inevitably fails miserably there.  Tried latest
> master
> > and TINKERPOP-1434 but no joy. Even rebuilt docker containers from
> scratch.
> >
> > Robert Dale
> >
> > On Fri, Oct 28, 2016 at 6:22 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> > > That's probably fair. Perhaps that's a good rule. At least one person
> > > should use CPU to do the docker build and mention that on the PR as
> part
> > of
> > > their VOTE. I also suppose some PRs won't even need the docker run at
> > all.
> > > We can just use some common sense with that "rule".
> > >
> > > On Fri, Oct 28, 2016 at 5:29 PM, Daniel Kuppitz <me...@gremlin.guru>
> wrote:
> > >
> > > > Talking about sparing CPU cycles: docker/build.sh -t -i -n takes
> time,
> > > lots
> > > > of time. I used to run it regardless of whether somebody else already
> > did
> > > > it or not. I won't do that anymore and instead rely on idempotent
> > results
> > > > (in the end that's why we originally created the Docker containers).
> > > Hence,
> > > > if anybody already ran the full build, I will only look through the
> > code
> > > > changes and if they look good to me - VOTE: +1.
> > > >
> > > > Cheers,
> > > > Daniel
> > > >
> > > >
> > > > On Fri, Oct 28, 2016 at 9:28 PM, Stephen Mallette <
> > spmallette@gmail.com>
> > > > wrote:
> > > >
> > > > > We're backing up pretty heavily on pull request reviews/votes.
> > > > >
> > > > > https://github.com/apache/tinkerpop/pulls
> > > > >
> > > > > Can any committers help us along with some reviews/votes?
> > > > >
> > > > > Remember - you don't need to have full knowledge of the code to
> > > > participate
> > > > > in a code review. If the code comes from a core contributor (e.g.
> > marko
> > > > > submitting a gremlin-spark bug fix), you mostly need to focus on
> the
> > > big
> > > > > picture and generalities of the change and then do a "mvn clean
> > > install"
> > > > or
> > > > > better yet "docker.build.sh -t -i -n". If it all works then "VOTE
> > +1".
> > > > >
> > > > > Most pull requests for review are super simple, like this one:
> > > > >
> > > > > https://github.com/apache/tinkerpop/pull/465 (removed deprecated
> > > > classes)
> > > > >
> > > > > All you need to do is check if the classes are gone, if upgrade
> docs
> > > are
> > > > > updated and if the thing builds...done. VOTE +1.
> > > > >
> > > > > Doing reviews is a huge help to the project (even for those who are
> > not
> > > > > committers and don't have binding votes, as it is a great way to
> get
> > > > > visibility in the project and learn more about how it works). It
> > would
> > > be
> > > > > great if folks could make a habit of sparing a few CPU cycles for
> > > > TinkerPop
> > > > > at the end of a work day or while you sleep so that we can continue
> > to
> > > > stay
> > > > > agile in our development.
> > > > >
> > > > > Thanks
> > > > >
> > > >
> > >
> >
>



-- 
null

Re: Code Reviews - for committers

[Jason Plurad <pl...@gmail.com>]

For Docker, I was running into the OOME, but then I bumped up the
container/virtualbox memory to 4 GB (default is 2 GB) and it works fine.

-- Jason

On Sun, Nov 6, 2016 at 6:37 PM, Stephen Mallette <sp...@gmail.com>
wrote:

> That stinks. Never got OOME with docker. Any other kinds of issues you can
> recall or is it pretty much just that?
>
> On Nov 6, 2016 5:43 PM, "Robert Dale" <ro...@gmail.com> wrote:
>
> > One day I'll figure out why docker always fails for me.  It's usually
> > around some java.lang.OutOfMemoryError: unable to create new native
> > thread.  Usually happens in tinkergraph-gremlin. Sometimes it makes it to
> > spark or hadoop, but inevitably fails miserably there.  Tried latest
> master
> > and TINKERPOP-1434 but no joy. Even rebuilt docker containers from
> scratch.
> >
> > Robert Dale
> >
> > On Fri, Oct 28, 2016 at 6:22 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> > > That's probably fair. Perhaps that's a good rule. At least one person
> > > should use CPU to do the docker build and mention that on the PR as
> part
> > of
> > > their VOTE. I also suppose some PRs won't even need the docker run at
> > all.
> > > We can just use some common sense with that "rule".
> > >
> > > On Fri, Oct 28, 2016 at 5:29 PM, Daniel Kuppitz <me...@gremlin.guru>
> wrote:
> > >
> > > > Talking about sparing CPU cycles: docker/build.sh -t -i -n takes
> time,
> > > lots
> > > > of time. I used to run it regardless of whether somebody else already
> > did
> > > > it or not. I won't do that anymore and instead rely on idempotent
> > results
> > > > (in the end that's why we originally created the Docker containers).
> > > Hence,
> > > > if anybody already ran the full build, I will only look through the
> > code
> > > > changes and if they look good to me - VOTE: +1.
> > > >
> > > > Cheers,
> > > > Daniel
> > > >
> > > >
> > > > On Fri, Oct 28, 2016 at 9:28 PM, Stephen Mallette <
> > spmallette@gmail.com>
> > > > wrote:
> > > >
> > > > > We're backing up pretty heavily on pull request reviews/votes.
> > > > >
> > > > > https://github.com/apache/tinkerpop/pulls
> > > > >
> > > > > Can any committers help us along with some reviews/votes?
> > > > >
> > > > > Remember - you don't need to have full knowledge of the code to
> > > > participate
> > > > > in a code review. If the code comes from a core contributor (e.g.
> > marko
> > > > > submitting a gremlin-spark bug fix), you mostly need to focus on
> the
> > > big
> > > > > picture and generalities of the change and then do a "mvn clean
> > > install"
> > > > or
> > > > > better yet "docker.build.sh -t -i -n". If it all works then "VOTE
> > +1".
> > > > >
> > > > > Most pull requests for review are super simple, like this one:
> > > > >
> > > > > https://github.com/apache/tinkerpop/pull/465 (removed deprecated
> > > > classes)
> > > > >
> > > > > All you need to do is check if the classes are gone, if upgrade
> docs
> > > are
> > > > > updated and if the thing builds...done. VOTE +1.
> > > > >
> > > > > Doing reviews is a huge help to the project (even for those who are
> > not
> > > > > committers and don't have binding votes, as it is a great way to
> get
> > > > > visibility in the project and learn more about how it works). It
> > would
> > > be
> > > > > great if folks could make a habit of sparing a few CPU cycles for
> > > > TinkerPop
> > > > > at the end of a work day or while you sleep so that we can continue
> > to
> > > > stay
> > > > > agile in our development.
> > > > >
> > > > > Thanks
> > > > >
> > > >
> > >
> >
>

Re: Code Reviews - for committers

[Stephen Mallette <sp...@gmail.com>]

That stinks. Never got OOME with docker. Any other kinds of issues you can
recall or is it pretty much just that?

On Nov 6, 2016 5:43 PM, "Robert Dale" <ro...@gmail.com> wrote:

> One day I'll figure out why docker always fails for me.  It's usually
> around some java.lang.OutOfMemoryError: unable to create new native
> thread.  Usually happens in tinkergraph-gremlin. Sometimes it makes it to
> spark or hadoop, but inevitably fails miserably there.  Tried latest master
> and TINKERPOP-1434 but no joy. Even rebuilt docker containers from scratch.
>
> Robert Dale
>
> On Fri, Oct 28, 2016 at 6:22 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
> > That's probably fair. Perhaps that's a good rule. At least one person
> > should use CPU to do the docker build and mention that on the PR as part
> of
> > their VOTE. I also suppose some PRs won't even need the docker run at
> all.
> > We can just use some common sense with that "rule".
> >
> > On Fri, Oct 28, 2016 at 5:29 PM, Daniel Kuppitz <me...@gremlin.guru> wrote:
> >
> > > Talking about sparing CPU cycles: docker/build.sh -t -i -n takes time,
> > lots
> > > of time. I used to run it regardless of whether somebody else already
> did
> > > it or not. I won't do that anymore and instead rely on idempotent
> results
> > > (in the end that's why we originally created the Docker containers).
> > Hence,
> > > if anybody already ran the full build, I will only look through the
> code
> > > changes and if they look good to me - VOTE: +1.
> > >
> > > Cheers,
> > > Daniel
> > >
> > >
> > > On Fri, Oct 28, 2016 at 9:28 PM, Stephen Mallette <
> spmallette@gmail.com>
> > > wrote:
> > >
> > > > We're backing up pretty heavily on pull request reviews/votes.
> > > >
> > > > https://github.com/apache/tinkerpop/pulls
> > > >
> > > > Can any committers help us along with some reviews/votes?
> > > >
> > > > Remember - you don't need to have full knowledge of the code to
> > > participate
> > > > in a code review. If the code comes from a core contributor (e.g.
> marko
> > > > submitting a gremlin-spark bug fix), you mostly need to focus on the
> > big
> > > > picture and generalities of the change and then do a "mvn clean
> > install"
> > > or
> > > > better yet "docker.build.sh -t -i -n". If it all works then "VOTE
> +1".
> > > >
> > > > Most pull requests for review are super simple, like this one:
> > > >
> > > > https://github.com/apache/tinkerpop/pull/465 (removed deprecated
> > > classes)
> > > >
> > > > All you need to do is check if the classes are gone, if upgrade docs
> > are
> > > > updated and if the thing builds...done. VOTE +1.
> > > >
> > > > Doing reviews is a huge help to the project (even for those who are
> not
> > > > committers and don't have binding votes, as it is a great way to get
> > > > visibility in the project and learn more about how it works). It
> would
> > be
> > > > great if folks could make a habit of sparing a few CPU cycles for
> > > TinkerPop
> > > > at the end of a work day or while you sleep so that we can continue
> to
> > > stay
> > > > agile in our development.
> > > >
> > > > Thanks
> > > >
> > >
> >
>