You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by Stephen Mallette <sp...@gmail.com> on 2016/08/27 10:05:58 UTC
Building gremlin-python
Now that gremlin-python has been merged to master, you might wonder about
what this has done to our build/release process. Well, not very much. If
you do your standard
mvn clean install
on master right now you should see that everything builds and is good to go
- even gremlin-python. So, everything is good, right? right? well, yes and
no.
The "yes" aspect here is that the entire project still builds with maven
which keeps our build toolchain simple. Users can just have java installed
as they always did and still get a clean build of TinkerPop. The "no"
aspect is that native python tests (and if you were deploy,
packaging/deployment tasks) did not execute.
What's good however is that even the native python build tasks are still
just part of the maven toolchain. You just need to have python 2.x
installed and, if you do, build with:
mvn clean install -DglvPython
You will now see in your output the results of native pytest execution. I
think this approach almost sets the basic pattern for future GLVs. I'd
prefer to not have -glvPython to some degree and simply detect python on
the system and then execute natively if it can, but then i think about what
happens as we add more GLVs and then i sorta like the idea of having the
specific option to turn things on and off.
Re: Building gremlin-python
Posted by David Brown <da...@gmail.com>.
Ah ok, well I had always used Aptitude `apt-get install
python-virtualenv` on Ubuntu or Homebrew on Mac `brew install
pyenv-virtualenv` although I guess the docs now suggest using pip. Not
really a big deal either way.
On Tue, Aug 30, 2016 at 6:30 PM, Stephen Mallette <sp...@gmail.com> wrote:
> I guess i say pip is required because you use pip to install virtualenv.....
>
> On Tue, Aug 30, 2016 at 6:12 PM, David Brown <da...@gmail.com> wrote:
>
>> Aha, that is in the virtualenv too...
>>
>> On Tue, Aug 30, 2016 at 6:06 PM, David Brown <da...@gmail.com> wrote:
>> > Ah wait, I guess you use it to install twine.
>> >
>> > On Tue, Aug 30, 2016 at 5:58 PM, David Brown <da...@gmail.com>
>> wrote:
>> >> Nice work with this Stephen! One thing: if I am thinking about this
>> >> correctly, pip isn't a requirement for building. I think we are only
>> >> using pip inside the virtualenv, and since the virtualenv binary
>> >> installs pip and setuptools by default when in creates the
>> >> environment, I don't think we need it to be installed on the system.
>> >> So basically, if you have Python(2) on on your machine, you only need
>> >> to make sure that you install virtualenv before building...make sense?
>> >> or am I missing something?
>> >>
>> >> On Tue, Aug 30, 2016 at 5:32 PM, Stephen Mallette <sp...@gmail.com>
>> wrote:
>> >>> With some help from Dave I've tweaked up the build for python some
>> more -
>> >>> the command to build is still:
>> >>>
>> >>> mvn clean install -pl gremlin-python -DglvPython
>> >>>
>> >>> but now:
>> >>>
>> >>> + The prereqs for building are just python 2.7, pip and virtualenv
>> >>> + The build uses virtualenv to isolate the python environment to the
>> >>> /target directories
>> >>> + -DskipTests is now better respected by Gremlin Server - won't bother
>> to
>> >>> start it, if tests are skipped
>> >>> + Minor adjustments to the deploy phase for python. We can't really
>> deploy
>> >>> to pypi on deploy in our release process because pypi has no staging
>> >>> environment (like sonatype does) so the release would be immediately
>> public
>> >>> prior to vote on the release. So i made a special "-Dpypi" that
>> activates a
>> >>> profile that will hook into "mvn deploy" and push the artifacts to pypi
>> >>> with twine.
>> >>>
>> >>> Note that these changes are all in the "virtualenv" branch at the
>> moment. I
>> >>> suspect I will merge it to master tomorrow after some more tests.
>> >>>
>> >>>
>> >>>
>> >>> On Mon, Aug 29, 2016 at 5:45 PM, Dylan Millikin <
>> dylan.millikin@gmail.com>
>> >>> wrote:
>> >>>
>> >>>> Yeah this is great. Will be useful for all GLVs.
>> >>>>
>> >>>> On Mon, Aug 29, 2016 at 6:13 PM, David Brown <da...@gmail.com>
>> wrote:
>> >>>>
>> >>>> > Wow Stephen thanks for all your hard work! This will really make
>> >>>> > driver development a lot easier.
>> >>>> >
>> >>>> > On Sun, Aug 28, 2016 at 6:48 PM, Stephen Mallette <
>> spmallette@gmail.com>
>> >>>> > wrote:
>> >>>> > > Took me half of my Sunday, but I just got Gremlin Server to start
>> up
>> >>>> and
>> >>>> > > shutdown in the standard maven integration-test phase of
>> >>>> gremlin-python.
>> >>>> > > That means we can now write native python tests that round-trip to
>> >>>> > Gremlin
>> >>>> > > Server in addition to the regular unit tests with pytest we
>> already
>> >>>> > have! I
>> >>>> > > actual Note that I actually start two gremlin servers one with
>> auth on
>> >>>> > and
>> >>>> > > one without (ports 8183/8182 respectively) so that we can test
>> >>>> > > authentication features. Further note that native tests are bound
>> to
>> >>>> the
>> >>>> > > integration-test phase of maven and run automatically on
>> -glvPython.
>> >>>> > that's
>> >>>> > > a bit different from our other projects where integration test
>> phases
>> >>>> > only
>> >>>> > > run if you -DskipIntegrationTests=false. gremlin-python won't
>> obey that
>> >>>> > > setting.
>> >>>> > >
>> >>>> > >
>> >>>> > >
>> >>>> > > On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <
>> >>>> spmallette@gmail.com>
>> >>>> > > wrote:
>> >>>> > >
>> >>>> > >> Now that gremlin-python has been merged to master, you might
>> wonder
>> >>>> > about
>> >>>> > >> what this has done to our build/release process. Well, not very
>> much.
>> >>>> If
>> >>>> > >> you do your standard
>> >>>> > >>
>> >>>> > >> mvn clean install
>> >>>> > >>
>> >>>> > >> on master right now you should see that everything builds and is
>> good
>> >>>> to
>> >>>> > >> go - even gremlin-python. So, everything is good, right? right?
>> well,
>> >>>> > yes
>> >>>> > >> and no.
>> >>>> > >>
>> >>>> > >> The "yes" aspect here is that the entire project still builds
>> with
>> >>>> maven
>> >>>> > >> which keeps our build toolchain simple. Users can just have java
>> >>>> > installed
>> >>>> > >> as they always did and still get a clean build of TinkerPop. The
>> "no"
>> >>>> > >> aspect is that native python tests (and if you were deploy,
>> >>>> > >> packaging/deployment tasks) did not execute.
>> >>>> > >>
>> >>>> > >> What's good however is that even the native python build tasks
>> are
>> >>>> still
>> >>>> > >> just part of the maven toolchain. You just need to have python
>> 2.x
>> >>>> > >> installed and, if you do, build with:
>> >>>> > >>
>> >>>> > >> mvn clean install -DglvPython
>> >>>> > >>
>> >>>> > >> You will now see in your output the results of native pytest
>> >>>> execution.
>> >>>> > I
>> >>>> > >> think this approach almost sets the basic pattern for future
>> GLVs. I'd
>> >>>> > >> prefer to not have -glvPython to some degree and simply detect
>> python
>> >>>> on
>> >>>> > >> the system and then execute natively if it can, but then i think
>> about
>> >>>> > what
>> >>>> > >> happens as we add more GLVs and then i sorta like the idea of
>> having
>> >>>> the
>> >>>> > >> specific option to turn things on and off.
>> >>>> > >>
>> >>>> > >>
>> >>>> > >>
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > David M. Brown
>> >>>> > R.A. CulturePlex Lab, Western University
>> >>>> >
>> >>>>
>> >>
>> >>
>> >>
>> >> --
>> >> David M. Brown
>> >> R.A. CulturePlex Lab, Western University
>> >
>> >
>> >
>> > --
>> > David M. Brown
>> > R.A. CulturePlex Lab, Western University
>>
>>
>>
>> --
>> David M. Brown
>> R.A. CulturePlex Lab, Western University
>>
--
David M. Brown
R.A. CulturePlex Lab, Western University
Re: Building gremlin-python
Posted by Stephen Mallette <sp...@gmail.com>.
I guess i say pip is required because you use pip to install virtualenv.....
On Tue, Aug 30, 2016 at 6:12 PM, David Brown <da...@gmail.com> wrote:
> Aha, that is in the virtualenv too...
>
> On Tue, Aug 30, 2016 at 6:06 PM, David Brown <da...@gmail.com> wrote:
> > Ah wait, I guess you use it to install twine.
> >
> > On Tue, Aug 30, 2016 at 5:58 PM, David Brown <da...@gmail.com>
> wrote:
> >> Nice work with this Stephen! One thing: if I am thinking about this
> >> correctly, pip isn't a requirement for building. I think we are only
> >> using pip inside the virtualenv, and since the virtualenv binary
> >> installs pip and setuptools by default when in creates the
> >> environment, I don't think we need it to be installed on the system.
> >> So basically, if you have Python(2) on on your machine, you only need
> >> to make sure that you install virtualenv before building...make sense?
> >> or am I missing something?
> >>
> >> On Tue, Aug 30, 2016 at 5:32 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
> >>> With some help from Dave I've tweaked up the build for python some
> more -
> >>> the command to build is still:
> >>>
> >>> mvn clean install -pl gremlin-python -DglvPython
> >>>
> >>> but now:
> >>>
> >>> + The prereqs for building are just python 2.7, pip and virtualenv
> >>> + The build uses virtualenv to isolate the python environment to the
> >>> /target directories
> >>> + -DskipTests is now better respected by Gremlin Server - won't bother
> to
> >>> start it, if tests are skipped
> >>> + Minor adjustments to the deploy phase for python. We can't really
> deploy
> >>> to pypi on deploy in our release process because pypi has no staging
> >>> environment (like sonatype does) so the release would be immediately
> public
> >>> prior to vote on the release. So i made a special "-Dpypi" that
> activates a
> >>> profile that will hook into "mvn deploy" and push the artifacts to pypi
> >>> with twine.
> >>>
> >>> Note that these changes are all in the "virtualenv" branch at the
> moment. I
> >>> suspect I will merge it to master tomorrow after some more tests.
> >>>
> >>>
> >>>
> >>> On Mon, Aug 29, 2016 at 5:45 PM, Dylan Millikin <
> dylan.millikin@gmail.com>
> >>> wrote:
> >>>
> >>>> Yeah this is great. Will be useful for all GLVs.
> >>>>
> >>>> On Mon, Aug 29, 2016 at 6:13 PM, David Brown <da...@gmail.com>
> wrote:
> >>>>
> >>>> > Wow Stephen thanks for all your hard work! This will really make
> >>>> > driver development a lot easier.
> >>>> >
> >>>> > On Sun, Aug 28, 2016 at 6:48 PM, Stephen Mallette <
> spmallette@gmail.com>
> >>>> > wrote:
> >>>> > > Took me half of my Sunday, but I just got Gremlin Server to start
> up
> >>>> and
> >>>> > > shutdown in the standard maven integration-test phase of
> >>>> gremlin-python.
> >>>> > > That means we can now write native python tests that round-trip to
> >>>> > Gremlin
> >>>> > > Server in addition to the regular unit tests with pytest we
> already
> >>>> > have! I
> >>>> > > actual Note that I actually start two gremlin servers one with
> auth on
> >>>> > and
> >>>> > > one without (ports 8183/8182 respectively) so that we can test
> >>>> > > authentication features. Further note that native tests are bound
> to
> >>>> the
> >>>> > > integration-test phase of maven and run automatically on
> -glvPython.
> >>>> > that's
> >>>> > > a bit different from our other projects where integration test
> phases
> >>>> > only
> >>>> > > run if you -DskipIntegrationTests=false. gremlin-python won't
> obey that
> >>>> > > setting.
> >>>> > >
> >>>> > >
> >>>> > >
> >>>> > > On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <
> >>>> spmallette@gmail.com>
> >>>> > > wrote:
> >>>> > >
> >>>> > >> Now that gremlin-python has been merged to master, you might
> wonder
> >>>> > about
> >>>> > >> what this has done to our build/release process. Well, not very
> much.
> >>>> If
> >>>> > >> you do your standard
> >>>> > >>
> >>>> > >> mvn clean install
> >>>> > >>
> >>>> > >> on master right now you should see that everything builds and is
> good
> >>>> to
> >>>> > >> go - even gremlin-python. So, everything is good, right? right?
> well,
> >>>> > yes
> >>>> > >> and no.
> >>>> > >>
> >>>> > >> The "yes" aspect here is that the entire project still builds
> with
> >>>> maven
> >>>> > >> which keeps our build toolchain simple. Users can just have java
> >>>> > installed
> >>>> > >> as they always did and still get a clean build of TinkerPop. The
> "no"
> >>>> > >> aspect is that native python tests (and if you were deploy,
> >>>> > >> packaging/deployment tasks) did not execute.
> >>>> > >>
> >>>> > >> What's good however is that even the native python build tasks
> are
> >>>> still
> >>>> > >> just part of the maven toolchain. You just need to have python
> 2.x
> >>>> > >> installed and, if you do, build with:
> >>>> > >>
> >>>> > >> mvn clean install -DglvPython
> >>>> > >>
> >>>> > >> You will now see in your output the results of native pytest
> >>>> execution.
> >>>> > I
> >>>> > >> think this approach almost sets the basic pattern for future
> GLVs. I'd
> >>>> > >> prefer to not have -glvPython to some degree and simply detect
> python
> >>>> on
> >>>> > >> the system and then execute natively if it can, but then i think
> about
> >>>> > what
> >>>> > >> happens as we add more GLVs and then i sorta like the idea of
> having
> >>>> the
> >>>> > >> specific option to turn things on and off.
> >>>> > >>
> >>>> > >>
> >>>> > >>
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > David M. Brown
> >>>> > R.A. CulturePlex Lab, Western University
> >>>> >
> >>>>
> >>
> >>
> >>
> >> --
> >> David M. Brown
> >> R.A. CulturePlex Lab, Western University
> >
> >
> >
> > --
> > David M. Brown
> > R.A. CulturePlex Lab, Western University
>
>
>
> --
> David M. Brown
> R.A. CulturePlex Lab, Western University
>
Re: Building gremlin-python
Posted by David Brown <da...@gmail.com>.
Aha, that is in the virtualenv too...
On Tue, Aug 30, 2016 at 6:06 PM, David Brown <da...@gmail.com> wrote:
> Ah wait, I guess you use it to install twine.
>
> On Tue, Aug 30, 2016 at 5:58 PM, David Brown <da...@gmail.com> wrote:
>> Nice work with this Stephen! One thing: if I am thinking about this
>> correctly, pip isn't a requirement for building. I think we are only
>> using pip inside the virtualenv, and since the virtualenv binary
>> installs pip and setuptools by default when in creates the
>> environment, I don't think we need it to be installed on the system.
>> So basically, if you have Python(2) on on your machine, you only need
>> to make sure that you install virtualenv before building...make sense?
>> or am I missing something?
>>
>> On Tue, Aug 30, 2016 at 5:32 PM, Stephen Mallette <sp...@gmail.com> wrote:
>>> With some help from Dave I've tweaked up the build for python some more -
>>> the command to build is still:
>>>
>>> mvn clean install -pl gremlin-python -DglvPython
>>>
>>> but now:
>>>
>>> + The prereqs for building are just python 2.7, pip and virtualenv
>>> + The build uses virtualenv to isolate the python environment to the
>>> /target directories
>>> + -DskipTests is now better respected by Gremlin Server - won't bother to
>>> start it, if tests are skipped
>>> + Minor adjustments to the deploy phase for python. We can't really deploy
>>> to pypi on deploy in our release process because pypi has no staging
>>> environment (like sonatype does) so the release would be immediately public
>>> prior to vote on the release. So i made a special "-Dpypi" that activates a
>>> profile that will hook into "mvn deploy" and push the artifacts to pypi
>>> with twine.
>>>
>>> Note that these changes are all in the "virtualenv" branch at the moment. I
>>> suspect I will merge it to master tomorrow after some more tests.
>>>
>>>
>>>
>>> On Mon, Aug 29, 2016 at 5:45 PM, Dylan Millikin <dy...@gmail.com>
>>> wrote:
>>>
>>>> Yeah this is great. Will be useful for all GLVs.
>>>>
>>>> On Mon, Aug 29, 2016 at 6:13 PM, David Brown <da...@gmail.com> wrote:
>>>>
>>>> > Wow Stephen thanks for all your hard work! This will really make
>>>> > driver development a lot easier.
>>>> >
>>>> > On Sun, Aug 28, 2016 at 6:48 PM, Stephen Mallette <sp...@gmail.com>
>>>> > wrote:
>>>> > > Took me half of my Sunday, but I just got Gremlin Server to start up
>>>> and
>>>> > > shutdown in the standard maven integration-test phase of
>>>> gremlin-python.
>>>> > > That means we can now write native python tests that round-trip to
>>>> > Gremlin
>>>> > > Server in addition to the regular unit tests with pytest we already
>>>> > have! I
>>>> > > actual Note that I actually start two gremlin servers one with auth on
>>>> > and
>>>> > > one without (ports 8183/8182 respectively) so that we can test
>>>> > > authentication features. Further note that native tests are bound to
>>>> the
>>>> > > integration-test phase of maven and run automatically on -glvPython.
>>>> > that's
>>>> > > a bit different from our other projects where integration test phases
>>>> > only
>>>> > > run if you -DskipIntegrationTests=false. gremlin-python won't obey that
>>>> > > setting.
>>>> > >
>>>> > >
>>>> > >
>>>> > > On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <
>>>> spmallette@gmail.com>
>>>> > > wrote:
>>>> > >
>>>> > >> Now that gremlin-python has been merged to master, you might wonder
>>>> > about
>>>> > >> what this has done to our build/release process. Well, not very much.
>>>> If
>>>> > >> you do your standard
>>>> > >>
>>>> > >> mvn clean install
>>>> > >>
>>>> > >> on master right now you should see that everything builds and is good
>>>> to
>>>> > >> go - even gremlin-python. So, everything is good, right? right? well,
>>>> > yes
>>>> > >> and no.
>>>> > >>
>>>> > >> The "yes" aspect here is that the entire project still builds with
>>>> maven
>>>> > >> which keeps our build toolchain simple. Users can just have java
>>>> > installed
>>>> > >> as they always did and still get a clean build of TinkerPop. The "no"
>>>> > >> aspect is that native python tests (and if you were deploy,
>>>> > >> packaging/deployment tasks) did not execute.
>>>> > >>
>>>> > >> What's good however is that even the native python build tasks are
>>>> still
>>>> > >> just part of the maven toolchain. You just need to have python 2.x
>>>> > >> installed and, if you do, build with:
>>>> > >>
>>>> > >> mvn clean install -DglvPython
>>>> > >>
>>>> > >> You will now see in your output the results of native pytest
>>>> execution.
>>>> > I
>>>> > >> think this approach almost sets the basic pattern for future GLVs. I'd
>>>> > >> prefer to not have -glvPython to some degree and simply detect python
>>>> on
>>>> > >> the system and then execute natively if it can, but then i think about
>>>> > what
>>>> > >> happens as we add more GLVs and then i sorta like the idea of having
>>>> the
>>>> > >> specific option to turn things on and off.
>>>> > >>
>>>> > >>
>>>> > >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > David M. Brown
>>>> > R.A. CulturePlex Lab, Western University
>>>> >
>>>>
>>
>>
>>
>> --
>> David M. Brown
>> R.A. CulturePlex Lab, Western University
>
>
>
> --
> David M. Brown
> R.A. CulturePlex Lab, Western University
--
David M. Brown
R.A. CulturePlex Lab, Western University
Re: Building gremlin-python
Posted by David Brown <da...@gmail.com>.
Ah wait, I guess you use it to install twine.
On Tue, Aug 30, 2016 at 5:58 PM, David Brown <da...@gmail.com> wrote:
> Nice work with this Stephen! One thing: if I am thinking about this
> correctly, pip isn't a requirement for building. I think we are only
> using pip inside the virtualenv, and since the virtualenv binary
> installs pip and setuptools by default when in creates the
> environment, I don't think we need it to be installed on the system.
> So basically, if you have Python(2) on on your machine, you only need
> to make sure that you install virtualenv before building...make sense?
> or am I missing something?
>
> On Tue, Aug 30, 2016 at 5:32 PM, Stephen Mallette <sp...@gmail.com> wrote:
>> With some help from Dave I've tweaked up the build for python some more -
>> the command to build is still:
>>
>> mvn clean install -pl gremlin-python -DglvPython
>>
>> but now:
>>
>> + The prereqs for building are just python 2.7, pip and virtualenv
>> + The build uses virtualenv to isolate the python environment to the
>> /target directories
>> + -DskipTests is now better respected by Gremlin Server - won't bother to
>> start it, if tests are skipped
>> + Minor adjustments to the deploy phase for python. We can't really deploy
>> to pypi on deploy in our release process because pypi has no staging
>> environment (like sonatype does) so the release would be immediately public
>> prior to vote on the release. So i made a special "-Dpypi" that activates a
>> profile that will hook into "mvn deploy" and push the artifacts to pypi
>> with twine.
>>
>> Note that these changes are all in the "virtualenv" branch at the moment. I
>> suspect I will merge it to master tomorrow after some more tests.
>>
>>
>>
>> On Mon, Aug 29, 2016 at 5:45 PM, Dylan Millikin <dy...@gmail.com>
>> wrote:
>>
>>> Yeah this is great. Will be useful for all GLVs.
>>>
>>> On Mon, Aug 29, 2016 at 6:13 PM, David Brown <da...@gmail.com> wrote:
>>>
>>> > Wow Stephen thanks for all your hard work! This will really make
>>> > driver development a lot easier.
>>> >
>>> > On Sun, Aug 28, 2016 at 6:48 PM, Stephen Mallette <sp...@gmail.com>
>>> > wrote:
>>> > > Took me half of my Sunday, but I just got Gremlin Server to start up
>>> and
>>> > > shutdown in the standard maven integration-test phase of
>>> gremlin-python.
>>> > > That means we can now write native python tests that round-trip to
>>> > Gremlin
>>> > > Server in addition to the regular unit tests with pytest we already
>>> > have! I
>>> > > actual Note that I actually start two gremlin servers one with auth on
>>> > and
>>> > > one without (ports 8183/8182 respectively) so that we can test
>>> > > authentication features. Further note that native tests are bound to
>>> the
>>> > > integration-test phase of maven and run automatically on -glvPython.
>>> > that's
>>> > > a bit different from our other projects where integration test phases
>>> > only
>>> > > run if you -DskipIntegrationTests=false. gremlin-python won't obey that
>>> > > setting.
>>> > >
>>> > >
>>> > >
>>> > > On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <
>>> spmallette@gmail.com>
>>> > > wrote:
>>> > >
>>> > >> Now that gremlin-python has been merged to master, you might wonder
>>> > about
>>> > >> what this has done to our build/release process. Well, not very much.
>>> If
>>> > >> you do your standard
>>> > >>
>>> > >> mvn clean install
>>> > >>
>>> > >> on master right now you should see that everything builds and is good
>>> to
>>> > >> go - even gremlin-python. So, everything is good, right? right? well,
>>> > yes
>>> > >> and no.
>>> > >>
>>> > >> The "yes" aspect here is that the entire project still builds with
>>> maven
>>> > >> which keeps our build toolchain simple. Users can just have java
>>> > installed
>>> > >> as they always did and still get a clean build of TinkerPop. The "no"
>>> > >> aspect is that native python tests (and if you were deploy,
>>> > >> packaging/deployment tasks) did not execute.
>>> > >>
>>> > >> What's good however is that even the native python build tasks are
>>> still
>>> > >> just part of the maven toolchain. You just need to have python 2.x
>>> > >> installed and, if you do, build with:
>>> > >>
>>> > >> mvn clean install -DglvPython
>>> > >>
>>> > >> You will now see in your output the results of native pytest
>>> execution.
>>> > I
>>> > >> think this approach almost sets the basic pattern for future GLVs. I'd
>>> > >> prefer to not have -glvPython to some degree and simply detect python
>>> on
>>> > >> the system and then execute natively if it can, but then i think about
>>> > what
>>> > >> happens as we add more GLVs and then i sorta like the idea of having
>>> the
>>> > >> specific option to turn things on and off.
>>> > >>
>>> > >>
>>> > >>
>>> >
>>> >
>>> >
>>> > --
>>> > David M. Brown
>>> > R.A. CulturePlex Lab, Western University
>>> >
>>>
>
>
>
> --
> David M. Brown
> R.A. CulturePlex Lab, Western University
--
David M. Brown
R.A. CulturePlex Lab, Western University
Re: Building gremlin-python
Posted by David Brown <da...@gmail.com>.
Nice work with this Stephen! One thing: if I am thinking about this
correctly, pip isn't a requirement for building. I think we are only
using pip inside the virtualenv, and since the virtualenv binary
installs pip and setuptools by default when in creates the
environment, I don't think we need it to be installed on the system.
So basically, if you have Python(2) on on your machine, you only need
to make sure that you install virtualenv before building...make sense?
or am I missing something?
On Tue, Aug 30, 2016 at 5:32 PM, Stephen Mallette <sp...@gmail.com> wrote:
> With some help from Dave I've tweaked up the build for python some more -
> the command to build is still:
>
> mvn clean install -pl gremlin-python -DglvPython
>
> but now:
>
> + The prereqs for building are just python 2.7, pip and virtualenv
> + The build uses virtualenv to isolate the python environment to the
> /target directories
> + -DskipTests is now better respected by Gremlin Server - won't bother to
> start it, if tests are skipped
> + Minor adjustments to the deploy phase for python. We can't really deploy
> to pypi on deploy in our release process because pypi has no staging
> environment (like sonatype does) so the release would be immediately public
> prior to vote on the release. So i made a special "-Dpypi" that activates a
> profile that will hook into "mvn deploy" and push the artifacts to pypi
> with twine.
>
> Note that these changes are all in the "virtualenv" branch at the moment. I
> suspect I will merge it to master tomorrow after some more tests.
>
>
>
> On Mon, Aug 29, 2016 at 5:45 PM, Dylan Millikin <dy...@gmail.com>
> wrote:
>
>> Yeah this is great. Will be useful for all GLVs.
>>
>> On Mon, Aug 29, 2016 at 6:13 PM, David Brown <da...@gmail.com> wrote:
>>
>> > Wow Stephen thanks for all your hard work! This will really make
>> > driver development a lot easier.
>> >
>> > On Sun, Aug 28, 2016 at 6:48 PM, Stephen Mallette <sp...@gmail.com>
>> > wrote:
>> > > Took me half of my Sunday, but I just got Gremlin Server to start up
>> and
>> > > shutdown in the standard maven integration-test phase of
>> gremlin-python.
>> > > That means we can now write native python tests that round-trip to
>> > Gremlin
>> > > Server in addition to the regular unit tests with pytest we already
>> > have! I
>> > > actual Note that I actually start two gremlin servers one with auth on
>> > and
>> > > one without (ports 8183/8182 respectively) so that we can test
>> > > authentication features. Further note that native tests are bound to
>> the
>> > > integration-test phase of maven and run automatically on -glvPython.
>> > that's
>> > > a bit different from our other projects where integration test phases
>> > only
>> > > run if you -DskipIntegrationTests=false. gremlin-python won't obey that
>> > > setting.
>> > >
>> > >
>> > >
>> > > On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <
>> spmallette@gmail.com>
>> > > wrote:
>> > >
>> > >> Now that gremlin-python has been merged to master, you might wonder
>> > about
>> > >> what this has done to our build/release process. Well, not very much.
>> If
>> > >> you do your standard
>> > >>
>> > >> mvn clean install
>> > >>
>> > >> on master right now you should see that everything builds and is good
>> to
>> > >> go - even gremlin-python. So, everything is good, right? right? well,
>> > yes
>> > >> and no.
>> > >>
>> > >> The "yes" aspect here is that the entire project still builds with
>> maven
>> > >> which keeps our build toolchain simple. Users can just have java
>> > installed
>> > >> as they always did and still get a clean build of TinkerPop. The "no"
>> > >> aspect is that native python tests (and if you were deploy,
>> > >> packaging/deployment tasks) did not execute.
>> > >>
>> > >> What's good however is that even the native python build tasks are
>> still
>> > >> just part of the maven toolchain. You just need to have python 2.x
>> > >> installed and, if you do, build with:
>> > >>
>> > >> mvn clean install -DglvPython
>> > >>
>> > >> You will now see in your output the results of native pytest
>> execution.
>> > I
>> > >> think this approach almost sets the basic pattern for future GLVs. I'd
>> > >> prefer to not have -glvPython to some degree and simply detect python
>> on
>> > >> the system and then execute natively if it can, but then i think about
>> > what
>> > >> happens as we add more GLVs and then i sorta like the idea of having
>> the
>> > >> specific option to turn things on and off.
>> > >>
>> > >>
>> > >>
>> >
>> >
>> >
>> > --
>> > David M. Brown
>> > R.A. CulturePlex Lab, Western University
>> >
>>
--
David M. Brown
R.A. CulturePlex Lab, Western University
Re: Building gremlin-python
Posted by Stephen Mallette <sp...@gmail.com>.
With some help from Dave I've tweaked up the build for python some more -
the command to build is still:
mvn clean install -pl gremlin-python -DglvPython
but now:
+ The prereqs for building are just python 2.7, pip and virtualenv
+ The build uses virtualenv to isolate the python environment to the
/target directories
+ -DskipTests is now better respected by Gremlin Server - won't bother to
start it, if tests are skipped
+ Minor adjustments to the deploy phase for python. We can't really deploy
to pypi on deploy in our release process because pypi has no staging
environment (like sonatype does) so the release would be immediately public
prior to vote on the release. So i made a special "-Dpypi" that activates a
profile that will hook into "mvn deploy" and push the artifacts to pypi
with twine.
Note that these changes are all in the "virtualenv" branch at the moment. I
suspect I will merge it to master tomorrow after some more tests.
On Mon, Aug 29, 2016 at 5:45 PM, Dylan Millikin <dy...@gmail.com>
wrote:
> Yeah this is great. Will be useful for all GLVs.
>
> On Mon, Aug 29, 2016 at 6:13 PM, David Brown <da...@gmail.com> wrote:
>
> > Wow Stephen thanks for all your hard work! This will really make
> > driver development a lot easier.
> >
> > On Sun, Aug 28, 2016 at 6:48 PM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> > > Took me half of my Sunday, but I just got Gremlin Server to start up
> and
> > > shutdown in the standard maven integration-test phase of
> gremlin-python.
> > > That means we can now write native python tests that round-trip to
> > Gremlin
> > > Server in addition to the regular unit tests with pytest we already
> > have! I
> > > actual Note that I actually start two gremlin servers one with auth on
> > and
> > > one without (ports 8183/8182 respectively) so that we can test
> > > authentication features. Further note that native tests are bound to
> the
> > > integration-test phase of maven and run automatically on -glvPython.
> > that's
> > > a bit different from our other projects where integration test phases
> > only
> > > run if you -DskipIntegrationTests=false. gremlin-python won't obey that
> > > setting.
> > >
> > >
> > >
> > > On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <
> spmallette@gmail.com>
> > > wrote:
> > >
> > >> Now that gremlin-python has been merged to master, you might wonder
> > about
> > >> what this has done to our build/release process. Well, not very much.
> If
> > >> you do your standard
> > >>
> > >> mvn clean install
> > >>
> > >> on master right now you should see that everything builds and is good
> to
> > >> go - even gremlin-python. So, everything is good, right? right? well,
> > yes
> > >> and no.
> > >>
> > >> The "yes" aspect here is that the entire project still builds with
> maven
> > >> which keeps our build toolchain simple. Users can just have java
> > installed
> > >> as they always did and still get a clean build of TinkerPop. The "no"
> > >> aspect is that native python tests (and if you were deploy,
> > >> packaging/deployment tasks) did not execute.
> > >>
> > >> What's good however is that even the native python build tasks are
> still
> > >> just part of the maven toolchain. You just need to have python 2.x
> > >> installed and, if you do, build with:
> > >>
> > >> mvn clean install -DglvPython
> > >>
> > >> You will now see in your output the results of native pytest
> execution.
> > I
> > >> think this approach almost sets the basic pattern for future GLVs. I'd
> > >> prefer to not have -glvPython to some degree and simply detect python
> on
> > >> the system and then execute natively if it can, but then i think about
> > what
> > >> happens as we add more GLVs and then i sorta like the idea of having
> the
> > >> specific option to turn things on and off.
> > >>
> > >>
> > >>
> >
> >
> >
> > --
> > David M. Brown
> > R.A. CulturePlex Lab, Western University
> >
>
Re: Building gremlin-python
Posted by Dylan Millikin <dy...@gmail.com>.
Yeah this is great. Will be useful for all GLVs.
On Mon, Aug 29, 2016 at 6:13 PM, David Brown <da...@gmail.com> wrote:
> Wow Stephen thanks for all your hard work! This will really make
> driver development a lot easier.
>
> On Sun, Aug 28, 2016 at 6:48 PM, Stephen Mallette <sp...@gmail.com>
> wrote:
> > Took me half of my Sunday, but I just got Gremlin Server to start up and
> > shutdown in the standard maven integration-test phase of gremlin-python.
> > That means we can now write native python tests that round-trip to
> Gremlin
> > Server in addition to the regular unit tests with pytest we already
> have! I
> > actual Note that I actually start two gremlin servers one with auth on
> and
> > one without (ports 8183/8182 respectively) so that we can test
> > authentication features. Further note that native tests are bound to the
> > integration-test phase of maven and run automatically on -glvPython.
> that's
> > a bit different from our other projects where integration test phases
> only
> > run if you -DskipIntegrationTests=false. gremlin-python won't obey that
> > setting.
> >
> >
> >
> > On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <sp...@gmail.com>
> > wrote:
> >
> >> Now that gremlin-python has been merged to master, you might wonder
> about
> >> what this has done to our build/release process. Well, not very much. If
> >> you do your standard
> >>
> >> mvn clean install
> >>
> >> on master right now you should see that everything builds and is good to
> >> go - even gremlin-python. So, everything is good, right? right? well,
> yes
> >> and no.
> >>
> >> The "yes" aspect here is that the entire project still builds with maven
> >> which keeps our build toolchain simple. Users can just have java
> installed
> >> as they always did and still get a clean build of TinkerPop. The "no"
> >> aspect is that native python tests (and if you were deploy,
> >> packaging/deployment tasks) did not execute.
> >>
> >> What's good however is that even the native python build tasks are still
> >> just part of the maven toolchain. You just need to have python 2.x
> >> installed and, if you do, build with:
> >>
> >> mvn clean install -DglvPython
> >>
> >> You will now see in your output the results of native pytest execution.
> I
> >> think this approach almost sets the basic pattern for future GLVs. I'd
> >> prefer to not have -glvPython to some degree and simply detect python on
> >> the system and then execute natively if it can, but then i think about
> what
> >> happens as we add more GLVs and then i sorta like the idea of having the
> >> specific option to turn things on and off.
> >>
> >>
> >>
>
>
>
> --
> David M. Brown
> R.A. CulturePlex Lab, Western University
>
Re: Building gremlin-python
Posted by David Brown <da...@gmail.com>.
Wow Stephen thanks for all your hard work! This will really make
driver development a lot easier.
On Sun, Aug 28, 2016 at 6:48 PM, Stephen Mallette <sp...@gmail.com> wrote:
> Took me half of my Sunday, but I just got Gremlin Server to start up and
> shutdown in the standard maven integration-test phase of gremlin-python.
> That means we can now write native python tests that round-trip to Gremlin
> Server in addition to the regular unit tests with pytest we already have! I
> actual Note that I actually start two gremlin servers one with auth on and
> one without (ports 8183/8182 respectively) so that we can test
> authentication features. Further note that native tests are bound to the
> integration-test phase of maven and run automatically on -glvPython. that's
> a bit different from our other projects where integration test phases only
> run if you -DskipIntegrationTests=false. gremlin-python won't obey that
> setting.
>
>
>
> On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <sp...@gmail.com>
> wrote:
>
>> Now that gremlin-python has been merged to master, you might wonder about
>> what this has done to our build/release process. Well, not very much. If
>> you do your standard
>>
>> mvn clean install
>>
>> on master right now you should see that everything builds and is good to
>> go - even gremlin-python. So, everything is good, right? right? well, yes
>> and no.
>>
>> The "yes" aspect here is that the entire project still builds with maven
>> which keeps our build toolchain simple. Users can just have java installed
>> as they always did and still get a clean build of TinkerPop. The "no"
>> aspect is that native python tests (and if you were deploy,
>> packaging/deployment tasks) did not execute.
>>
>> What's good however is that even the native python build tasks are still
>> just part of the maven toolchain. You just need to have python 2.x
>> installed and, if you do, build with:
>>
>> mvn clean install -DglvPython
>>
>> You will now see in your output the results of native pytest execution. I
>> think this approach almost sets the basic pattern for future GLVs. I'd
>> prefer to not have -glvPython to some degree and simply detect python on
>> the system and then execute natively if it can, but then i think about what
>> happens as we add more GLVs and then i sorta like the idea of having the
>> specific option to turn things on and off.
>>
>>
>>
--
David M. Brown
R.A. CulturePlex Lab, Western University
Re: Building gremlin-python
Posted by Stephen Mallette <sp...@gmail.com>.
Took me half of my Sunday, but I just got Gremlin Server to start up and
shutdown in the standard maven integration-test phase of gremlin-python.
That means we can now write native python tests that round-trip to Gremlin
Server in addition to the regular unit tests with pytest we already have! I
actual Note that I actually start two gremlin servers one with auth on and
one without (ports 8183/8182 respectively) so that we can test
authentication features. Further note that native tests are bound to the
integration-test phase of maven and run automatically on -glvPython. that's
a bit different from our other projects where integration test phases only
run if you -DskipIntegrationTests=false. gremlin-python won't obey that
setting.
On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <sp...@gmail.com>
wrote:
> Now that gremlin-python has been merged to master, you might wonder about
> what this has done to our build/release process. Well, not very much. If
> you do your standard
>
> mvn clean install
>
> on master right now you should see that everything builds and is good to
> go - even gremlin-python. So, everything is good, right? right? well, yes
> and no.
>
> The "yes" aspect here is that the entire project still builds with maven
> which keeps our build toolchain simple. Users can just have java installed
> as they always did and still get a clean build of TinkerPop. The "no"
> aspect is that native python tests (and if you were deploy,
> packaging/deployment tasks) did not execute.
>
> What's good however is that even the native python build tasks are still
> just part of the maven toolchain. You just need to have python 2.x
> installed and, if you do, build with:
>
> mvn clean install -DglvPython
>
> You will now see in your output the results of native pytest execution. I
> think this approach almost sets the basic pattern for future GLVs. I'd
> prefer to not have -glvPython to some degree and simply detect python on
> the system and then execute natively if it can, but then i think about what
> happens as we add more GLVs and then i sorta like the idea of having the
> specific option to turn things on and off.
>
>
>